Hyperspectral remote sensing image unmixing method and system under mixed noise condition

Abstract

The application provides a hyperspectral remote sensing image unmixing method and system under mixed noise conditions, which can fully extract spatial-spectral features, improve the unmixing effect, and reduce the training amount. The method comprises the following steps: step 1, obtaining a hyperspectral image as a training sample; step 2, building a spatial-spectral multi-scale unmixing network model; the multi-scale asymmetric attention unit comprises a three-dimensional multi-scale asymmetric attention unit and a two-dimensional multi-scale asymmetric attention unit; a multi-scale asymmetric convolution module adopts a joint operation to combine the spatial-spectral feature maps output by convolution layers C2, C4 and C6; the module further comprises a three-dimensional convolution layer C7 arranged in series, which is used for fusing the spatial-spectral features output by C2, C4 and C6; step 3, inputting the training sample into the spatial-spectral multi-scale unmixing network model for training; step 4, obtaining a hyperspectral image to be unixed; and step 5, inputting the hyperspectral image to be unixed into the trained model to obtain a hyperspectral remote sensing image unmixing result.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of remote sensing image processing, and particularly relates to a hyperspectral remote sensing image unmixing method and system under mixed noise conditions. BACKGROUND

[0002] Hyperspectral remote sensing technology can effectively distinguish objects that cannot be detected by traditional multispectral remote sensing technology, making the application of remote sensing technology more precise. Hyperspectral sensors can not only use imagers for ground imaging, but also obtain rich spectral information. However, due to the limitation of spatial resolution and the uniform mixing of different materials, the pixels in hyperspectral images are often mixed (Document 1). In addition, hyperspectral images are often damaged by certain types of noise, such as Gaussian noise, impulse noise, shot noise, horizontal or vertical stripes, etc. Therefore, it is of great significance to study the hyperspectral remote sensing image unmixing method under mixed noise conditions to improve the quantitative application of remote sensing and sub-pixel level target detection (Document 2). At present, deep learning is concerned by researchers and is introduced into the field of hyperspectral image unmixing (Document 3). However, under mixed noise conditions, the deep learning-based method often leads to unsatisfactory unmixing results due to insufficient spatial-spectral feature extraction, which undoubtedly limits the application range of hyperspectral technology.

[0003] [1] Zhuang L, Ng M K. Hyperspectral mixed noise removal by L1-based subspace representation [J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 13: 1143-1157.

[0004] [2] Zheng Y B, Huang T Z, Zhao X L, et al. Double-factor-regularized low-rank tensor factorization for mixed noise removal in hyperspectral image [J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(12): 8450-8464.

[0005] [3]Zhao M, Yan L, Chen J. LSTM-DNN based autoencoder network for nonlinear hyperspectral image unmixing[J]. IEEE Journal of Selected Topics in Signal Processing, 2021, 15(2): 295-309. SUMMARY

[0006] The present application is to solve the above problems, and aims to provide a hyperspectral remote sensing image unmixing method and system under mixed noise conditions, which can fully extract spatial and spectral features, improve the unmixing effect, and reduce the training amount.

[0007] In order to achieve the above-mentioned purpose, the present application adopts the following scheme:

[0008] <Method>

[0009] The present application provides a hyperspectral remote sensing image unmixing method under mixed noise conditions, characterized by comprising the following steps:

[0010] Step 1, obtaining a hyperspectral image as a training sample;

[0011] Step 2, building a spatial-spectral multi-scale unmixing network model:

[0012] The multiscale asymmetric attention unit is constructed to build a spatial-spectral multiscale unmixing network model; the multiscale asymmetric attention unit is formed by connecting a multiscale asymmetric convolution module and a spatial-spectral attention module in series; the multiscale asymmetric attention unit includes a three-dimensional multiscale asymmetric attention unit and a two-dimensional multiscale asymmetric attention unit; the three-dimensional multiscale asymmetric convolution unit includes three three-dimensional convolution layers C1-C2, C3-C4 and C5-C6 arranged in parallel; C1 and C2 are convolution layers arranged in series, the convolution kernel sizes are 1*3*3 and 3*1*1 respectively, and the step sizes are both 1; C3 and C4 are convolution layers arranged in series, the convolution kernel sizes are 1*5*5 and 5*1*1 respectively, and the step sizes are both 1; C5 and C6 are convolution layers arranged in series, the convolution kernel sizes are 1*7*7 and 7*1*1 respectively, and the step sizes are both 1; the multiscale asymmetric convolution module combines the spatial-spectral feature maps output by the convolution layers C2, C4 and C6 through joint operation; the multiscale asymmetric convolution module further includes a three-dimensional convolution layer C7 arranged in series with the output spatial-spectral feature maps, which is used for fusing the spatial-spectral features output by C2, C4 and C6; C7 is a convolution layer with a convolution kernel size of 1 and a step size of 1; normalization layers and rectified linear unit activation function layers are added after the three-dimensional convolution layers C1-C7; the difference between the two-dimensional multiscale asymmetric attention unit and the three-dimensional multiscale asymmetric attention unit is only that the convolution kernel size is reduced from three-dimensional to two-dimensional.

[0013] Step 3: inputting the training sample into the spatial-spectral multiscale unmixing network model for training;

[0014] Step 4: obtaining a hyperspectral image to be unixed;

[0015] Step 5: inputting the hyperspectral image to be unixed into the trained spatial-spectral multiscale unmixing network model to obtain a hyperspectral remote sensing image unmixing result.

[0016] Preferably, the hyperspectral remote sensing image unmixing method under mixed noise conditions provided by the application, in step 2, the spatial-spectral multiscale unmixing network model is established based on a three-dimensional and two-dimensional mixed network, including convolution layers C0 and C8, three-dimensional and two-dimensional multiscale asymmetric attention units, two residual modules and a full connection module;

[0017] In the spectral multi-scale unmixing network model: the input of the convolutional layer C0 is the high-resolution remote sensing image after blocking, C0 is a three-dimensional convolutional layer with a convolution kernel size of 3 and a step size of 1; the input of the first multi-scale asymmetric attention unit is the output feature F0 of C0; the input of the first residual module is the output feature F0 of C0; the input of the convolutional layer C8 is the feature after the residual fusion of the features F0 and the output feature F1 of the first multi-scale asymmetric attention unit and the matrix transformation operation, C8 is a two-dimensional convolutional layer with a convolution kernel size of 1 and a step size of 1; the matrix transformation operation converts the three-dimensional spectral matrix after the residual fusion of F0 and F1 into a two-dimensional spectral matrix; the input of the second multi-scale asymmetric attention unit is the output feature F2 of C8; the second multi-scale asymmetric attention unit is based on a two-dimensional multi-scale asymmetric convolution module and is a two-dimensional multi-scale asymmetric attention unit; the input of the second residual module is the output feature F2 of C8; the input of the full connection module is the feature after the residual fusion of F2 and the input feature of the second multi-scale asymmetric attention unit; a normalization layer and a rectified linear unit activation function layer are added after the convolutional layers C0 and C8.

[0018] The generalization ability of the model is further improved and the overfitting phenomenon is alleviated by introducing the residual cross-layer connection.

[0019] Preferably, the method for unmixing high-resolution remote sensing images under mixed noise conditions provided by the application, in step 2, the spectral attention module comprises channel attention and spatial attention; let the input feature be F, then the calculation formula of the channel attention CA(F) is:

[0020]

[0021] In the formula, Relu() and Sigmoid() are nonlinear activation functions, W1 and W0 are the weights after average pooling and maximum pooling respectively, and F1 and F0 are the feature descriptors obtained by average pooling and maximum pooling in the spatial dimension respectively, and the output feature F' is obtained by matrix multiplication of the channel attention CA(F) and the input feature F;

[0022] The output feature F' is introduced into the spatial attention SA(F), and the calculation formula is:

[0023]

[0024] In the formula, and F1' and F0' are the feature descriptors obtained by average pooling and maximum pooling in the channel direction respectively, and the output feature F'' is obtained by matrix multiplication of the channel attention SA(F) and the input feature F';

[0025] The channel dimension attention and the spatial dimension attention are concatenated to form a space-spectrum attention module.

[0026] Preferably, the hyperspectral remote sensing image unmixing method under mixed noise conditions provided by the present application adopts a gradient descent algorithm to reversely update the model parameters after the training samples are input into the space-spectrum multi-scale unmixing network model in step 3; a cross-entropy loss function is used to optimize the model by using the gradient descent algorithm with an adaptive learning rate, the learning rate is initially set to 0.001, and the iteration number is set to 200.

[0027] Preferably, the hyperspectral remote sensing image unmixing method under mixed noise conditions provided by the present application adopts the following method to obtain the training samples in step 1: for the input hyperspectral remote sensing image, first, data preprocessing is performed, including geometric correction and atmospheric correction, to obtain a preprocessed hyperspectral remote sensing image; the preprocessed hyperspectral remote sensing image is added with mixed noise to make a mixed noise hyperspectral remote sensing image data set, and a blocking operation is performed; training samples are obtained from each type of surface feature in the blocked mixed noise hyperspectral remote sensing image data set in a proportional manner.

[0028] Preferably, the hyperspectral remote sensing image unmixing method under mixed noise conditions provided by the present application performs the same preprocessing on the obtained hyperspectral image as in step 1 to obtain a hyperspectral image to be unixed in step 4.

[0029] <system>

[0030] Further, the present application also provides a hyperspectral remote sensing image unmixing system under mixed noise conditions capable of automatically implementing the above <method>, characterized in that it comprises:

[0031] a sample acquisition unit for acquiring a hyperspectral image as a training sample;

[0032] The model building department constructs a spatial-spectral multi-scale unmixing network model: It builds multi-scale asymmetric attention units (MSUs) to construct the MSUs. Each MSU is formed by connecting MSUs and MSUs in series. The MSUs include both 3D and 2D MSUs. The 3D MSUs consist of three parallel 3D convolutional layers C1-C2, C3-C4, and C5-C6. C1 and C2 are convolutional layers with kernel sizes of 1×3×3 and 3×1×1, respectively, and a stride of 1. C3 and C4 are convolutional layers with kernel sizes of 1×5×5 and 5×1×1, respectively. The stride is 1 for all three convolutional layers. C5 and C6 are concatenated convolutional layers with kernel sizes of 1×7×7 and 7×1×1, respectively, and both have a stride of 1. The multi-scale asymmetric convolution module uses a joint operation to merge the spatial spectral feature maps output by convolutional layers C2, C4, and C6. The multi-scale asymmetric convolution module also includes a three-dimensional convolutional layer C7, which is concatenated with the output spatial spectral feature maps to fuse the spatial spectral features output by C2, C4, and C6. C7 is a convolutional layer with a kernel size of 1 and a stride of 1. After the three-dimensional convolutional layers C1-C7, normalization layers and modified linear unit activation function layers are added. The difference between the two-dimensional and three-dimensional multi-scale asymmetric attention units is only that the convolutional kernel is reduced from three-dimensional to two-dimensional.

[0033] The training department inputs training samples into the spatial-spectral multi-scale unmixing network model for training;

[0034] The image acquisition unit acquires the hyperspectral image to be unmixed.

[0035] The unmixing unit inputs the hyperspectral image to be unmixed into the trained spatial-spectral multi-scale unmixing network model to obtain the unmixing result of the hyperspectral remote sensing image.

[0036] The control unit is communicatively connected to the sample acquisition unit, model building unit, training unit, image acquisition unit, and demixing unit, and controls their operation.

[0037] Preferably, the hyperspectral remote sensing image demixing system under mixed noise conditions provided by the present invention may further include: an input display unit, which is communicatively connected to the control unit, allowing the operator to input operation commands and displaying corresponding information according to the control commands.

[0038] Preferably, the hyperspectral remote sensing image unmixing system under mixed noise conditions provided by the present application, in the model building part, the spatial-spectral multi-scale unmixing network model is established based on a three-dimensional and two-dimensional mixed network, including convolution layers C0 and C8, three-dimensional and two-dimensional multi-scale asymmetric attention units, two residual modules and a full connection module; in the spatial-spectral multi-scale unmixing network model: the input of the convolution layer C0 is the blocked hyperspectral remote sensing image, C0 is a three-dimensional convolution layer with a convolution kernel size of 3, and the step is 1; the input of the first multi-scale asymmetric attention unit is the output feature F0 of C0; the input of the first residual module is the output feature F0 of C0; the input of the convolution layer C8 is the feature after the residual fusion of the feature F0 and the output feature F1 of the first multi-scale asymmetric attention unit and the matrix transformation operation, C8 is a two-dimensional convolution layer with a convolution kernel size of 1, and the step is 1; the matrix transformation operation converts the three-dimensional spatial-spectral matrix after the residual fusion of F0 and F1 into a two-dimensional spatial-spectral matrix; the input of the second multi-scale asymmetric attention unit is the output feature F2 of C8; the second multi-scale asymmetric attention unit is constructed based on a two-dimensional multi-scale asymmetric convolution module, which is a two-dimensional multi-scale asymmetric attention unit; the input of the second residual module is the output feature F2 of C8; the input of the full connection module is the feature after the residual fusion of F2 and the input feature of the second multi-scale asymmetric attention unit; the normalization layer and the rectified linear unit activation function layer are added after the convolution layers C0 and C8.

[0039] Preferably, the hyperspectral remote sensing image unmixing system under mixed noise conditions provided by the present application, in the model building part, the spatial-spectral attention module includes channel attention and spatial attention; let the input feature be F, then the calculation formula of the channel attention CA(F) is:

[0040]

[0041] In the formula, Relu() and Sigmoid() are nonlinear activation functions, W1 and W0 are weights after average pooling and maximum pooling respectively, and The channel attention CA(F) and the input feature F are multiplied by a matrix to obtain the output feature F′;

[0042] The output feature F′ is introduced into the spatial attention SA(F), and the calculation formula is:

[0043]

[0044] In the formula, and The channel attention SA(F) is multiplied with the input feature F' to obtain an output feature F'' by matrix multiplication;

[0045] The channel dimension attention and the spatial dimension attention are connected in series to form a spectral attention module.

[0046] Effects of the application

[0047] The application relates to a hyperspectral remote sensing image unmixing method and system under mixed noise conditions, a multiscale asymmetric attention unit is constructed, and a spectral multiscale unmixing network model is built; the multiscale asymmetric attention unit is formed by connecting a multiscale asymmetric convolution module and a spectral attention module in series; the multiscale asymmetric attention unit comprises a three-dimensional multiscale asymmetric attention unit and a two-dimensional multiscale asymmetric attention unit; the three-dimensional multiscale asymmetric convolution unit comprises three three-dimensional convolution layers C1-C2, C3-C4 and C5-C6 which are arranged in parallel; C1 and C2 are convolution layers which are arranged in series, the convolution kernels are 1*3*3 and 3*1*1 respectively, and the steps are both 1; C3 and C4 are convolution layers which are arranged in series, the convolution kernel sizes are 1*5*5 and 5*1*1 respectively, and the steps are both 1; C5 and C6 are convolution layers which are arranged in series, the convolution kernel sizes are 1*7*7 and 7*1*1 respectively, and the steps are both 1; the multiscale asymmetric convolution module adopts a joint operation to combine the spectral feature maps output by the convolution layers C2, C4 and C6; the multiscale asymmetric convolution module further comprises a three-dimensional convolution layer C7 which is arranged in series with the output spectral feature maps and is used for fusing the spectral features output by C2, C4 and C6; C7 is a convolution layer with a convolution kernel size of 1 and a step of 1; a normalization layer and a rectified linear unit activation function layer are added after the three-dimensional convolution layers C1-C7; the two-dimensional multiscale asymmetric attention unit has the same structure as the three-dimensional multiscale asymmetric attention unit, and the only difference is that the convolution kernel is reduced from three-dimensional to two-dimensional; the special spectral multiscale unmixing network model established by the asymmetric convolution and the three-dimensional and two-dimensional mixed network can deepen the network layers while requiring fewer training parameters, can fully extract spectral features, can improve the generalization ability, can relieve the overfitting phenomenon, and can effectively improve the unmixing effect of the model and the unmixing efficiency under mixed noise conditions. The application solves the defect that the spectral feature extraction of the model under mixed noise conditions is insufficient, the hyperspectral remote sensing image unmixing effect is not ideal, and can be widely applied to the hyperspectral data mixed pixel decomposition application in the fields of environmental monitoring, precision agriculture and food safety. BRIEF DESCRIPTION OF DRAWINGS

[0048] Figure 1 A flowchart of the hyperspectral remote sensing image unmixing method under mixed noise conditions according to the embodiments of the application;

[0049] Figure 2A structure diagram of a spatial-spectral multi-scale unmixing network model related to an embodiment of the present application. DETAILED DESCRIPTION

[0050] The specific implementation of the hyperspectral remote sensing image unmixing method and system under mixed noise conditions related to the present application will be described in detail below with reference to the accompanying drawings.

[0051] Embodiment I

[0052] As shown in the figure, the hyperspectral remote sensing image unmixing method under mixed noise conditions related to the present embodiment I includes the following steps: Figure 1

[0053] Step 1, obtaining training samples:

[0054] (1) For the input hyperspectral remote sensing image, first perform data preprocessing, mainly including geometric correction, atmospheric correction, etc. This part of work will be realized based on existing research, and the preprocessed hyperspectral remote sensing image is obtained.

[0055] (2) Add mixed noise to the preprocessed hyperspectral remote sensing image to make a mixed noise hyperspectral remote sensing image dataset, and perform block operation.

[0056] (3) Obtain training samples from each type of surface feature in the mixed noise hyperspectral remote sensing image dataset after block operation in proportion.

[0057] Step 2, establishing a spatial-spectral multi-scale unmixing network model; based on the mixed network, an end-to-end spatial-spectral multi-scale unmixing network is established.

[0058] As shown in the figure, the spatial-spectral multi-scale unmixing network model of the present embodiment includes a multi-scale asymmetric convolution module, a spatial-spectral attention module, a multi-scale asymmetric attention unit, a residual module, and a full connection module. Figure 2

[0059] ​​The multi-scale asymmetric convolution module includes three three-dimensional convolution layers C1-C2, C3-C4 and C5-C6 arranged in parallel, C1 and C2 are convolution layers with convolution kernel sizes of 1*3*3 and 3*1*1 arranged in series, and the step lengths are both 1; C3 and C4 are convolution layers with convolution kernel sizes of 1*5*5 and 5*1*1 arranged in series, and the step lengths are both 1; C5 and C6 are convolution layers with convolution kernel sizes of 1*7*7 and 7*1*1 arranged in series, and the step lengths are both 1; the hyperspectral multi-scale unmixing network merges the hyperspectral feature maps output by the convolution layers C2, C4 and C6 by joint operation; the multi-scale hyperspectral feature extraction module includes a three-dimensional convolution layer C7 arranged in series, which is used for fusing the hyperspectral features output by C2, C4 and C6, C7 is a convolution layer with a convolution kernel size of 1 and a step length of 1; normalization layers and rectified linear unit activation function layers are added after the three-dimensional convolution layers C1-C7;

[0060] The hyperspectral attention module is used for reassigning weights to the feature maps output by the three-dimensional convolution layer C7, and includes a channel attention module and a spatial attention module; the channel attention module and the spatial attention module are arranged in series at the beginning and the end; the hyperspectral attention module includes channel dimension attention and spatial dimension attention, reassigns weights, and further refines the feature maps; assuming that the input feature is F, the calculation formula of the channel attention CA(F) is:

[0061]

[0062] In the formula, Relu() and Sigmoid() are nonlinear activation functions, W1 and W0 are weights after average pooling and maximum pooling respectively, and are feature descriptors obtained by average pooling and maximum pooling in the spatial dimension respectively, the channel attention CA(F) is multiplied by the input feature F to obtain the output feature F′:

[0063]

[0064] In the formula, is a matrix multiplication operation.

[0065] The output feature F′ is introduced into the spatial dimension attention SA(F), and the mathematical calculation formula is:

[0066]

[0067] In the formula, and are feature descriptors obtained by average pooling and maximum pooling in the channel direction respectively, the channel attention SA(F) is multiplied by the input feature F′ to obtain the output feature F′′:

[0068]

[0069] The channel dimension attention and the spatial dimension attention are concatenated to establish a space spectrum attention module.

[0070] The multi-scale asymmetric attention unit is generated in series by a multi-scale asymmetric convolution module and the space spectrum attention module; the multi-scale asymmetric attention unit includes a three-dimensional multi-scale asymmetric attention unit and a two-dimensional multi-scale asymmetric attention unit; the two-dimensional multi-scale asymmetric attention unit is similar to the three-dimensional multi-scale asymmetric attention unit, and the difference lies in that the convolution kernel is reduced from three dimensions to two dimensions.

[0071] The residual module makes the input features of the asymmetric convolution module and the output features of the space spectrum attention module in the same scale through a convolution kernel with a size of 1 and a normalization layer, and performs residual feature fusion.

[0072] The full connection module includes one unfolding layer and three full connection layers arranged in series. A Dropout layer is arranged after the first two full connection layers of the three full connection layers, and the third full connection layer is attached with a Softmax classification function to obtain the abundance estimation of the hyperspectral remote sensing image.

[0073] Specifically, the space spectrum multi-scale unmixing network model is established based on a three-dimensional and two-dimensional hybrid network, mainly composed of convolution layers C0 and C8, two multi-scale asymmetric attention units, two residual modules and a full connection module; the input of the convolution layer C0 is the blocked hyperspectral remote sensing image, C0 is a three-dimensional convolution layer with a convolution kernel size of 3 and a step size of 1; the input of the first multi-scale asymmetric attention unit in the space spectrum multi-scale unmixing network is the output feature F0 of C0; the input of the first residual module in the space spectrum multi-scale unmixing network is the output feature F0 of C0; the input of the convolution layer C8 is the feature after the residual fusion of the features F0 and the output feature F1 of the first multi-scale asymmetric attention unit and the matrix transformation operation, C8 is a two-dimensional convolution layer with a convolution kernel size of 1 and a step size of 1; the matrix transformation operation converts the three-dimensional space spectrum matrix after the residual fusion of F0 and F1 into a two-dimensional space spectrum matrix; the input of the second multi-scale asymmetric attention unit in the space spectrum multi-scale unmixing network is the output feature F2 of C8; the second multi-scale asymmetric attention unit is constructed based on a two-dimensional multi-scale asymmetric convolution module; the input of the second residual module in the space spectrum multi-scale unmixing network is the output feature F2 of C8; the input of the full connection module in the space spectrum multi-scale unmixing network is the feature after the residual fusion of F2 and the input feature of the second multi-scale asymmetric attention unit; the normalization layer and the rectified linear unit activation function layer are added after the convolution layers C0 and C8.

[0074] Step 3, Training the Spatial-Spectral Multi-Scale Unmixing Network: Input the hyperspectral image to be trained into the spatial-spectral multi-scale unmixing network for training;

[0075] (1) Input the training samples into the spatial spectrum multi-scale unmixing network for training, and use the gradient descent algorithm to update the model parameters in reverse.

[0076] (2) The cross-entropy loss function is adopted and the gradient descent algorithm with adaptive learning rate is used for model optimization. The initial learning rate is set to 0.001 and the number of iterations is set to 200.

[0077] Step 4, Acquisition of the sample to be unmixed: Acquire the hyperspectral image to be unmixed and perform preprocessing. The preprocessing is the same as in Step 1.

[0078] Step 5, Obtaining the Hyperspectral Remote Sensing Image Unmixing Result: Input the preprocessed hyperspectral image into the trained spatial-spectral multi-scale unmixing network to obtain the hyperspectral remote sensing image unmixing result.

[0079] In addition, the evaluation criteria for the unmixing results of hyperspectral remote sensing images are root mean square error, average abundance angular distance, and average spectral angular distance.

[0080] The root mean square error is used to assess the accuracy of abundance estimation, and its mathematical formula is as follows:

[0081]

[0082] In the formula, ∑ and ∑ represent the square root operation and the summation operation respectively, N represents the number of pixels, R is the number of endmembers, || || 2 This represents the 2-norm operation, a i and These represent the true abundance vector and the estimated abundance vector of the i-th pixel, respectively.

[0083] The average abundance angular distance is used to measure the similarity between estimated abundance and true abundance elements, and its formula is as follows:

[0084]

[0085] In the formula, cos represents the cosine operation. Let || represent the transpose of the true abundance vector of the i-th pixel, where || represents the 1-norm operation.

[0086] The average spectral angular distance is used to measure the similarity between the estimated endmember and the true endmember, and its formula is as follows:

[0087]

[0088] In the formula, e k and respectively represent the kth real endmember vector and the estimated endmember vector, respectively represent the kth real endmember vector and the estimated endmember vector,

[0089] <Embodiment Two>

[0090] The embodiment two provides a system for hyperspectral remote sensing image unmixing under mixed noise conditions based on the above method of the application, which comprises a sample acquisition part, a model building part, a training part, a to-be-solved image acquisition part, an unmixing part, an input display part and a control part.

[0091] The sample acquisition part can perform the content described in step 1 above, and acquire the hyperspectral image as a training sample.

[0092] The model building part can perform the content described in step 2 above, and build a spectral multi-scale unmixing network model.

[0093] The training part can perform the content described in step 3 above, and input the training sample into the spectral multi-scale unmixing network model for training.

[0094] The to-be-solved image acquisition part can perform the content described in step 4 above, and acquire the hyperspectral image to be unixed.

[0095] The unmixing part can perform the content described in step 5 above, and input the hyperspectral image to be unixed into the trained spectral multi-scale unmixing network model to obtain the hyperspectral remote sensing image unmixing result.

[0096] The input display part can allow the operator to input operation instructions, and display corresponding information according to the control instructions. Specifically, the input display part can display the input, output data and processing process of the sample acquisition part, the model building part, the training part, the to-be-solved image acquisition part and the unmixing part in the form of a data table or an image statically or dynamically according to the operation instructions.

[0097] The control part is in communication connection with the sample acquisition part, the model building part, the training part, the to-be-solved image acquisition part, the unmixing part and the input display part, and controls the operation of them.

[0098] The above embodiment is only an example of the technical solution of the application. The hyperspectral remote sensing image unmixing method and system under mixed noise conditions involved in the application are not limited to the content described in the above embodiment, but are limited to the scope defined in the claims. Any modification, supplement or equivalent replacement made by the person skilled in the art on the basis of the embodiment is within the scope claimed by the claims of the application.

Claims

1. A method for unmixing hyperspectral remote sensing image under mixed noise condition, characterized in that, The method comprises the following steps: Step 1, obtaining a hyperspectral image as a training sample; Step 2, building a hyperspectral multi-scale unmixing network model: A multi-scale asymmetric attention unit is constructed for building the hyperspectral multi-scale unmixing network model; the multi-scale asymmetric attention unit is formed by connecting a multi-scale asymmetric convolution module and a hyperspectral attention module in series; the multi-scale asymmetric attention unit comprises a three-dimensional multi-scale asymmetric attention unit and a two-dimensional multi-scale asymmetric attention unit; the three-dimensional multi-scale asymmetric convolution unit comprises three three-dimensional convolution layers C1-C2, C3-C4 and C5-C6 arranged in parallel; C1 and C2 are convolution layers arranged in series, and the convolution kernel sizes are 1x3x3 and 3x1x1 respectively, and the step sizes are both 1; C3 and C4 are convolution layers arranged in series, and the convolution kernel sizes are 1x5x5 and 5x1x1 respectively, and the step sizes are both 1; C5 and C6 are convolution layers arranged in series, and the convolution kernel sizes are 1x7x7 and 7x1x1 respectively, and the step sizes are both 1; the multi-scale asymmetric convolution module combines the output hyperspectral feature maps of the convolution layers C2, C4 and C6 by joint operation; the multi-scale asymmetric convolution module further comprises a three-dimensional convolution layer C7 arranged in series with the output hyperspectral feature maps, for fusing the hyperspectral features output by C2, C4 and C6, C7 is a convolution layer with a convolution kernel size of 1 and a step size of 1; a normalization layer and a rectified linear unit activation function layer are added after each of the three-dimensional convolution layers C1-C7; the difference between the two-dimensional multi-scale asymmetric attention unit and the three-dimensional multi-scale asymmetric attention unit is only that the convolution kernel size is reduced from three-dimensional to two-dimensional; Step 3, inputting the training sample into the hyperspectral multi-scale unmixing network model for training; Step 4, obtaining a hyperspectral image to be unmixing; Step 5, inputting the hyperspectral image to be unmixing into the trained hyperspectral multi-scale unmixing network model to obtain a hyperspectral remote sensing image unmixing result; In step 2, the hyperspectral attention module comprises channel attention and spatial attention; assuming that the input feature is F, the channel attention CA(F) is calculated according to the following formula: wherein, Relu () and Sigmoid () are nonlinear activation functions, W 1 and W 0 are the weights after average pooling and max pooling, respectively, and are the feature descriptors obtained by average pooling and max pooling in the spatial dimension, respectively, and the channel attention CA(F) is multiplied by the input feature F to obtain the output feature ; The output features A spatial dimension attention SA(F) is introduced, and the calculation formula is as follows: In the formula, and respectively, the feature descriptors obtained by average pooling and maximum pooling in the channel direction, and the channel attention SA(F) is multiplied with the input feature to obtain the output feature ; The channel attention and the spatial attention are connected in series to form the hyperspectral attention module.

2. The method of claim 1, wherein the method is a method of unmixing hyperspectral remote sensing imagery under mixed noise conditions, The method is characterized in that: wherein, In step 2, the hyperspectral multi-scale unmixing network model is built based on a three-dimensional and two-dimensional hybrid network, comprising convolution layers C0 and C8, three-dimensional and two-dimensional multi-scale asymmetric attention units, two residual modules and a full connection module; In the hyperspectral remote sensing image unmixing network model, an input of a convolutional layer C0 is the hyperspectral remote sensing image after the blocking; the C0 is a three-dimensional convolutional layer with a convolution kernel size of 3 and a step size of 1; an input of a first multi-scale asymmetric attention unit is an output feature F0 of the C0; an input of a first residual module is the output feature F0 of the C0; an input of a convolutional layer C8 is a feature after a matrix transformation operation of the output feature F0 of the C0 and an output feature F1 of the first multi-scale asymmetric attention unit; the C8 is a two-dimensional convolutional layer with a convolution kernel size of 1 and a step size of 1; the matrix transformation operation converts a three-dimensional spatial-spectral matrix after the residual fusion of the F0 and the F1 into a two-dimensional spatial-spectral matrix; an input of a second multi-scale asymmetric attention unit is an output feature F2 of the C8; the second multi-scale asymmetric attention unit is based on a two-dimensional multi-scale asymmetric convolution module and is a two-dimensional multi-scale asymmetric attention unit; an input of a second residual module is the output feature F2 of the C8; an input of a full connection module is a feature after the residual fusion of the F2 and an input feature of the second multi-scale asymmetric attention unit. Normalization layers and rectified linear unit activation function layers are added after the convolutional layers C0 and C8.

3. The hyperspectral remote sensing image unmixing method under mixed noise conditions according to claim 1, characterized in that: wherein In step 3, after the training sample is input into the hyperspectral remote sensing image unmixing network model, the model parameters are updated in a reverse direction by using a gradient descent algorithm; a cross-entropy loss function is used to optimize the model by using the gradient descent algorithm with an adaptive learning rate, and the learning rate is initially set to 0.001 and the iteration number is set to 200.

4. The method of claim 1, wherein the method is a method of unmixing hyperspectral remote sensing imagery under mixed noise conditions, characterized in that: wherein, In step 1, the training sample is obtained in the following manner: For the input hyperspectral remote sensing image, first, data preprocessing is performed, including geometric correction and atmospheric correction, to obtain a hyperspectral remote sensing image after the preprocessing; The hyperspectral remote sensing image after the preprocessing is added with mixed noise to make a mixed noise hyperspectral remote sensing image dataset, and a blocking operation is performed; From each type of surface feature in the mixed noise hyperspectral remote sensing image dataset after the blocking, training samples are obtained in proportion.

5. The hyperspectral remote sensing image unmixing method under mixed noise conditions according to claim 4, characterized in that: wherein In step 4, the obtained hyperspectral image is preprocessed in the same manner as in step 1 to serve as a hyperspectral image to be unixed.

6. A hyperspectral remote sensing image unmixing system under mixed noise conditions, characterized in that, It comprises: a sample acquisition unit that acquires a hyperspectral image as a training sample; a model building unit that builds the hyperspectral remote sensing image unmixing network model: a multi-scale asymmetric attention unit is constructed to build the hyperspectral remote sensing image unmixing network model; the multi-scale asymmetric attention unit is formed by connecting a multi-scale asymmetric convolution module and a spatial-spectral attention module in series; the multi-scale asymmetric attention unit comprises a three-dimensional multi-scale asymmetric attention unit and a two-dimensional multi-scale asymmetric attention unit; the three-dimensional multi-scale asymmetric convolution unit comprises three three-dimensional convolutional layers C1-C2, C3-C4 and C5-C6 arranged in parallel; C1 and C2 are convolutional layers arranged in series, with a convolution kernel of 1x3x3 and 3x1x1 respectively, and a step size of 1; C3 and C4 are convolutional layers arranged in series, with a convolution kernel size of 1x5x5 and 5x1x1 respectively, and a step size of 1; C5 and C6 are convolutional layers arranged in series, with a convolution kernel size of 1x7x7 and 7x1x1 respectively, and a step size of 1; the multi-scale asymmetric convolution module combines the output spectral feature maps of the convolutional layers C2, C4 and C6 by joint operation; the multi-scale asymmetric convolution module further includes a three-dimensional convolutional layer C7 arranged in series with the output spectral feature maps, for fusing the spectral features output by C2, C4 and C6, C7 is a convolutional layer with a convolution kernel size of 1 and a step size of 1; a normalization layer and a rectified linear unit activation function layer are added after each of the three-dimensional convolutional layers C1-C7; the difference between the two-dimensional multi-scale asymmetric attention unit and the three-dimensional multi-scale asymmetric attention unit is only that the convolution kernel is reduced from three-dimensional to two-dimensional; a training unit, which inputs training samples into the hyperspectral unmixing network model for training; a to-be-unmixed image acquisition unit, which acquires a hyperspectral image to be unixed; an unmixing unit, which inputs the hyperspectral image to be unixed into the trained hyperspectral unmixing network model to obtain a hyperspectral remote sensing image unmixing result; a control unit, which is in communication with the sample acquisition unit, the model building unit, the training unit, the to-be-unmixed image acquisition unit and the unmixing unit, and controls the operation of the units; In the model building unit, the spectral attention module includes channel attention and spatial attention; let the input feature be F, then the channel attention CA(F) is calculated according to the formula: wherein, Relu () and Sigmoid () are both nonlinear activation functions, W 1 and W 0 are the weights after average pooling and max pooling respectively, and are the feature descriptors obtained by average pooling and max pooling respectively in the spatial dimension, the channel attention CA(F) is multiplied by the input feature F to obtain the output feature ; The output features A spatial dimension attention SA(F) is introduced, and the calculation formula is as follows: In the formula, and respectively, are the feature descriptors obtained by average pooling and maximum pooling in the channel direction, and the channel attention SA(F) is multiplied with the input feature to obtain the output feature ; The channel attention and the spatial attention are connected in series to form the spectral attention module.

7. The hyperspectral remote sensing image unmixing system under mixed noise conditions according to claim 6, wherein, Further comprising: an input display unit, which is in communication with the control unit, allows an operator to input operation instructions, and displays corresponding information according to the control instructions.

8. The hyperspectral remote sensing image unmixing system under mixed noise conditions of claim 6, Characterized in that: In the model building unit, the hyperspectral multi-scale unmixing network is based on a three-dimensional and two-dimensional hybrid network, including convolutional layers C0 and C8, three-dimensional and two-dimensional multi-scale asymmetric attention units, two residual modules and a full connection module; in the hyperspectral multi-scale unmixing network: the input of the convolutional layer C0 is the blocked hyperspectral remote sensing image, C0 is a three-dimensional convolutional layer with a convolution kernel size of 3 and a step size of 1; the input of the first multi-scale asymmetric attention unit is the output feature F0 of C0; the input of the first residual module is the output feature F0 of C0; the input of the convolutional layer C8 is the feature after the residual fusion of the features F0 and F1 output by the first multi-scale asymmetric attention unit and the matrix transformation operation, C8 is a two-dimensional convolutional layer with a convolution kernel size of 1 and a step size of 1; the matrix transformation operation converts the three-dimensional spectral matrix after the residual fusion of F0 and F1 into a two-dimensional spectral matrix; the input of the second multi-scale asymmetric attention unit is the output feature F2 of C8; the second multi-scale asymmetric attention unit is based on a two-dimensional multi-scale asymmetric convolution module and is a two-dimensional multi-scale asymmetric attention unit; the input of the second residual module is the output feature F2 of C8; the input of the full connection module is the feature after the residual fusion of F2 and the input feature of the second multi-scale asymmetric attention unit. ​ Normalization layers and rectified linear unit activation function layers are added after both convolutional layers C0 and C8. Normalization layers and rectified linear unit activation function layers are added after both convolutional layers C0 and C8.

Images