Hyperspectral unmixing method of multi-scale collaborative attention network based on initialized end members

Abstract

The invention discloses a hyperspectral unmixing method of a multi-scale collaborative attention network based on an initialized end member, and the method comprises the steps: an abundance estimation branch module extracts the spatial features and spectral features of an original hyperspectral image through a sequence module, and carries out the fusion of the spatial features and spectral features; the multi-scale collaborative attention module performs spatial enhancement and channel feature representation on the features output by the sequence module to obtain an estimated abundance matrix; the end member estimation branch module processes an original hyperspectral image through a vertex component analysis method to obtain an initialized learnable end member matrix, and optimizes the learnable end member matrix to obtain an estimated end member matrix; carrying out joint training on the estimated abundance matrix and the estimated end member matrix; in the training process, matrix multiplication is carried out on the estimated abundance matrix and the estimated end member matrix to obtain a reconstructed hyperspectral image, and a loss function is used to adjust the network parameter weight to optimize the network unmixing effect. According to the method, the problem of insufficient degree of freedom of flexibility of end member estimation can be solved, and spatial information and spectral information existing in the unmixing process are utilized.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of hyperspectral unmixing, double-branch structure, multi-scale collaborative attention mechanism and endmember estimation technology, and in particular to a hyperspectral unmixing method based on a multi-scale collaborative attention network of initialized endmembers. BACKGROUND

[0002] Hyperspectral images can record spectral reflectance covering a wide range of wavelengths. Due to its rich spectral information, it has attracted widespread attention. This makes hyperspectral images have wide application prospects in pollution monitoring, mineral exploration, food safety, target detection and other fields. However, due to the limitations of the performance of the imaging spectrometer and the complexity of the real scene, the spectral contributions of multiple materials are contained in the mixed pixels. Hyperspectral unmixing technology aims to solve this problem by estimating the endmembers and their abundances in each pixel.

[0003] Deep autoencoders have achieved remarkable success in the field of hyperspectral unmixing, and many unmixing methods based on autoencoders have emerged. Gao et al. used two cascaded autoencoders with cycle consistency constraints for unmixing, reducing the loss of material and detail information in the reconstruction process. Xu et al. used a stationary wavelet transform with a sequence to decompose the hyperspectral image into low-frequency and high-frequency components, and used a parallel multi-domain feature encoding module to capture spatial-spectral features across different frequency domains.

[0004] However, most existing methods treat the weights of the decoder as endmembers rather than directly estimating them, which limits the flexibility and freedom of endmember learning. To overcome this limitation, various methods for independent endmember estimation have been proposed. Shi et al. estimated the abundance and endmember separately by introducing an inference model and a generative model in the model, and considered the variability of the endmember at the same time. Tao et al. enhanced the hyperspectral image by calculating the pixel and band similarity using a feature fusion module, and then estimated the abundance and endmember separately. Although these methods can achieve independent endmember estimation, it is still difficult to fully utilize spatial-spectral information in the unmixing process, which will limit the performance of unmixing.

[0005] For example, Chinese patent application (application number 2023115830409) discloses a dual autoencoder-based hyperspectral unmixing method based on abnormal point weighting, which uses a double network series structure of endmembers and abundance, and improves the unmixing accuracy through a log(SAD) loss function and an abnormal point weighting mechanism. However, the endmember extraction network in the double network structure only relies on fully connected layers to process spectral information, and the one-dimensional convolution used in the abundance estimation network also fails to capture spatial features, resulting in the model failing to fully integrate spatial-spectral collaborative information, and the spatial detail resolution is insufficient in complex mixed scenes, limiting the generalization performance.

[0006] For example, a hyperspectral image unmixing method based on end member oriented Transformer and end member beam is disclosed in a Chinese patent application (application number 2024115868523), which separates end member extraction and abundance estimation, suppresses spectral variability through an end member oriented attention mechanism, and generates physically constrained end member curves using end member beams. However, this method fails to fully exploit the spatial context correlation between pixels during unmixing, and does not fully utilize the spectral-spatial joint features, resulting in limited unmixing accuracy in weak texture areas. SUMMARY

[0007] The purpose of the present application is to solve the problems existing in the prior art and provide a hyperspectral unmixing method based on an initialization end member multi-scale collaborative attention network. The present application can solve the problem of insufficient flexibility of end member estimation and fully utilize the spatial-spectral information existing in the unmixing process.

[0008] Technical solution: The present application is a hyperspectral unmixing method based on an initialization end member multi-scale collaborative attention network, which constructs a hyperspectral unmixing network model. The hyperspectral unmixing network model includes an abundance estimation branch module and an end member estimation branch module. After inputting the original hyperspectral image Y into the hyperspectral unmixing network model, the following operations are performed:

[0009] Step 1, the abundance estimation branch module includes four sequential modules, and a multi-scale collaborative attention module is arranged between each adjacent two sequential modules; the sequential module extracts the spatial features and spectral features in the original hyperspectral image Y and fuses them, and the multi-scale collaborative attention module performs spatial enhancement and channel feature representation on the features output by the sequential module, and finally obtains the estimated abundance matrix

[0010] The end member estimation branch module processes the original hyperspectral image Y by vertex component analysis (VCA) to obtain a learnable end member matrix after initialization, and optimizes it to obtain an estimated end member matrix

[0011] Step 2, the estimated abundance matrix and the estimated end member matrix are jointly trained; during each joint training process, the estimated abundance matrix and the estimated end member matrix are multiplied to obtain a reconstructed hyperspectral image and calculate the overall loss function to adjust the network parameter weight and continuously optimize the unmixing effect of the network;

[0012] Step 3, train the hyperspectral unmixing network model until the overall loss function Converge to obtain a trained network; input the original hyperspectral image Y into the trained network to obtain a final estimated abundance matrix estimate the endmember matrix and reconstruct the hyperspectral image achieve hyperspectral unmixing.

[0013] Further, the first three sequential modules in the abundance estimation branch module (effectively fusing spatial and spectral information) each sequentially perform 3x3 two-dimensional convolution, two-dimensional batch normalization and LeakyReLU operation, and the last sequential module sequentially performs 3x3 two-dimensional convolution and Softmax operation.

[0014] The output formula of the first three modules is as follows:

[0015]

[0016] In the above formula, denotes the output of the i-th sequential module, and the feature map sizes output by the first three sequential modules are Bx32PxHxW, Bx16PxHxW and Bx4PxHxW respectively; Conv(·) denotes a convolution layer with a convolution kernel size of 3x3, BN(·) denotes batch normalization, and LeakyReLU(·) denotes a LeakyReLU activation function.

[0017] The outputs of the first three sequential modules are all input to the next multi-scale collaborative attention module for processing to obtain enhanced feature maps The feature map output by the third multi-scale collaborative attention module is passed to the fourth sequential module to generate an estimated abundance matrix The formula is as follows:

[0018]

[0019] wherein softmax(·) denotes a softmax function, which is used to ensure the sum of the abundances (ASC) and the non-negative constraint (ANC), and the estimated abundance matrix has a size of PxN.

[0020] Further, the multi-scale collaborative attention module in the abundance estimation branch module includes a hybrid multi-scale spatial attention mechanism for constructing the dependency relationship between spatial features and an RMS enhanced channel linear attention mechanism for constructing the dependency relationship between spectral features, and the multi-scale collaborative attention module can be seamlessly integrated into each layer of the network, improving the feature expression capability while not introducing significant computational overhead.

[0021] The mixed multi-scale spatial attention module is used to capture the spatial dependency at multiple scales, and the specific working process is as follows:

[0022] First, the input feature is down-sampled by using average pooling and group normalization with a group number of one, and a compressed representation with a size of BxCxH'xW' is obtained. The dimensions of

[0023] Then, the two spatial representations are evenly divided into four groups along the channel dimension, and each group is processed by a shared depth one-dimensional convolution (SDConv) with a kernel size of 1, 3, 5, and 7, respectively, and the output is as follows:

[0024]

[0025] wherein represents the jth group of features in , Concat(·) represents the tensor concatenation operation, GN-4(·) represents the group normalization with a group number of four, and σ(·) represents the sigmoid activation function.

[0026] Finally, the two refined features are multiplied element-wise with the input feature to obtain the output of the mixed multi-scale spatial attention module, and the formula is as follows:

[0027]

[0028] The RMS enhanced channel linear attention module is used to effectively model the dependency between channels, and the specific working process is as follows:

[0029] First, the input spatial feature map is down-sampled by using average pooling and group normalization with a group number of one, and a compressed representation with a size of BxCxH'xW' is obtained.

[0030] Secondly, three depth two-dimensional convolutions with a kernel size of 1x1 are applied to respectively, and the query key and value with a size of BxCxN' are obtained through the SiLU activation function, where N' = H'xW'.

[0031] Then, the obtained three tensors are reshaped to obtain and with a size of where N h represents the number of attention heads.

[0032] The reshaped tensor is used for multi-head linear attention to realize the modeling of global spatial-spectral information with lower computational cost.

[0033] Then, the attention weight is calculated, and the formula is as follows:

[0034]

[0035] where Permute(·) represents a tensor dimension ordering operation, Matmul(·) represents a tensor multiplication operation, is a standard scaling factor, which is used to alleviate the influence of vector dimension on the stability of the attention mechanism;

[0036] Subsequently, the attention weight is used to calculate the attention map, and the formula is as follows:

[0037]

[0038] where the size of the attention map is BxCxH'xW', and after further normalization, the attention map is element-wise multiplied with the refined spatial feature to optimize the spatial prior, and the formula is as follows:

[0039]

[0040] where RMSNorm(·) represents the RMS norm, which normalizes the feature according to the RMS value of the feature to enhance the stability and convergence of training.

[0041] Further, in the optimization learning process of the endmember estimation branch module on the endmember matrix, the estimated endmember matrix The dimension of the endmember matrix is [L, P], L is the number of bands, and P is the number of endmembers.

[0042] First, a learnable matrix with dimension [L, P] is defined; then, the original hyperspectral image is processed by vertex component analysis VCA to also obtain a matrix with dimension [L, P], which can be understood as a preliminary estimated endmember matrix, because VCA is essentially a method of extracting endmembers from an image; finally, the matrix obtained by VCA initialization covers the previously defined learnable matrix, and at this time, the initial endmember matrix is obtained, and then the endmember matrix is put into the network for training to obtain the final estimated endmember matrix

[0043] Further, the overall loss function includes the mean square error loss function the spectral angle distance loss function and the endmember distance loss function The calculation formula is as follows:

[0044]

[0045] λ MSE , λ SAD and λ ED are hyperparameters used to balance the individual loss contributions;

[0046] the mean squared error loss function L MSE is used to ensure accurate reconstruction of the hyperspectral image, and the calculation formula is as follows:

[0047]

[0048] In the above formula, N represents the number of pixels, y n and respectively represent the nth pixel in the original hyperspectral image Y and the reconstructed hyperspectral image .

[0049] To further enhance the spectral similarity between the input pixels and the reconstructed pixels, the spectral angle distance loss function is used as the second term, and the calculation formula is as follows:

[0050]

[0051] To improve the stability and accuracy of endmember estimation, the endmember distance loss function is introduced as the third term, and the calculation formula is as follows:

[0052]

[0053] In the above formula, L represents the number of bands, P represents the number of endmembers, represents the pth endmember in the endmember matrix .

[0054] Advantages: Compared with the prior art, the present application has the following advantages:

[0055] 1. The present application proposes a dual-branch network architecture, which includes an independent abundance estimation branch and an endmember estimation branch. The endmember estimation is performed through a learnable endmember matrix (rather than the traditional method relying on decoder weights), which significantly improves the flexibility and accuracy of endmember estimation.

[0056] 2. The present application innovatively designs a multi-scale collaborative attention mechanism, which integrates a hybrid multi-scale spatial attention module and an RMS-enhanced channel linear attention module. This mechanism effectively captures the multi-scale spatial-spectral dependence in hyperspectral images and enhances the feature representation capability. BRIEF DESCRIPTION OF DRAWINGS

[0057] Figure 1 The overall network structure diagram of the application;

[0058] Figure 2 The estimated abundance map obtained by the technical solution of the application and the existing unmixing method on the Apex data set;

[0059] Figure 3 The experimental results of the technical solution of the application on the Muffle data set;

[0060] Figure 4 The endmember comparison map of two minerals obtained by the technical solution of the application on the Cuprite data set: Figure 4 (a) in which is the endmember comparison map corresponding to kaolinite, Figure 4 (b) in which is the endmember comparison map corresponding to montmorillonite;

[0061] Figure 5 The estimated abundance map (estimated abundance matrix) of two minerals obtained by the technical solution of the application on the Cuprite data set: Figure 5 (a) in which is the estimated abundance map (estimated abundance matrix) corresponding to kaolinite, Figure 5 (b) in which is the estimated abundance map (estimated abundance matrix) corresponding to montmorillonite. DETAILED DESCRIPTION

[0062] The technical solution of the application will be described in detail below, but the protection scope of the application is not limited to the described embodiments.

[0063] As shown in Figure 1 , the hyperspectral unmixing method based on the initialized endmember multi-scale collaborative attention network of the application constructs a hyperspectral unmixing network model, the hyperspectral unmixing network model includes an abundance estimation branch module and an endmember estimation branch module, and after the original hyperspectral image Y is input into the hyperspectral unmixing network model, the following operations are performed:

[0064]

[0065] Step 1, the abundance estimation branch module includes four sequential modules, and a multi-scale collaborative attention module is arranged between each adjacent two sequential modules; the sequential module extracts spatial features and spectral features in the original hyperspectral image Y and fuses them, and the multi-scale collaborative attention module performs spatial enhancement and channel feature representation on the features output by the sequential module, and finally obtains the estimated abundance matrix

[0066] The endmember estimation branch module processes the original hyperspectral image Y by the vertex component analysis method VCA to obtain an initialized learnable endmember matrix, and optimizes it to obtain an estimated endmember matrix​

[0067] Step 2, the estimated abundance matrix obtained in step 1 is multiplied by the estimated endmember matrix to obtain a reconstructed hyperspectral image Step 3, the estimated abundance matrix obtained in step 2 is multiplied by the estimated endmember matrix to obtain a reconstructed hyperspectral image Step 4, the estimated abundance matrix obtained in step 3 is multiplied by the estimated endmember matrix to obtain a reconstructed hyperspectral image Step 5, the estimated abundance matrix obtained in step 4 is multiplied by the estimated endmember matrix to obtain a reconstructed hyperspectral image Step 6, the estimated abundance matrix obtained in step 5 is multiplied by the estimated endmember matrix to obtain a reconstructed hyperspectral image Step 7, the estimated abundance matrix obtained in step 6 is multiplied by the estimated endmember matrix to obtain a reconstructed hyperspectral image Step 8, the estimated abundance matrix obtained in step 7 is multiplied by the estimated endmember matrix to obtain a reconstructed hyperspectral image

[0068] Step 3, the estimated abundance matrix obtained in step 2 is multiplied by the estimated endmember matrix to obtain a reconstructed hyperspectral image Step 4, the estimated abundance matrix obtained in step 3 is multiplied by the estimated endmember matrix to obtain a reconstructed hyperspectral image Step 5, the estimated abundance matrix obtained in step 4 is multiplied by the estimated endmember matrix to obtain a reconstructed hyperspectral image Step 6, the estimated abundance matrix obtained in step 5 is multiplied by the estimated endmember matrix to obtain a reconstructed hyperspectral image Step 7, the estimated abundance matrix obtained in step 6 is multiplied by the estimated endmember matrix to obtain a reconstructed hyperspectral image

[0069] The present application adopts a double-branch architecture: one branch is used for estimating abundance, and the other branch is used for estimating endmembers. In the abundance branch, a multi-scale collaborative attention mechanism is introduced to effectively capture the spatial-spectral information in the hyperspectral image. Finally, by multiplying the learned abundance matrix and the endmember matrix, the hyperspectral image can be accurately reconstructed, realizing efficient hyperspectral unmixing. Overall, the present application breaks through the traditional method based on decoder weight constraint, adopts a double-branch network structure, significantly improves the flexibility and accuracy of endmember estimation, and thus realizes accurate hyperspectral unmixing. The first three sequential modules in the abundance estimation branch module of the present embodiment each perform 3x3 two-dimensional convolution, two-dimensional batch normalization and LeakyReLU operation in sequence, and the last sequential module performs 3x3 two-dimensional convolution and Softmax operation in sequence;

[0070] The output formulas of the first three modules are as follows:

[0071]

[0072] In the above formula, represents the output of the i-th sequential module, and the feature map sizes of the outputs of the first three sequential modules are Bx32PxHxW, Bx16PxHxW and Bx4PxHxW respectively; Conv(·) represents a convolution layer with a convolution kernel size of 3x3, BN(·) represents batch normalization, and LeakyReLU(·) represents a LeakyReLU activation function;

[0073] The outputs of the first three sequential modules are The enhanced feature maps are obtained by inputting the next multi-scale collaborative attention module for processing The feature map output by the third multi-scale collaborative attention module is passed to the fourth sequential module to generate an estimated abundance matrix The formula is as follows:

[0074]

[0075] wherein softmax(·) represents a softmax function, used to ensure that the sum of the abundance is one, ASC and non-negative constraint ANC, the estimated abundance matrix has a size of P x N.

[0076] The multi-scale collaborative attention module in the embodiment abundance estimation branch module includes a hybrid multi-scale spatial attention mechanism for constructing the dependency relationship between spatial features and an RMS enhanced channel linear attention mechanism for constructing the dependency relationship between spectral features;

[0077] The specific working process of the hybrid multi-scale spatial attention module is as follows:

[0078] First, the input feature is subjected to average pooling and maximum pooling in the width direction and the height direction respectively, thereby generating two spatial representations: and with sizes of B x C x H and B x C x W respectively.

[0079] Then, the two spatial representations are each divided into four groups along the channel dimension, and each group is processed by a shared depth one-dimensional convolution (SDConv) with a convolution kernel size of 1, 3, 5 and 7 respectively, and the output is as follows:

[0080]

[0081] wherein represents the jth group of features in , Concat(·) represents a tensor concatenation operation, GN-4(·) represents group normalization with a group number of four, and σ(·) represents a sigmoid activation function.

[0082] Finally, the two features after refinement are multiplied element by element with the input feature to obtain the output of the hybrid multi-scale spatial attention module, and the formula is as follows:

[0083]

[0084] The specific working process of the RMS enhanced channel linear attention module is as follows:

[0085] First, the input spatial feature map is down-sampled using average pooling and group normalization with a group number of one to obtain a compressed representation with a size of BxCxH'xW';

[0086] Second, three depth two-dimensional convolutions with a kernel size of 1x1 are applied to respectively, and the query key and value are obtained through SiLU activation functions, all with a size of BxCxN', where N'=H'xW';

[0087] Then, the three resulting tensors are reshaped to obtain and all with a size of where N h represents the number of attention heads;

[0088] Next, the attention weights are calculated as follows:

[0089]

[0090] where Permute(·) represents a tensor dimension ordering operation, Matmul(·) represents a tensor multiplication operation, is a standard scaling factor;

[0091] Subsequently, the attention weights are used to calculate the attention map as follows:

[0092]

[0093] where the attention map has a size of BxCxH'xW', which is further normalized and element-wise multiplied with the refined spatial features as follows:

[0094]

[0095] where RMSNorm(·) represents the RMS norm.

[0096] In the optimization learning process of the endmember matrix by the endmember estimation branch module of this embodiment, the estimated endmember matrix is finally obtained by continuously adjusting through back propagation. The dimension of the endmember matrix is [L, P], L is the number of wavebands, and P is the number of endmembers.

[0097] The overall loss function of this embodiment includes a mean square error loss function a spectral angle distance loss function and an endmember distance loss function The calculation formula is as follows:

[0098]

[0099] λ MSE , λ SAD and λ ED The super parameter for balancing the respective loss contributions;

[0100] The calculation formula of the mean square error loss function is as follows:

[0101]

[0102] In the above formula, N represents the number of pixels, y n and respectively represent the nth pixel in the original hyperspectral image Y and the reconstructed hyperspectral image

[0103] The calculation formula of the spectral angle distance loss function is as follows:

[0104]

[0105] The calculation formula of the endmember distance loss function is as follows:

[0106]

[0107] In the above formula, L represents the number of bands, P represents the number of endmembers, represents the nth pixel in the endmember matrix .

[0108] The endmember branch of the embodiment is different from the traditional unmixing method of extracting endmembers from the decoder weight, and the application constructs the endmember extraction as an independent and learnable network branch module; the endmember matrix is initialized by the vertex component analysis algorithm and is optimized end-to-end as a trainable parameter. In the training process, the endmember matrix is constantly updated through back propagation, and is optimized jointly with the abundance matrix, which enables the model to learn more accurate and flexible spectral features, thereby improving the unmixing performance and interpretability.

[0109] To verify the performance of the application, the technical solution of the application is applied to specific data sets for verification. The data sets used include: Apex data set, Muffle data set and Cuprite data set

[0110] ​Apex dataset: This dataset was collected by APEX airborne imaging spectrometer with a spatial resolution of 110x110 pixels, containing 285 spectral bands (wavelength range 0.413-2.412 μm), covering four typical endmembers: road, tree, roof, and water.

[0111] Muffle dataset: This dataset was collected from the University of Mississippi campus with a spatial resolution of 130x90 pixels, containing 64 spectral bands (wavelength range 0.375-1.050 μm). It mainly contains five types of endmembers: roof, grass, tree, shadow, and asphalt.

[0112] Cuprite dataset: This dataset is from the Cuprite mining area in Nevada, USA, containing 250x190 pixels and 224 spectral bands (wavelength range 0.370-2.480 μm). It contains twelve mineral endmembers: alunite, andradite, buddingtonite, dumortierite, kaolinite1, kaolinite2, muscovite, montmorillonite, nontronite, pyrope, sphene, and chalcedony.

[0113] The initialization endmember-based multi-scale collaborative attention network of the present application runs under the PyTorch deep learning framework, and the hardware aspect uses a single Nvidia GeForce RTX 4090 GPU with 64G of running memory and a Windows 10 operating system. The Adam optimizer is used for parameter updating to facilitate stable training and effective convergence. Hyperparameter tuning plays a crucial role in the performance of unmixing, and the hyperparameter settings for each dataset are shown in Table 1.

[0114] Table 1

[0115] Dataset Learning Rate MSE ]]> ​ SAD ]]> ​ ED ]]> ​ Apex 1 x 10 -3 ]] 0.1 4 x 10 -2 ]] 0.3 Muffle 1 x 10 -3 ]]> 9 x 10 -2 ]] 9 x 10 -3 ]] 4 x 10 -2 ]]> Cuprite 5 x 10 -5 ]] 3 x 10 -2 ]] 1 x 10 -4 ]] 0.1

[0116] The performance comparison results of the technical solution of the present application on the Apex dataset are shown in Table 2.

[0117] Table 2

[0118]

[0119] As can be seen from Table 2, the MSSA-IEm method of the present application achieves the optimal overall performance in the endmember estimation task, with a 7.92% improvement over the suboptimal method SSAF-Net. In terms of abundance estimation, MSSA-IEm also shows significant advantages, achieving a 20.86% performance improvement over the suboptimal method SWC-Net. As shown in Table 2, the MSSA-IEm method of the present application achieves the optimal overall performance in the endmember estimation task, with a 7.92% improvement over the suboptimal method SSAF-Net. In terms of abundance estimation, MSSA-IEm also shows significant advantages, achieving a 20.86% performance improvement over the suboptimal method SWC-Net.Figure 2 As shown, the abundance map generated by the present application is highly consistent with the true value.

[0120] The performance comparison results of the technical scheme of the present application on the Muffle dataset are as shown in the following table 2. Figure 3 As shown, it can be observed that the MSSA-IEm method of the present application is superior to the comparative methods in terms of both abundance estimation and endmember extraction, and the aRMSE value (without diagonal line) and aSAD value (with diagonal line) obtained are all the minimum, achieving the best comprehensive performance. In addition, the standard deviation obtained by the MSSA-IEm method of the present application in multiple experiments is also the minimum (error bar), showing better stability, which shows that it has excellent robustness and reliability in the endmember extraction and unmixing task.

[0121] The estimated endmember map and abundance map obtained by the technical scheme of the present application on the Cuprite dataset are as shown in the following table 3. Figure 4 , Figure 5 As shown, since the dataset only provides endmember reference spectrum and does not provide ground true abundance map, in order to evaluate the performance of the MSSA-IEm method of the present application, two representative minerals are studied: (a) kaolinite and (b) montmorillonite. It can be seen from Figure 4 that the extracted spectrum is highly consistent with the reference spectrum, verifying the accuracy of the MSSA-IEm. The estimated abundance map shown in Figure 5 clearly depicts the spatial distribution of the selected minerals, fully demonstrating the effectiveness of the model in capturing the distribution characteristics of material composition in real scenes.

[0122] The ablation experiment results of the technical scheme of the present application on the Apex dataset and the Muffle dataset are as shown in the following table 3.

[0123] Table 3 Ablation experiment results

[0124]

[0125] As can be seen from table 3, removing any module will cause the unmixing accuracy to decrease significantly, which confirms that each component plays an indispensable role. Specifically, the multi-scale collaborative attention module MSSA utilizes the double attention mechanism to capture the spatial and channel dependency, thereby realizing more effective spatial-spectral feature representation. The IEm (endmember extraction) module realizes flexible and accurate endmember estimation by breaking through the constraint of traditional decoder weight. Finally, the complete model achieves the highest performance on all indicators of the two datasets, fully demonstrating the strong complementarity between the multi-scale joint spatial-channel attention modeling and the adaptive endmember estimation. These findings not only highlight the effectiveness of each module, but also further confirm the overall robustness of the MSSA-IEm model of the present application in the hyperspectral unmixing task.

Claims

1. A hyperspectral unmixing method based on initialization endmember-based multi-scale collaborative attention network, characterized in that, A hyperspectral unmixing network model is constructed, and the hyperspectral unmixing network model includes an abundance estimation branch module and an endmember estimation branch module. The step 1, the abundance estimation branch module includes four sequential modules, and a multi-scale collaborative attention module is arranged between each adjacent two sequential modules; the sequential module extracts spatial features and spectral features in the original hyperspectral image Y and fuses them, the multi-scale collaborative attention module performs spatial enhancement and channel feature representation on the features output by the sequential module, and finally an estimated abundance matrix is obtained The end member estimation branch module processes the original hyperspectral image Y by a vertex component analysis method (VCA) to obtain an initialized learnable end member matrix, and optimizes to obtain an estimated end member matrix Step 2, estimate the abundance matrix from the estimated endmember matrix and the estimated endmember matrix Step 3, jointly train the network; in each joint training process, the estimated abundance matrix and the estimated endmember matrix Step 4, multiply the estimated abundance matrix and the estimated endmember matrix to obtain a reconstructed hyperspectral image and calculate the overall loss function to adjust the network parameter weights and continuously optimize the network unmixing effect; Step 3, training the network until the overall loss function converges, obtaining the trained network; inputting the original hyperspectral image Y into the trained network to obtain the final estimated abundance matrix estimating the endmember matrix and reconstructing the hyperspectral image to realize hyperspectral unmixing.

2. The initialization-endom member-based multi-scale co-attention network method for hyperspectral unmixing according to claim 1, wherein, The first three sequential modules in the abundance estimation branch module sequentially perform 3*3 two-dimensional convolution, two-dimensional batch normalization and LeakyReLU operation, and the last sequential module sequentially performs 3*3 two-dimensional convolution and Softmax operation. The output formula of the first three sequential modules is as follows: In the above formula, represents the output of the i-th sequential module, and the feature map sizes of the outputs of the first three sequential modules are Bx32PxHxW, Bx16PxHxW, and Bx4PxHxW, respectively; Conv(·) represents a convolution layer with a kernel size of 3x3, BN(·) represents batch normalization, and LeakyReLU(·) represents a LeakyReLU activation function; The outputs of the first three sequential modules are input into the next multi-scale collaborative attention module for processing to obtain enhanced feature maps The feature maps output by the third multi-scale collaborative attention module are passed to the fourth sequential module to generate an estimated abundance matrix The formula is as follows: where softmax(·) denotes the softmax function to ensure the sum of abundances is one ASC and non-negative constraint ANC, estimates the abundance matrix of size P x N.

3. The initialization-endom member-based multi-scale co-attention network method for hyperspectral unmixing according to claim 1 or 2, characterized in that, The multi-scale collaborative attention module in the abundance estimation branch module includes a mixed multi-scale spatial attention mechanism for constructing the dependency relationship between spatial features and an RMS enhanced channel linear attention mechanism for constructing the dependency relationship between spectral features. The specific working process of the mixed multi-scale spatial attention module is as follows: First, input features are normalized Average pooling and max pooling are performed in the width direction and the height direction, respectively, to generate two spatial representations: and with sizes BxCxH and BxCxW, respectively. Then, the two spatial representations are divided into four groups along the channel dimension, and each group is processed by a shared depth one-dimensional convolution (SDConv) with a convolution kernel size of 1, 3, 5 and 7 respectively, and the output is as follows: wherein respectively represent the jth group of features in , Concat(·) represents a tensor concatenation operation, GN-4(·) represents group normalization with group number four, and σ(·) represents a sigmoid activation function. Finally, the two refined features are multiplied element-wise with the input feature to obtain the output of the hybrid multi-scale spatial attention module, which can be represented as follows: The specific working process of the RMS enhanced channel linear attention mechanism is as follows: First, the input spatial feature maps are down-sampled using average pooling and group normalization with a group number of one to obtain compressed representations with dimensions B x C x H ′ x W ′ ; Second, to respectively apply three depth two-dimensional convolutions with a kernel size of 1x1, and obtain the query key and value with a size of BxCxN ′ , where N ′ =H ′ xW ′ ; The resulting three tensors are then reshaped to obtain and each of size where N h denotes the number of attention heads; Then, the attention weight is calculated, and the formula is as follows: where Permute(·) denotes a tensor dimension permutation operation, Matmul(·) denotes a tensor multiplication operation, is a standard scaling factor; Subsequently, the attention weight is used to calculate the attention map, and the formula is as follows: wherein the attention map has a size of B x C x H ′ x W ′ The attention map is further normalized and element-wise multiplied with the refined spatial features, as follows: Wherein, RMSNorm(·) represents the RMS norm.

4. The initialization-endmember-based multi-scale co-attention network method for hyperspectral unmixing according to claim 1, wherein, The end member estimation branch module continuously adjusts the estimated end member matrix through back propagation in an optimization learning process of the end member matrix The dimension of the end member matrix is [L, P], L is the number of wave bands, and P is the number of end members.

5. The initialization-endom member-based multi-scale co-attention network method for hyperspectral unmixing according to claim 1, wherein, Overall loss function Including mean square error loss function Spectral angle distance loss function And end member distance loss function The calculation formula is as follows: λ MSE , λ SAD and λ ED hyperparameters for balancing the individual loss contributions; mean squared error loss function The calculation formula is as follows: N denotes the number of pixels, y n and denote the n-th pixel in the original hyperspectral image Y and the reconstructed hyperspectral image Y, respectively. Spectral angular distance loss function The calculation formula is specifically as follows: Endmember distance loss function The calculation formula is specifically as follows: L represents the number of bands, P represents the number of endmembers, indicates the pth endmember in the endmember matrix ​

Images