Hyperspectral image unmixing method and system based on end member guided Transform and end member beam

Abstract

The invention discloses a hyperspectral image unmixing method and system based on an end member guided Transform and an end member beam, and relates to the field of hyperspectral unmixing. The method comprises the steps of image preprocessing; constructing a neural network based on an end member oriented Transform and an end member beam; the neural network is trained, and the trained neural network is obtained; and obtaining an unmixing result of the hyperspectral image to be unmixed by using the trained neural network. According to the invention, on the basis of an end member oriented Transform, the characteristics of each end member category are excavated and the influence of the spectrum variability on the unmixing performance is weakened, and an end member oriented Transform encoder module, a heterogeneous information fusion module and an end member generator module based on an end member beam are designed in a targeted manner, so that the influence of the spectrum variability on the unmixing performance can be better inhibited; and the precision of end member extraction and abundance estimation is improved.

Description

Technical Field

[0001] The present invention belongs to the field of image processing, and in particular relates to a hyperspectral image unmixing method and system based on an endmember-guided Transformer and an endmember bundle. Background Art

[0002] Hyperspectral unmixing refers to using the advantages of hyperspectral images in the spectral dimension to analyze the internal composition of the pixels of the image with limited spatial resolution, solve the basic components and corresponding proportions of the hyperspectral image, and obtain the end members and abundance of the image. With the advantage of its spectral resolution, hyperspectral images can depict spectral curves with richer details, providing an effective way to achieve material analysis of imaging targets.

[0003] Due to the differences in lighting, environment and other factors in real imaging scenes, or changes in the imaging target, the phenomenon of "same object, different spectrum" is often presented in hyperspectral images, which brings challenges to hyperspectral unmixing. In order to reduce the impact of the above phenomenon on the performance of hyperspectral unmixing, unmixing algorithms that consider spectral variability have been widely studied. At present, unmixing algorithms that consider spectral variability can be generally divided into algorithms based on endmember bundles, algorithms based on mixture models and algorithms based on deep learning. Traditional unmixing algorithms are usually based on fixed prior assumptions and cannot provide a suitable general solution for many complex data, so their unmixing performance is limited. Algorithms based on deep learning use modules such as fully connected layers and convolutional layers to mine information about spectral variability and fit its impact on the spectrum, and based on this, model the variable information to weaken its impact on abundance estimation. When considering spectral variability, current deep learning-based algorithms often directly output the fitted spectral variable terms based on generative models, and urge network training based on various regularization terms. However, the performance of existing algorithms is unstable, and endmembers with unclear physical meanings may be generated, resulting in inaccurate abundance estimates.

[0004] The endmember beam algorithm can maintain the physical meaning of solving the endmember by extracting multiple spectra to represent the same endmember. However, its optimization speed based on traditional heuristic algorithms is slow and inefficient. Currently, there are few deep learning-based algorithms that can bring out the advantages of the endmember beam algorithm. Summary of the invention

[0005] In order to solve the problems in the prior art, the present invention proposes a hyperspectral image unmixing method and system based on endmember-guided Transformer and endmember bundle.

[0006] The technical solution adopted by the present invention is as follows:

[0007] In a first aspect, the present invention discloses a hyperspectral image unmixing method based on an endmember-guided Transformer and an endmember bundle, comprising the following steps:

[0008] Step 1): Obtain the initial endmembers of the hyperspectral image and obtain the data feature map and endmember tokens;

[0009] Step 2): constructing a neural network based on endmember-guided Transformer and endmember bundles, wherein the neural network is used to obtain abundance estimation results and endmember extraction results of hyperspectral images based on data feature maps and endmember tokens, and the neural network includes an endmember-guided Transformer encoder module, an endmember generator module based on endmember bundles, a heterogeneous information fusion module, and an abundance decoder module;

[0010] Step 3): Train the neural network based on the endmember-guided Transformer and the endmember bundle to obtain a trained neural network;

[0011] Step 4): Use the trained neural network to obtain the abundance estimation result and endmember extraction result of the hyperspectral image to be unmixed, so as to realize the unmixing of the hyperspectral image.

[0012] In a second aspect, the present invention discloses a hyperspectral image unmixing system based on an endmember-guided Transformer and an endmember bundle of the method, comprising:

[0013] An image preprocessing module, which is used to obtain the initial endmembers of the hyperspectral image and obtain the data feature map and endmember tokens;

[0014] A neural network building module, which is used to construct a neural network based on an endmember-guided Transformer and an endmember bundle, wherein the neural network is used to obtain an abundance estimation result and an endmember extraction result of a hyperspectral image based on a data feature map and an endmember token, and the neural network includes an endmember-guided Transformer encoder module, an endmember generator module based on an endmember bundle, a heterogeneous information fusion module, and an abundance decoder module;

[0015] A neural network training module, which is used to train a neural network based on an endmember-guided Transformer and an endmember bundle to obtain a trained neural network;

[0016] The hyperspectral image unmixing module is used to obtain the abundance estimation result and endmember extraction result of the hyperspectral image to be unmixed by using the trained neural network, so as to realize the unmixing of the hyperspectral image.

[0017] Compared with the prior art, the present invention has the following beneficial effects:

[0018] 1) The present invention proposes an end-member-guided Transformer encoder module, which realizes feature extraction of data feature maps based on directional projection, and for the first time proposes a low-redundancy attention mechanism based on a hybrid expert system, which can effectively constrain the solution space of the model, provide more accurate feature extraction results, and improve the network fitting efficiency.

[0019] 2) The present invention proposes an endmember generator module based on endmember bundles, which for the first time converts endmember guidance features into intra-class weights and combines them with endmember bundles to generate endmember estimation results.

[0020] 3) The present invention proposes a heterogeneous information fusion module, which can introduce the complementary prior of the spatial distribution of end members during information fusion, and realize the full fusion of different end member guidance features. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A flowchart of the basic steps of an embodiment of the hyperspectral unmixing method of the present invention;

[0022] Figure 2 It is a structural schematic diagram of the hyperspectral unmixing system of the present invention;

[0023] Figure 3 A schematic diagram of an end-member-guided Transformer module of the present invention;

[0024] Figure 4 This is a schematic diagram of a heterogeneous information fusion module of the present invention;

[0025] Figure 5 Urban hyperspectral image dataset is used for experiments;

[0026] Figure 6 This is a graph of unmixed abundance results of an Urban hyperspectral image dataset after unmixing using the embodiments of the present invention and different methods. DETAILED DESCRIPTION

[0027] The present invention is further described and illustrated below in conjunction with specific embodiments. The embodiments are merely exemplary of the present disclosure and do not define the scope of limitation. The technical features of each embodiment of the present invention may be combined accordingly without conflicting with each other.

[0028] like Figure 1 As shown, it is a basic step flow chart of the hyperspectral image unmixing method of the present invention, which mainly includes:

[0029] Step 1): (Image preprocessing) For the hyperspectral image to be processed, this embodiment uses the vertex component analysis algorithm VCA to extract the initial end member, let Y∈R N×Lrepresents the input hyperspectral image, N represents the total number of pixels in the image, and L represents the number of bands in the hyperspectral image. The calculation formula is as follows:

[0030] M VCA =VCA(Y)

[0031] Among them, M VCA ∈R P×L is the initial endmember, indicating the number of endmembers in the image; P represents the number of endmembers in the hyperspectral image.

[0032] Then, for the hyperspectral image and the initial endmember, a projection layer consisting of a 1×1 two-dimensional convolution is used to project the high-dimensional raw data to a lower dimension. The calculation formula is as follows:

[0033] X=Proj(Y)

[0034] E=Proj(M VCA )

[0035] Where X∈R N×D Represents the data feature map obtained by projection, Proj(*) is the projection function; E∈R P×D represents the end member token obtained by projection, D represents the number of channels of the data feature map and the end member token, and in this embodiment, D=32.

[0036] Step 2): Construct a neural network based on endmember-guided Transformer and endmember bundles, the neural network includes an endmember-guided Transformer encoder module, an endmember generator module based on endmember bundles, a heterogeneous information fusion module, and an abundance decoder module; wherein the endmember-guided Transformer encoder module is used to mine endmember-guided features that are highly correlated with endmember tokens from the data feature map; the endmember generator module based on the endmember bundle converts the endmember-guided features into intra-class weights, and weights the endmember bundles based on the intra-class weights to obtain the final endmember curve; the heterogeneous information fusion module fuses multiple endmember-guided features from the perspective of spatial complementarity and channel interaction to obtain heterogeneous information fusion features; the abundance decoder module processes the heterogeneous information fusion features and outputs an abundance map. This embodiment uses random parameters as the initial weights of the network.

[0037] The specific working process of each module in the neural network based on endmember-guided Transformer and endmember bundle is as follows:

[0038] Step 21): The end-member-guided Transformer encoder module includes P parallel end-member-guided Transformer modules; the end-member-guided Transformer module structure is as follows Figure 3As shown in Figure 2, the endmember-guided Transformer module first constructs an endmember-guided projection operator P based on the input endmember token E. oriented , the calculation formula is as follows:

[0039] P oriented =(E i ) T ·(E i (E i ) T ) -1 ·E i

[0040] Among them, E i is the i-th vector in the end-member token, () T represents the transpose of the matrix, () -1 Represents the inverse of a matrix.

[0041] Then, the endmember-guided Transformer module projects the endmember tokens and data feature maps and performs layer normalization, calculated as follows:

[0042] [E′ i , X'] = LN(P oriented ·[E i , X])

[0043] Among them, E′ i ,X′represent the endmember token and data feature map after endmember-guided projection, []represents the concatenation operation of the matrix; LN(*) is the function of layer normalization processing.

[0044] Then, the endmember-guided Transformer module performs a low-redundancy attention mechanism calculation on the endmember tokens and data feature maps after the endmember-guided projection. The process can be expressed as:

[0045] Q=E′ i W q , K = X'W k , V=X'W v

[0046] head 1 , ..., head n =Split([Q, K, V])

[0047]

[0048] Among them, W q , W k , W v Represents the projection matrices of query matrix Q, key matrix K and value matrix V respectively; head irepresents the ith attention head after the data in the data feature graph after the endmember-guided projection is split. In this embodiment, there are h attention heads in total. Split([Q, K, V]) is the attention head splitting function. Softmax is an activation function that can convert the input into a weight whose sum is 1 and weight it to the value matrix V. head′ i represents the i-th attention head after attention weighting; Q i is the query matrix of the i-th attention head; is the transpose of the key matrix of the i-th attention head; V i is the value matrix of the i-th attention head.

[0049] Then, the de-redundancy module based on the hybrid expert system uses the spectral information divergence to measure the similarity between the attention heads after attention weighting. The calculation formula is expressed as:

[0050]

[0051] Among them, Sim(i,j) is head′ i and head′ j Similarity between i is the i-th attention head after attention weighting; head′ j is the jth attention head after attention weighting.

[0052] Based on the calculation of the similarity between all attention heads after weighting, the similarity measurement matrix can be obtained Among them, h is the number of attention heads.

[0053] The de-redundancy module based on the hybrid expert system calculates the importance of each attention head after weighting according to the similarity measurement matrix, and calculates the weighted sum as the output of the low-redundancy attention mechanism, that is, the fused attention weighted end-member token is obtained. The calculation process can be expressed as:

[0054] g = 1-Softmax(FC(Sim))

[0055]

[0056] in, Represents the gating weight of each attention head after weighting, g i is the gating weight of the i-th attention head after attention weighting; FC represents the fully connected network layer, Represents the fused attention-weighted endmember token.

[0057] In order to achieve feature fusion between channels, the end-member-oriented Transformer module uses layer normalization and multi-layer perceptron to process the attention-weighted data. The formula is as follows:

[0058] [E″′ i , I]=[E″ i ,X]+MLP(LN([E″ i , X]))

[0059] Among them, E″′ i is the final end member token, is the end-member-guided feature, and the end-member-guided feature is used as the final output of the end-member-guided Transformer encoder module. Therefore, for P parallel end-member-guided Transformer modules, the output of the end-member-guided Transformer encoder module can be expressed as:

[0060] I i =EOT(E i , X)

[0061] Among them, I i represents the endmember-guided feature of the i-th endmember; EOT(*) is the function of the endmember-guided Transformer encoder module. Therefore, the encoder outputs a total of P features: I 1 ,…,I P .

[0062] Step 22): The endmember-guided features include the exclusive features belonging to each endmember category. The present invention designs an endmember generator module based on endmember bundles to generate a set of endmembers for each pixel. The structure is as follows: Figure 1 The endmember generator module based on endmember bundles first uses a two-dimensional convolutional layer and a Softmax activation function to transform the endmember-guided features into the intra-class weights of the endmember bundles. The formula is as follows:

[0063] I weight,i =Softmax(Conv(I i ))

[0064] in, represents the intra-class weight of the i-th endmember bundle corresponding to each pixel in the image, H is the height of the hyperspectral image, W is the width of the hyperspectral image, and K is the number of spectra in the endmember bundle. Therefore, the endmember generator module based on the endmember bundle weights each endmember bundle based on the intra-class weight to obtain the final extracted endmember curve, which is expressed as follows:

[0065] M i =I weight,i ·EM i

[0066] M=Concat(M 1 ,...,M P )

[0067] Among them, EM i is the i-th endmember bundle among the P endmember bundles, represents a bundle of P end members, Represents the i-th endmember corresponding to each position in the hyperspectral image; Concat(*) is the concatenation function; thus, the concatenation of each endmember tensor can obtain the final endmember result extracted by the model.

[0068] Step 23): The heterogeneous information fusion module aims to fully fuse the end-member-guided features belonging to different end-members. Its structure is as follows: Figure 4 Taking the processing of the i-th end-member guided feature as an example, the heterogeneous information fusion module first rearranges the end-member guided feature into a three-dimensional tensor, that is,

[0069] In order to extract the meaningful parts of each end-member-guided feature, the heterogeneous information fusion module generates a spatial mask based on two-dimensional convolution and Sigmoid activation function. The formula can be expressed as:

[0070] m i =Sigmoid(Conv(I i ′))

[0071] m j→i =Sigmoid(Conv(I j ′))

[0072] in, For I i ′ contains a mask with valuable spatial information, and isI j 'To Fusion I i ′ is a mask with valuable spatial information.

[0073] Based on the prior information that the spatial distribution of endmembers is heterogeneous, the present invention designs a complementary feature fusion mechanism to enhance the endmember-guided features of the current category based on the complement of the endmember-guided features of other categories. The specific formula is expressed as follows:

[0074]

[0075] in, is the spatial fusion feature of the i-th end member, ⊙ represents the Hadamard product, which can realize element-by-element multiplication. After implementing the above spatial fusion enhancement for each end member pointing feature, the heterogeneous information fusion module generates P spatial fusion features I″ 1 ,…,I″ P.

[0076] Before realizing the channel dimension fusion, the heterogeneous information fusion module uses the splicing layer to implement the splicing of all spatial fusion features along the channel dimension. The formula can be expressed as:

[0077] I spa =Concat(I″ 1 ,…,I″ P )

[0078] in, It is the spatial fusion feature after splicing.

[0079] In order to achieve full interaction in the channel dimension and retain valuable information to the maximum extent, the heterogeneous information fusion module realizes the fusion of the channel dimension based on channel rearrangement and grouped convolutional layer to obtain the features after heterogeneous information fusion. The formula is as follows:

[0080] I fuse =GConv(CS(I spa ))

[0081] in, represents heterogeneous information fusion features, CS represents channel reordering layer, and GConv represents grouped convolutional layer.

[0082] Step 24): The abundance decoder module aims to mine effective abundance information from heterogeneous information fusion features and output the final abundance map. Its structure is as follows Figure 1 As shown. The whole process is implemented based on a multi-layer neural network and is expressed as follows:

[0083] A=Softmax(Conv(GELU(Conv(I fuse ))))

[0084] in, is the abundance map estimated by the model, GELU and Softmax represent two non-linear activation functions, and Conv represents a two-dimensional convolutional layer.

[0085] Step 3: (Neural network model training) The entire hyperspectral image is used as a training sample. The network adopts a self-supervised training method. The sample is input into the network model. The reconstructed root mean square error and the reconstructed spectral angular distance are used as the loss function. The network weights are updated based on the gradient descent method Adam with adaptively adjusted learning rate. This embodiment is iterated for a total of 200 times.

[0086] Step 4: Based on the trained neural network model, the output of the endmember generator module based on the endmember bundle is used as the endmember extracted by the network, and the output of the abundance decoder is used as the abundance map estimated by the network to obtain the unmixing result.

[0087] Figure 2 FIG. 4 is a block diagram of a hyperspectral image unmixing system based on an endmember-guided Transformer and an endmember beam according to an embodiment. Figure 2 As shown, the system includes:

[0088] Image preprocessing module (by Figure 2 The image acquisition module and the image preprocessing module in the image acquisition module are composed of the image acquisition module and the image preprocessing module, which are used to obtain the initial end members of the hyperspectral image and obtain the data feature map and the end member token;

[0089] A neural network building module, which is used to construct a neural network based on an endmember-guided Transformer and an endmember bundle, wherein the neural network is used to obtain an abundance estimation result and an endmember extraction result of a hyperspectral image based on a data feature map and an endmember token, and the neural network includes an endmember-guided Transformer encoder module, an endmember generator module based on an endmember bundle, a heterogeneous information fusion module, and an abundance decoder module;

[0090] The neural network training module is used to train the neural network based on the end-member-guided Transformer and the end-member bundle to obtain a trained neural network; (the neural network construction module and the neural network training module together constitute Figure 2 The neural network module based on endmember-guided Transformer and endmember bundle in

[0091] Hyperspectral image unmixing module (by Figure 2 The unmixing result prediction module and the unmixing result output module in the image are used to obtain the abundance estimation result and the endmember extraction result of the hyperspectral image to be unmixed by using the trained neural network, so as to realize the unmixing of the hyperspectral image.

[0092] Regarding the system in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0093] For the system embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The system embodiment described above is only schematic. For example, the image preprocessing module can be divided into a logical function. There may be other division methods in actual implementation, such as multiple modules can be combined or integrated into another unit. Another point is that the connection between the modules displayed or discussed can be a communication connection through some interfaces, which can be electrical or other forms. Some or all of the modules can be selected according to actual needs to achieve the purpose of the present application. Ordinary technicians in this field can understand and implement it without paying creative labor. The following takes real hyperspectral images as an example to illustrate the specific implementation method to reflect the technical effect of the present invention, and the specific steps in the embodiment will not be repeated.

[0094] Example

[0095] Next, the Urban hyperspectral dataset is used as the research object to verify the unmixing method of the present invention. In order to comprehensively compare the unmixing performance and display the unmixing results from the perspective of visualization and quantification, the estimated abundance map and evaluation indicators: the root mean square error of abundance (aRMSE) and the spectral angular distance of end members (eSAD) are used to evaluate the proposed unmixing method; the formula is as follows:

[0096]

[0097]

[0098] Among them, a ji represents the true abundance value of the jth end member in the ith pixel; represents the estimated abundance value of the jth end member in the ith pixel; m j represents the true value of the jth end member; Represents the estimation result of the jth endmember.

[0099] The Urban dataset contains 307×307 pixels, each pixel contains 162 bands, the wavelength range is 0.4 to 2.5 microns, and the spectral resolution is 10 nanometers. The dataset contains five end members, namely roads, grass, trees, roofs, and soil. Figure 5 Pseudo-color image of the hyperspectral image.

[0100] Figure 6 The unmixed abundance map is obtained for the Urban data set using the embodiments of the present invention and different unmixing algorithms.

[0101] Table 1 Evaluation indicators of unmixing results of Urban hyperspectral image dataset

[0102]

[0103] The comparative method FCLS comes from: DCHeinz, "Fully constrained least squares linearspectral mixture analysis method for material quantification in hyperspectralimagery," IEEE Trans.Geosci.Remote Sensing, vol.39, no.3, pp.529–545, 2001.

[0104] Comparison method MiSiCNet from: B. Rasti, B. Koirala, P. Scheunders, and J. Chanussot, “Misicnet: Minimum simplex convolutional network for deep hyperspectral unmixing,” IEEE Trans. Geosci. Remote Sensing, vol. 60, pp. 1–15, 2022.

[0105] The comparison method IMPSO comes from: R. Liu, P. Wang, B. Du, and B. Qu, "Endmember bundle extraction based on improved multiobjective particle swarm optimization," IEEE Geosci. Remote Sensing Lett., vol. 20, pp. 1–5, 2023.

[0106] The comparison method DOEBE comes from: R.Liu, C.Lei, L.Xie, and X.Qin, "Anovel endmemberbundle extraction framework for capturing endmember variability by dynamicoptimization," IEEE Trans.Geosci.Remote Sensing, vol.62, pp.1–17, 2024.

[0107] The comparative method PGMSU comes from: S.Shi, M.Zhao, L.Zhang, Y.Altmann, and J.Chen, "Probabilistic generative model for hyperspectral unmixing accounting forendmember variability," IEEE Trans.Geosci.Remote Sensing, vol.60, pp.1–15, 2022.

[0108] Comparative method PPM comes from: W.Gao, J.Yang, and J.Chen, "Proportional perturbationmodel for hyperspectral unmixing accounting for endmember variability," IEEEGeosci.Remote Sensing Lett., vol.21, pp.1–5, 2024.

[0109] The comparative method DGMSSU comes from: S.Shi, L.Zhang, Y.Altmann, and J.Chen, "Deepgenerative model for spatial–spectral unmixing with multiple endmemberpriors," IEEE Trans.Geosci.Remote Sensing, vol.60, pp.1–14, 2022.

[0110] The unmixing results of the Urban dataset are as follows Figure 6 As shown, the quantification results are shown in Table 1. In the quantification results, the present invention achieved the best performance, and achieved better unmixing effects for the two endmembers of roof and soil with complex distribution and small area. Based on the heterogeneous information fusion module, this algorithm can introduce the spatial heterogeneity of endmember distribution and achieve more accurate abundance estimation when estimating abundance; in addition, the endmember generator based on the endmember bundle can provide more stable and physically meaningful endmember results.

[0111] The above-mentioned embodiments only express several implementation modes of the present invention, and the description is relatively specific and detailed, but it cannot be understood as limiting the scope of the present invention. For those of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A hyperspectral image unmixing method based on endmember-guided Transformer and endmember bundle, characterized in that: The following steps are involved: Step 1): Obtain the initial endmembers of the hyperspectral image and obtain the data feature map and endmember tokens; Step 2): constructing a neural network based on endmember-guided Transformer and endmember bundles, wherein the neural network is used to obtain abundance estimation results and endmember extraction results of hyperspectral images based on data feature maps and endmember tokens, and the neural network includes an endmember-guided Transformer encoder module, an endmember generator module based on endmember bundles, a heterogeneous information fusion module, and an abundance decoder module; Step 3): Train the neural network based on the endmember-guided Transformer and the endmember bundle to obtain a trained neural network; Step 4): Use the trained neural network to obtain the abundance estimation result and endmember extraction result of the hyperspectral image to be unmixed, so as to realize the unmixing of the hyperspectral image.

2. The hyperspectral image unmixing method based on endmember-guided Transformer and endmember bundle according to claim 1 is characterized in that: The step 1) comprises: The initial endmembers of the hyperspectral image are extracted based on the vertex component analysis algorithm; the calculation formula is as follows: M VCA =VCA(Y) Where Y is the input hyperspectral image, N is the total number of pixels in the image, L is the number of bands in the hyperspectral image; M VCA represents the initial end member, M VCA ∈R P×L , P represents the number of endmembers of the hyperspectral image; The hyperspectral image and the initial endmember are projected using a projection layer consisting of a 1×1 two-dimensional convolution to obtain the data feature map and endmember token; the calculation formula is as follows: X=Proj(Y) E=Proj(M VCA ) Among them, X represents the data feature map obtained by projection, X∈R N×D ; E∈R P×D represents the end-member token obtained by projection, E∈R P ×D ; D represents the number of channels of the data feature map and the end-member token; Proj(*) is the projection function.

3. The hyperspectral image unmixing method based on endmember-guided Transformer and endmember bundle according to claim 1 is characterized in that: In step 2), the endmember-guided Transformer encoder module mines endmember-guided features related to endmember tokens from the data feature graph; the endmember generator module based on endmember bundles converts the endmember-guided features into intra-class weights of endmember bundles, and then weights the endmember bundles based on the intra-class weights to obtain the final endmember curve, and uses the endmember curve as the endmember extraction result; the heterogeneous information fusion module fuses the endmember-guided features from the perspective of spatial complementarity and channel interaction to obtain heterogeneous information fusion features; The abundance decoder module obtains an estimated abundance map based on heterogeneous information fusion features, and uses the abundance map as the abundance estimation result.

4. The hyperspectral image unmixing method based on endmember-guided Transformer and endmember bundle according to claim 3 is characterized in that: The endmember-guided Transformer encoder module mines endmember-guided features related to endmember tokens from the data feature graph; including: The endmember-guided Transformer encoder module includes P parallel endmember-guided Transformer modules; First, the endmember-guided Transformer module constructs the endmember-guided projection operator P based on the input endmember tokens. oriented , the formula is as follows: P oriented =(And i ) T ·(AND i (AND i ) T ) -1 ·AND i Among them, E i is the i-th vector in the end-element token; () T represents the transpose of a matrix, () -1 represents the inverse of a matrix; Then, the endmember tokens and data feature maps are subjected to endmember-guided projection and layer normalization to obtain the endmember tokens and data feature maps after endmember-guided projection; the calculation formula is as follows: [HAVE BEEN' i ,X′]=LN(P oriented ·[HAVE BEEN i ,X]) Where E is the end-member token, E∈R P×D , D represents the number of channels of data feature graph and endmember token, P is the number of endmembers of hyperspectral image; E′ i is the endmember token after endmember-guided projection; X is the data feature map, X∈R N×D , N represents the total number of pixels in the image; X′ is the data feature map after end-member guided projection; [] represents the concatenation operation of the matrix; LN(*) is the function of layer normalization processing; Perform low-redundancy attention mechanism calculation on the endmember tokens and data feature maps after endmember-guided projection to obtain fused attention-weighted endmember tokens; Based on the layer normalization method and multi-layer perceptron processing fused attention weighted endmember tokens, the endmember-oriented features are obtained; the calculation formula is as follows: [E″′ i ,I]=[E″ i ,X]+MLP(LN([E″ i ,X])) Among them, E″′ i is the final end member token; I is the end member guide feature, E″ i is the fused attention weighted endmember token, Finally, the output of the endmember-guided Transformer encoder module is represented as: I i =EOT(E i ,X) Among them, I i is the endmember-guided feature of the i-th endmember; EOT(*) is the function of the endmember-guided Transformer encoder module; The endmember-guided Transformer encoder module outputs a total of P endmember-guided features, namely: I1,…,I P .

5. The hyperspectral image unmixing method based on endmember-guided Transformer and endmember bundle according to claim 4 is characterized in that: The method performs low-redundancy attention mechanism calculation on the endmember tokens and data feature graphs after the endmember-guided projection to obtain fused attention-weighted endmember tokens; including: The low-redundancy attention mechanism is calculated for the endmember tokens and data feature maps after the endmember-guided projection, and the formula is: Q=E′ i W q ,K=X′W k ,V=X′W v head1,…,head h =Split([Q,K,V]) Among them, W q , W k , W v are the projection matrices of the query matrix Q, key matrix K, and value matrix V respectively; head i It represents the i-th attention head after the data in the data feature map after the end-member guided projection is split, and there are h attention heads in total; Split([Q,K,V]) is the attention head splitting function; Softmax is the activation function; head′ i is the i-th attention head after attention weighting; Q i is the query matrix of the i-th attention head; K i T is the transpose of the key matrix of the i-th attention head; V i is the value matrix of the i-th attention head; Then, the spectral information divergence is used to measure the similarity between the attention heads after attention weighting. The calculation formula is expressed as: Among them, Sim(i,j) is head′ i and head′ j Similarity between i is the i-th attention head after attention weighting; head′ j is the jth attention head after attention weighting; Based on the calculation of the similarity between all attention heads after weighting, the similarity measurement matrix is ​​obtained Among them, h is the number of attention heads; The importance of each attention-weighted attention head is measured according to the similarity measurement matrix, and the weighted sum is calculated to obtain the fused attention-weighted end-member token. The calculation formula is: g = 1-Softmax(FC(Sim)) Among them, g represents the gating weight of the attention head after attention weighting, FC(*) represents the function of the fully connected network layer, E″ o is the fused attention weighted endmember token, g i is the gating weight of the i-th attention head after attention weighting.

6. The hyperspectral image unmixing method based on endmember-guided Transformer and endmember bundle according to claim 3, characterized in that: The endmember generator module based on endmember bundles converts the endmember guidance features into intra-class weights of the endmember bundles, and then weights the endmember bundles based on the intra-class weights to obtain the final endmember curve, including: The two-dimensional convolutional layer and the Softmax activation function are used to transform the endmember-guided features into the intra-class weights of the endmember bundles. The calculation formula is: I weight,i =Softmax(Conv(I i )) Among them, I weight,i is the intra-class weight of the i-th endmember bundle corresponding to each pixel in the image; I weight,i ∈ H is the height of the hyperspectral image, W is the width of the hyperspectral image, and K is the number of spectra in the endmember beam; Then, each endmember bundle is weighted based on the intra-class weight to obtain the final endmember curve. The calculation formula is: M i =I weight,i ·EM i M=Concat(M1,...,M P ) Among them, EM i is the i-th endmember bundle among the P endmember bundles, M i is the i-th endmember corresponding to each position in the hyperspectral image, M is the final end member curve, Concat(*) is a concatenation function.

7. The hyperspectral image unmixing method based on endmember-guided Transformer and endmember bundle according to claim 4, characterized in that: The heterogeneous information fusion module fuses the end-member guidance features from the perspective of spatial complementarity and channel interaction to obtain heterogeneous information fusion features; including: First, the end member is directed to feature I i Rearrange into a three-dimensional tensor I i ′, Generate a spatial mask based on two-dimensional convolution and Sigmoid activation function, the formula is: m i =Sigmoid(Conv(I i ′)) m j→i =Sigmoid(Conv(I j ′)) Among them, m i For I i ′ contains a mask with valuable spatial information, m h→u isI j 'Fusion I i ′, a mask with valuable spatial information; Then, based on the complement of the endmember-guided features of other categories, the endmember-guided features of the current category are enhanced to obtain the spatial fusion features of the current category. The specific formula is: Among them, I″ i is the spatial fusion feature of the i-th endmember, ⊙ represents Hadamard; Perform spatial fusion enhancement on each end-member guided feature to obtain P spatial fusion features I″1,…,I″ P ; Then all the spatial fusion features are spliced ​​along the channel dimension to obtain the spliced ​​spatial fusion features; the formula is expressed as: I spa =Concat(I″1,…,I″ P ) Among them, I spa is the spatial fusion feature after splicing, Finally, based on the channel rearrangement layer and the grouped convolution layer, the spliced ​​spatial fusion features are fused in the channel dimension to obtain the heterogeneous information fusion features. The formula is expressed as: I fuse =GConv(CS(I spa )) Among them, I fuse is the heterogeneous information fusion feature, CS(*) is the function of the channel reordering layer, and GConv(*) is the function of the grouped convolutional layer.

8. The hyperspectral image unmixing method based on endmember-guided Transformer and endmember bundle according to claim 3 is characterized in that: The abundance decoder module obtains an abundance map based on heterogeneous information fusion features, including: The abundance decoder module obtains the abundance map through a multi-layer neural network and based on heterogeneous information fusion features, and the formula is expressed as: A=Softmax(Conv(GELU(Conv(I fuse )))) Where A is the abundance map estimated by the abundance decoder model, GELU and Softmax represent two nonlinear activation functions, respectively, and Conv(*) represents the function of the two-dimensional convolutional layer.

9. The hyperspectral image unmixing method based on endmember-guided Transformer and endmember bundle according to claim 1, characterized in that: In the step 3), The reconstructed root mean square error and the reconstructed spectral angle distance are used as loss functions, and the weights of the neural network are updated based on a gradient descent method with adaptively adjusted learning rate.

10. A hyperspectral image unmixing system based on endmember-guided Transformer and endmember bundle for implementing the method of claim 1, characterized in that: include: An image preprocessing module, which is used to obtain the initial endmembers of the hyperspectral image and obtain the data feature map and endmember tokens; A neural network building module, which is used to construct a neural network based on an endmember-guided Transformer and an endmember bundle, wherein the neural network is used to obtain an abundance estimation result and an endmember extraction result of a hyperspectral image based on a data feature map and an endmember token, and the neural network includes an endmember-guided Transformer encoder module, an endmember generator module based on an endmember bundle, a heterogeneous information fusion module, and an abundance decoder module; A neural network training module, which is used to train a neural network based on an endmember-guided Transformer and an endmember bundle to obtain a trained neural network; The hyperspectral image unmixing module is used to obtain the abundance estimation result and endmember extraction result of the hyperspectral image to be unmixed by using the trained neural network, so as to realize the unmixing of the hyperspectral image.

Images