Hyperspectral image demixing method and device, electronic equipment and computer program product

Abstract

The invention discloses a hyperspectral image demixing method and device, electronic equipment and a computer program product, and relates to the technical field of spectral image demixing. Inputting the hyperspectral image data into a hyperspectral unmixing deep learning model to perform local spatial-spectral feature down-sampling and global spectral information perception to obtain down-sampled local spatial-spectral features and a global spectral feature map; stacking the local spatial-spectral features and the global spectral feature map, and decoding to obtain a plurality of end member spectral signals; the hyperspectral unmixing deep learning model comprises a local spatial spectrum feature down-sampling branch and a global spectral information sensing network branch; the local spatial-spectral feature down-sampling branch is used for local spatial-spectral feature extraction and down-sampling; and the global spectral information sensing network branch is used for calculating correlation weights among different spectral channels and fusing space attention weights from the convolution branch. According to the method, global spectrum sensing can be realized, the utilization rate of hyperspectral information is remarkably improved, and the unmixing effect of a spectral image is improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of spectral image unmixing, and particularly relates to a hyperspectral image unmixing method and device, an electronic device and a computer program product. BACKGROUND

[0002] Due to the correlation between ground objects and ground objects and the correlation between spectral bands in hyperspectral images, there is a large amount of redundancy in the spectral information contained in the hyperspectral images, so it is necessary to unmix the hyperspectral images.

[0003] At present, for the unmixing of hyperspectral images, the commonly used way is to unmix through a single convolutional neural network or a sampling Transformer architecture model, however, the unmixing method using a single convolutional neural network is insufficient in utilizing spectral information, and it is difficult to capture long-range dependencies in the spectral dimension, which can easily lead to loss of spectral details in the down-sampling process, ultimately resulting in poor spectral unmixing effect. The Transformer architecture model relies on the self-attention mechanism, and has a greater advantage than the convolutional neural network in capturing long-range relationships, which can assign different attention weights to the spectral sequence to achieve global spectral perception, but it also has the disadvantage of insufficient ability to capture local spatial-spectral features, which can also lead to poor spectral unmixing effect.

[0004] Therefore, how to provide an effective solution to improve the unmixing effect of spectral images has become a problem to be solved in the prior art. SUMMARY

[0005] The purpose of the present application is to provide a hyperspectral image unmixing method, device, electronic device and computer program product to solve the above problems existing in the prior art.

[0006] In order to achieve the above purpose, the present application adopts the following technical solutions: In a first aspect, the present application provides a hyperspectral image unmixing method, comprising: inputting the hyperspectral image data to be unmixing into a pre-trained hyperspectral unmixing deep learning model for local spatial-spectral feature down-sampling and global spectral information perception, to obtain down-sampled local spatial-spectral features Y and global spectral feature maps Xout; stacking and decoding the down-sampled local spatial-spectral features Y and the global spectral feature maps Xout to obtain a plurality of endmember spectral signals; The hyperspectral unmixing deep learning model comprises a local spatial-spectral feature down-sampling branch based on a convolutional neural network and a global spectral information perception network branch based on a self-attention mechanism. The local spatial-spectral feature downsampling branch based on the convolutional neural network is used for local spatial-spectral feature extraction and local spatial-spectral feature downsampling of the hyperspectral image data. The global spectral information perception network branch based on the self-attention mechanism is used for extracting self-attention SA between spectral channels in the hyperspectral image data, determining spatial attention weight Xp of each spectral channel based on the local spatial-spectral feature of the hyperspectral image data, determining a value matrix Vspa after spatial attention is given based on the spatial attention weight Xp of each spectral channel and a feature matrix obtained by linear projection of the hyperspectral image data in the self-attention extraction process, and linearly combining the value matrix Vspa after spatial attention is given and the self-attention SA between spectral channels to obtain the global spectral feature map Xout.

[0007] In a possible design, the local spatial-spectral feature downsampling branch based on the convolutional neural network includes a first convolutional layer of 1×1, a second convolutional layer of 3×3, and a third convolutional layer of 1×1, the first convolutional layer is used for feature extraction of the hyperspectral image data to obtain first features, the second convolutional layer is used for feature extraction of the first features to obtain the local spatial-spectral features, and the third convolutional layer is used for feature extraction of the local spatial-spectral features to obtain the down-sampled local spatial-spectral features.

[0008] In a possible design, the global spectral information perception network branch based on the self-attention mechanism includes a global spectral perception module and a convolutional network local spatial-spectral feature transmission module. The convolutional network local spatial-spectral feature transmission module is used for determining spatial attention weight Xp of each spectral channel based on the local spatial-spectral feature of the hyperspectral image data. The global spectral perception module is used for extracting self-attention SA between spectral channels in the hyperspectral image data, determining a value matrix Vspa after spatial attention is given based on the spatial attention weight Xp of each spectral channel and a feature matrix obtained by linear projection of the hyperspectral image data in the self-attention extraction process, and linearly combining the value matrix Vspa after spatial attention is given and the self-attention SA between spectral channels to obtain the global spectral feature map Xout.

[0009] In a possible design, the convolutional network local spatial-spectral feature transmission module includes a fourth convolutional layer of 1×1 and a fifth convolutional layer of 5×5, and the convolutional network local spatial-spectral feature transmission module is used for determining spatial attention weight Xp of each spectral channel in the following manner: The fourth convolutional layer is used for feature extraction of the local spatial-spectral feature of the hyperspectral image data to obtain second features. The third feature is obtained by extracting features from the second feature through the fifth convolutional layer; The third feature is reconstructed into features of multiple spectral channels, and the spatial attention weights Xp of each spectral channel are obtained.

[0010] In one possible design, the global spectral sensing module is used to determine the global spectral feature map Xout in the following manner: The spectral image data of each spectral channel in the hyperspectral image data is taken as a spectral data block to obtain a spectral data block sequence; The spectral data block sequence is linearly projected using three different matrices to obtain the query matrix Q, key matrix K, and value matrix V for each spectral channel; The query matrix Q, key matrix K, and value matrix V of each spectral channel are all divided into multiple attention heads along the spectral band dimension; Calculate the self-attention of each attention head in each spectral channel, and determine the self-attention (SA) between spectral channels in the hyperspectral image data based on the self-attention of each attention head in each spectral channel; The spatial attention weights Xp of each spectral channel are multiplied element-wise with the value matrices V of each spectral channel to obtain the value matrix Vspa after spatial attention is applied. The global spectral feature map Xout is obtained by linearly combining the value matrix Vspa after spatial attention is applied with the self-attention SA between the spectral channels.

[0011] In one possible design, the query matrix Q, key matrix K, and value matrix V of the spectral channels are: ; Wherein, X represents a spectral data block in the spectral data block sequence. , and Both represent matrix parameters.

[0012] In one possible design, the stacking and decoding of the downsampled local spatial spectral feature Y with the global spectral feature map Xout yields multiple endmember spectral signals, including: The downsampled local spatial spectral features Y and the global spectral feature map Xout are stacked and then extracted through a convolutional layer. The features extracted from the convolutional layer are activated into an abundance map using a non-linear activation function. The abundance map was decoded into a hyperspectral image by linear mapping, resulting in multiple endmember spectral signals.

[0013] In a second aspect, the present invention provides a hyperspectral image demixing apparatus, comprising: The feature sampling and information perception unit is used to input the hyperspectral image data to be unmixed into a pre-trained hyperspectral unmixing deep learning model to perform local spatial spectral feature downsampling and global spectral information perception, and obtain the downsampled local spatial spectral feature Y and global spectral feature map Xout. The stacking and decoding unit is used to stack the downsampled local spatial spectral feature Y with the global spectral feature map Xout and then decode them to obtain multiple endmember spectral signals. The hyperspectral unmixing deep learning model includes a local spatial spectral feature downsampling branch based on a convolutional neural network and a global spectral information perception network branch based on a self-attention mechanism. The local spatial spectral feature downsampling branch based on the convolutional neural network is used to extract and downsample the local spatial spectral features of the hyperspectral image data. The global spectral information sensing network branch based on the self-attention mechanism is used to extract the self-attention SA between spectral channels in the hyperspectral image data, determine the spatial attention weight Xp of each spectral channel based on the local spatial spectral features of the hyperspectral image data, determine the value matrix Vspa after spatial attention is applied based on the spatial attention weight Xp of each spectral channel and the feature matrix obtained by linear projection of the hyperspectral image data during the self-attention extraction process, and linearly combine the value matrix Vspa after spatial attention with the self-attention SA between spectral channels to obtain the global spectral feature map Xout.

[0014] Thirdly, the present invention provides an electronic device comprising a memory, a processor, and a transceiver connected in sequence and communication, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program and execute the hyperspectral image demixing method as described in the first aspect or any possible design of the first aspect.

[0015] Fourthly, the present invention provides a computer-readable storage medium storing instructions that, when executed on a computer, perform the hyperspectral image demixing method described in the first aspect or any possible design of the first aspect.

[0016] Fifthly, the present invention provides a computer program product containing instructions that, when executed on a computer, cause the computer to perform the hyperspectral image unmixing method as described in the first aspect or any possible design of the first aspect.

[0017] Beneficial effects: This invention employs a hyperspectral unmixing deep learning model comprising a local spatial spectral feature downsampling branch based on a convolutional neural network and a global spectral information perception network branch based on a self-attention mechanism. The local spatial spectral feature downsampling branch based on the convolutional neural network is used to extract and downsample local spatial spectral features from hyperspectral image data. The global spectral information perception network branch based on the self-attention mechanism is used to extract the self-attention (SA) between spectral channels in the hyperspectral image data, determine the spatial attention weights (Xp) of each spectral channel based on the local spatial spectral features of the hyperspectral image data, determine the value matrix (Vspa) after spatial attention is applied based on the feature matrix obtained by linear projection of the hyperspectral image data during the self-attention extraction process using the spatial attention weights (Xp) of each spectral channel and the feature matrix obtained by linear projection of the hyperspectral image data during the self-attention extraction process, and linearly combine the value matrix (Vspa) after spatial attention is applied with the self-attention (SA) between spectral channels to obtain the global spectral feature map (Xout). Thus, when demixing hyperspectral image data, the local spatial spectral features can be extracted using the local spatial spectral feature downsampling branch of the convolutional neural network, and the correlation weights between different spectral channels can be dynamically calculated using the global spectral information perception network branch based on the self-attention mechanism. The spatial attention weights from the convolutional branch are also fused to achieve global spectral modeling with spatial perception capabilities. This ensures spectral details while achieving global spectral perception, significantly improving the utilization rate of hyperspectral information, enhancing the demixing effect of spectral images, and facilitating practical applications and promotion. Attached Figure Description

[0018] Figure 1 A flowchart of the hyperspectral image demixing method provided in the embodiments of this application; Figure 2 This is a schematic diagram of the overall structure of the hyperspectral unmixing deep learning model provided in the embodiments of this application; Figure 3 A schematic diagram of the structure of a local spatial spectrum feature downsampling branch based on a convolutional neural network provided in an embodiment of this application; Figure 4 A schematic diagram of the structure of a global spectral information sensing network branch based on a self-attention mechanism provided in an embodiment of this application; Figure 5 A block diagram of the hyperspectral image demixing device provided in the embodiments of this application; Figure 6 This is a block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the present invention will be briefly introduced below in conjunction with the accompanying drawings and descriptions of the embodiments or the prior art. Obviously, the following description of the structure of the accompanying drawings is only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. It should be noted that the description of these embodiments is for the purpose of helping to understand the present invention, but does not constitute a limitation of the present invention.

[0020] It should be understood that although the terms first, second, etc., may be used herein to describe various units, these units should not be limited by these terms. These terms are only used to distinguish one unit from another. For example, a first unit may be referred to as a second unit, and similarly, a second unit may be referred to as a first unit, without departing from the scope of the exemplary embodiments of the invention.

[0021] It should be understood that the term "and / or" that may appear in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, B exists alone, and A and B exist simultaneously. The term " / and" that may appear in this document describes another relationship between related objects, indicating that two relationships can exist. For example, A / and B can mean: A exists alone, and A and B exist alone. In addition, the character " / " that may appear in this document generally indicates that the related objects before and after it are in an "or" relationship.

[0022] To improve the demixing effect of spectral images, embodiments of this application provide a hyperspectral image demixing method, apparatus, electronic device, and computer program product. This hyperspectral image demixing method, apparatus, electronic device, and computer program product can significantly improve the utilization rate of hyperspectral information and enhance the demixing effect of spectral images.

[0023] like Figure 1 As shown, the first aspect of the present application provides a hyperspectral image demixing method, which may include, but is not limited to, the following steps S101-S102.

[0024] Step S101. Input the hyperspectral image data to be unmixed into a pre-trained hyperspectral unmixing deep learning model to perform local spatial spectral feature downsampling and global spectral information perception, and obtain the downsampled local spatial spectral feature Y and global spectral feature map Xout.

[0025] Step S102. Stack the downsampled local spatial spectral feature Y with the global spectral feature map Xout and then decode to obtain multiple endmember spectral signals.

[0026] Please seeFigure 2 The hyperspectral unmixing deep learning model includes a local spatial spectral feature downsampling branch based on a convolutional neural network and a global spectral information perception network branch based on a self-attention mechanism. The local spatial spectral feature downsampling branch based on the convolutional neural network is used to extract and downsample local spatial spectral features from the hyperspectral image data. The global spectral information perception network branch based on the self-attention mechanism is used to extract the self-attention (SA) between spectral channels in the hyperspectral image data, determine the spatial attention weights (Xp) of each spectral channel based on the local spatial spectral features of the hyperspectral image data, determine the value matrix (Vspa) after applying spatial attention based on the spatial attention weights (Xp) and the feature matrix obtained by linear projection of the hyperspectral image data during the self-attention extraction process, and linearly combine the value matrix (Vspa) with the self-attention (SA) between spectral channels to obtain the global spectral feature map (Xout).

[0027] The local spatial spectral feature downsampling branch based on a convolutional neural network is used to extract and downsample local spatial spectral features from the hyperspectral image data. Please refer to [link / reference]. Figure 3 The local spatial spectrum feature downsampling branch based on convolutional neural network includes a 1×1 first convolutional layer, a 3×3 second convolutional layer, and a 1×1 third convolutional layer. The first convolutional layer is used to extract features from hyperspectral image data to obtain the first feature. The second convolutional layer is used to extract features from the first feature to obtain local spatial spectrum features. The third convolutional layer is used to extract features from the local spatial spectrum features to obtain downsampled local spatial spectrum features.

[0028] When demixing hyperspectral images, you can input the hyperspectral image data X to be demixed. R H×W×C Here, H and W represent the height and width of the hyperspectral image data, respectively, and C represents the number of bands. Feature extraction is then performed on the hyperspectral image data using a 1×1 first convolutional layer. This 1×1 first convolutional layer upscales the output features by N times, followed by batch normalization and Leaky ReLU activation to obtain the first feature. Next, a 3×3 second convolutional layer extracts features from the first feature, maintaining the same output feature map size, followed by batch normalization and Leaky ReLU activation to obtain local spatial spectral features. Simultaneously, a third convolutional layer can be used to extract features from the local spatial spectral features, followed by batch normalization and Leaky ReLU activation to obtain downsampled local spatial spectral features.

[0029] In this embodiment of the application, the first feature can be represented as: Where X represents the hyperspectral image data to be unmixed. This indicates that the hyperspectral image data is processed by convolution through a 1×1 first convolutional layer, and BN represents batch normalization. express Activation function. Local spatial spectral features can be represented as... ,in This indicates that the first feature is processed by a 3×3 second convolutional layer, and the downsampled local spatial spectral features can be represented as follows: ,in This indicates that the local spatial spectral features are processed by convolution through a 1×1 third convolutional layer.

[0030] Please see Figure 4 The global spectral information perception network branch based on the self-attention mechanism includes a global spectral perception module and a convolutional network local spatial spectral feature transfer module. The convolutional network local spatial spectral feature transfer module determines the spatial attention weights Xp of each spectral channel based on the local spatial spectral features of the hyperspectral image data. The global spectral perception module extracts the self-attention SA between spectral channels in the hyperspectral image data, determines the value matrix Vspa after applying spatial attention based on the spatial attention weights Xp of each spectral channel and the feature matrix obtained by linearly projecting the hyperspectral image data during the self-attention extraction process, and linearly combines the value matrix Vspa with the self-attention SA between spectral channels to obtain the global spectral feature map Xout.

[0031] Specifically, the local spatial spectral feature transfer module of the convolutional network includes a 1×1 fourth convolutional layer and a 5×5 fifth convolutional layer. This module can be used to determine the spatial attention weights Xp of each spectral channel as follows: The fourth convolutional layer extracts local spatial spectral features from the hyperspectral image data to obtain the second feature. The third feature is obtained by extracting features from the second feature through the fifth convolutional layer; The third feature is reconstructed into features of multiple spectral channels, and the spatial attention weights Xp of each spectral channel are obtained.

[0032] In this embodiment of the application, the spatial attention weight of each spectral channel can be expressed as: in This indicates batch normalization processing.

[0033] The global spectral sensing module can be used to determine the global spectral feature map Xout in the following manner: The spectral image data of each spectral channel in the hyperspectral image data is taken as a spectral data block to obtain a spectral data block sequence; The spectral data block sequence is linearly projected using three different matrices to obtain the query matrix Q, key matrix K, and value matrix V for each spectral channel; The query matrix Q, key matrix K, and value matrix V of each spectral channel are all divided into multiple attention heads along the spectral band dimension; Calculate the self-attention of each attention head in each spectral channel, and determine the self-attention (SA) between spectral channels in the hyperspectral image data based on the self-attention of each attention head in each spectral channel; The spatial attention weights Xp of each spectral channel are multiplied element-wise with the value matrices V of each spectral channel to obtain the value matrix Vspa after spatial attention is applied. The global spectral feature map Xout is obtained by linearly combining the value matrix Vspa after spatial attention is applied with the self-attention SA between the spectral channels.

[0034] Specifically, the hyperspectral image data can be X-rayed first. R H×W×C The spectral image data of each spectral channel is taken as a spectral data block, resulting in a spectral data block sequence X. R HW×C This operation specifically involves unfolding the two-dimensional hyperspectral image data into one-dimensional data within each channel. Subsequently, the sequence of spectral data blocks is linearly projected using three different learnable matrices to generate the query matrix Q, key matrix K, and value matrix V for each spectral channel required by the attention mechanism. The query matrix Q, key matrix K, and value matrix V for each spectral channel can be represented as: ; Wherein, X represents a spectral data block in the spectral data block sequence. , and Both represent matrix parameters.

[0035] After linear projection, the information contained in each spectral data block in the input sequence is linearly projected into a set of vector features. The feature vectors of different spectral data blocks are used to represent their positions in the entire feature distribution space. Therefore, the relationship between spectral data blocks is described through the relationship between feature vectors. To make the features more diverse and expressive, the query matrix Q, key matrix K, and value matrix V can be divided into multiple attention heads along the spectral channel (spectral band) dimension. Each attention head has a feature vector dimension of C / N. Self-attention is calculated for each head j according to the self-attention calculation method. Then, based on the self-attention of each attention head in each spectral channel, the self-attention SA between spectral channels in the hyperspectral image data is determined. The spatial attention weight Xp of each spectral channel is then element-wise multiplied with the value matrix V of each spectral channel to obtain the value matrix Vspa after spatial attention is applied. Finally, the value matrix Vspa after spatial attention is applied is linearly combined with the self-attention SA between spectral channels to obtain the global spectral feature map Xout.

[0036] The self-attention of each head j can be represented as: , ,in The attention weights between spectral data block i and spectral data block j in the spectral data block sequence are represented by softmax, which represents a non-linear activation function. K j This represents the j-th attention head in the key matrix K. This indicates the j-th attention head in the query matrix Q. This represents the j-th attention head in the value matrix V.

[0037] The value matrix Vspa after applying spatial attention can be represented as ,in This indicates element-wise multiplication.

[0038] In this embodiment of the application, the downsampled local spatial spectral feature Y is stacked with the global spectral feature map Xout and then decoded to obtain multiple endmember spectral signals. This may include, but is not limited to, the following steps: The downsampled local spatial spectral features Y and the global spectral feature map Xout are stacked and then extracted through a convolutional layer. The features extracted from the convolutional layer are activated into an abundance map using a non-linear activation function. The abundance map was decoded into a hyperspectral image by linear mapping, resulting in multiple endmember spectral signals.

[0039] This invention employs a hyperspectral unmixing deep learning model comprising a local spatial spectral feature downsampling branch based on a convolutional neural network and a global spectral information perception network branch based on a self-attention mechanism. The local spatial spectral feature downsampling branch based on the convolutional neural network is used to extract and downsample local spatial spectral features from hyperspectral image data. The global spectral information perception network branch based on the self-attention mechanism is used to extract the self-attention (SA) between spectral channels in the hyperspectral image data, determine the spatial attention weights (Xp) of each spectral channel based on the local spatial spectral features of the hyperspectral image data, determine the value matrix (Vspa) after spatial attention is applied based on the feature matrix obtained by linear projection of the hyperspectral image data during the self-attention extraction process using the spatial attention weights (Xp) of each spectral channel and the feature matrix obtained by linear projection of the hyperspectral image data during the self-attention extraction process, and linearly combine the value matrix (Vspa) after spatial attention is applied with the self-attention (SA) between spectral channels to obtain the global spectral feature map (Xout). Thus, when demixing hyperspectral image data, the local spatial spectral features can be extracted using the local spatial spectral feature downsampling branch of the convolutional neural network, and the correlation weights between different spectral channels can be dynamically calculated using the global spectral information perception network branch based on the self-attention mechanism. The spatial attention weights from the convolutional branch are also fused to achieve global spectral modeling with spatial perception capabilities. This ensures spectral details while achieving global spectral perception, significantly improving the utilization rate of hyperspectral information, enhancing the demixing effect of spectral images, and facilitating practical applications and promotion.

[0040] Please see Figure 5 The second aspect of this application provides a hyperspectral image demixing apparatus, which includes: The feature sampling and information perception unit is used to input the hyperspectral image data to be unmixed into a pre-trained hyperspectral unmixing deep learning model to perform local spatial spectral feature downsampling and global spectral information perception, and obtain the downsampled local spatial spectral feature Y and global spectral feature map Xout. The stacking and decoding unit is used to stack the downsampled local spatial spectral feature Y with the global spectral feature map Xout and then decode them to obtain multiple endmember spectral signals. The hyperspectral unmixing deep learning model includes a local spatial spectral feature downsampling branch based on a convolutional neural network and a global spectral information perception network branch based on a self-attention mechanism. The local spatial spectral feature downsampling branch based on the convolutional neural network is used to extract and downsample the local spatial spectral features of the hyperspectral image data. The global spectral information sensing network branch based on the self-attention mechanism is used to extract the self-attention SA between spectral channels in the hyperspectral image data, determine the spatial attention weight Xp of each spectral channel based on the local spatial spectral features of the hyperspectral image data, determine the value matrix Vspa after spatial attention is applied based on the spatial attention weight Xp of each spectral channel and the feature matrix obtained by linear projection of the hyperspectral image data during the self-attention extraction process, and linearly combine the value matrix Vspa after spatial attention with the self-attention SA between spectral channels to obtain the global spectral feature map Xout.

[0041] The working process, working details and technical effects of the hyperspectral image demixing device provided in the second aspect of this embodiment can be found in the first aspect of the embodiment, and will not be repeated here.

[0042] like Figure 6 As shown, a third aspect of this application provides an electronic device, including a memory, a processor, and a transceiver that are sequentially and communicatively connected, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program and execute the hyperspectral image demixing method as described in the first aspect of the embodiment.

[0043] Specifically, the memory may include, but is not limited to, random access memory (RAM), read-only memory (ROM), flash memory, first-in-first-out (FIFO) memory, and / or last-in-first-out (FILO) memory, etc.; the processor may not be limited to microprocessors of the STM32F105 series, ARM (Advanced RISC Machines), x86 architecture processors, or processors with integrated NPU (neural-network processing units); the transceiver may be, but is not limited to, WiFi (Wireless Fidelity) wireless transceivers, Bluetooth wireless transceivers, General Packet Radio Service (GPRS) wireless transceivers, ZigBee (a low-power LAN protocol based on the IEEE 802.15.4 standard), 3G transceivers, 4G transceivers, and / or 5G transceivers, etc.

[0044] This fourth aspect of the embodiment provides a computer-readable storage medium storing instructions comprising the hyperspectral image demixing method described in the first aspect of the embodiment. Specifically, the computer-readable storage medium stores instructions that, when executed on a computer, perform the hyperspectral image demixing method as described in the first aspect. The computer-readable storage medium refers to a data storage medium, which may include, but is not limited to, floppy disks, optical disks, hard disks, flash memory, USB flash drives, and / or Memory Sticks. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0045] The fifth aspect of this embodiment provides a computer program product containing instructions that, when executed on a computer, cause the computer to perform the hyperspectral image demixing method as described in the first aspect of the embodiment, wherein the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

[0046] Finally, it should be noted that the above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for unmixing hyperspectral images, characterized in that, include: The hyperspectral image data to be unmixed is input into a pre-trained hyperspectral unmixing deep learning model to perform local spatial spectral feature downsampling and global spectral information perception, resulting in the downsampled local spatial spectral feature Y and global spectral feature map Xout. The downsampled local spatial spectral feature Y is stacked with the global spectral feature map Xout and then decoded to obtain multiple endmember spectral signals; The hyperspectral unmixing deep learning model includes a local spatial spectral feature downsampling branch based on a convolutional neural network and a global spectral information perception network branch based on a self-attention mechanism. The local spatial spectral feature downsampling branch based on the convolutional neural network is used to extract and downsample the local spatial spectral features of the hyperspectral image data. The global spectral information sensing network branch based on the self-attention mechanism is used to extract the self-attention SA between spectral channels in the hyperspectral image data, determine the spatial attention weight Xp of each spectral channel based on the local spatial spectral features of the hyperspectral image data, determine the value matrix Vspa after spatial attention is applied based on the spatial attention weight Xp of each spectral channel and the feature matrix obtained by linear projection of the hyperspectral image data during the self-attention extraction process, and linearly combine the value matrix Vspa after spatial attention with the self-attention SA between spectral channels to obtain the global spectral feature map Xout.

2. The hyperspectral image unmixing method according to claim 1, characterized in that, The local spatial spectral feature downsampling branch based on convolutional neural network includes a 1×1 first convolutional layer, a 3×3 second convolutional layer, and a 1×1 third convolutional layer. The first convolutional layer is used to extract features from the hyperspectral image data to obtain a first feature. The second convolutional layer is used to extract features from the first feature to obtain the local spatial spectral feature. The third convolutional layer is used to extract features from the local spatial spectral feature to obtain the downsampled local spatial spectral feature Y.

3. The hyperspectral image unmixing method according to claim 1, characterized in that, The global spectral information perception network branch based on the self-attention mechanism includes a global spectral perception module and a convolutional network local spatial spectral feature transfer module. The local spatial spectral feature transfer module of the convolutional network is used to determine the spatial attention weights Xp of each spectral channel based on the local spatial spectral features of the hyperspectral image data. The global spectral sensing module is used to extract the self-attention (SA) between spectral channels in the hyperspectral image data, the spatial attention weight (Xp) of each spectral channel, and the feature matrix obtained by linear projection of the hyperspectral image data during the self-attention extraction process to determine the value matrix (Vspa) after spatial attention is applied, and to linearly combine the value matrix (Vspa) after spatial attention with the self-attention (SA) between spectral channels to obtain the global spectral feature map (Xout).

4. The hyperspectral image unmixing method according to claim 3, characterized in that, The convolutional network local spatial spectral feature transfer module includes a 1×1 fourth convolutional layer and a 5×5 fifth convolutional layer. This module determines the spatial attention weights Xp for each spectral channel as follows: The fourth convolutional layer extracts local spatial spectral features from the hyperspectral image data to obtain the second feature. The third feature is obtained by extracting features from the second feature through the fifth convolutional layer; The third feature is reconstructed into features of multiple spectral channels, and the spatial attention weights Xp of each spectral channel are obtained.

5. The hyperspectral image unmixing method according to claim 3, characterized in that, The global spectral sensing module is used to determine the global spectral feature map Xout in the following manner: The spectral image data of each spectral channel in the hyperspectral image data is taken as a spectral data block to obtain a spectral data block sequence; The spectral data block sequence is linearly projected using three different matrices to obtain the query matrix Q, key matrix K, and value matrix V for each spectral channel; The query matrix Q, key matrix K, and value matrix V of each spectral channel are all divided into multiple attention heads along the spectral band dimension; Calculate the self-attention of each attention head in each spectral channel, and determine the self-attention (SA) between spectral channels in the hyperspectral image data based on the self-attention of each attention head in each spectral channel; The spatial attention weights Xp of each spectral channel are multiplied element-wise with the value matrices V of each spectral channel to obtain the value matrix Vspa after spatial attention is applied. The global spectral feature map Xout is obtained by linearly combining the value matrix Vspa after spatial attention is applied with the self-attention SA between the spectral channels.

6. The hyperspectral image unmixing method according to claim 5, characterized in that, The query matrix Q, key matrix K, and value matrix V for the spectral channels are: ； Wherein, X represents a spectral data block in the spectral data block sequence. , and Both represent matrix parameters.

7. The hyperspectral image unmixing method according to claim 1, characterized in that, The process of stacking the downsampled local spatial spectral feature Y with the global spectral feature map Xout and then decoding it yields multiple endmember spectral signals, including: The downsampled local spatial spectral features Y and the global spectral feature map Xout are stacked and then extracted through a convolutional layer. The features extracted from the convolutional layer are activated into an abundance map using a non-linear activation function. The abundance map was decoded into a hyperspectral image by linear mapping, resulting in multiple endmember spectral signals.

8. A hyperspectral image demixing device, characterized in that, include: The feature sampling and information perception unit is used to input the hyperspectral image data to be unmixed into a pre-trained hyperspectral unmixing deep learning model to perform local spatial spectral feature downsampling and global spectral information perception, and obtain the downsampled local spatial spectral feature Y and global spectral feature map Xout. The stacking and decoding unit is used to stack the downsampled local spatial spectral feature Y with the global spectral feature map Xout and then decode them to obtain multiple endmember spectral signals. The hyperspectral unmixing deep learning model includes a local spatial spectral feature downsampling branch based on a convolutional neural network and a global spectral information perception network branch based on a self-attention mechanism. The local spatial spectral feature downsampling branch based on the convolutional neural network is used to extract and downsample the local spatial spectral features of the hyperspectral image data. The global spectral information sensing network branch based on the self-attention mechanism is used to extract the self-attention SA between spectral channels in the hyperspectral image data, determine the spatial attention weight Xp of each spectral channel based on the local spatial spectral features of the hyperspectral image data, determine the value matrix Vspa after spatial attention is applied based on the spatial attention weight Xp of each spectral channel and the feature matrix obtained by linear projection of the hyperspectral image data during the self-attention extraction process, and linearly combine the value matrix Vspa after spatial attention with the self-attention SA between spectral channels to obtain the global spectral feature map Xout.

9. An electronic device, characterized in that, The device includes a memory, a processor, and a transceiver that are sequentially and communicatively connected. The memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program and execute the hyperspectral image demixing method as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or the instructions are executed by the computer, they implement the hyperspectral image demixing method as described in any one of claims 1 to 7.

Images