Hyperspectral unmixing method, computer device and readable storage medium

Abstract

The application discloses a hyperspectral unmixing method, a computer device and a readable storage medium, and belongs to the technical field of hyperspectral remote sensing image processing. The method comprises the following steps: obtaining a hyperspectral image to be unixed; calling a hyperspectral unmixing network model based on the hyperspectral image to be unixed, and obtaining an abundance prediction map and a non-negative prediction endmember matrix; wherein the encoder of the hyperspectral unmixing network model extracts multi-scale enhanced spatial-spectral features from the hyperspectral image to be unixed based on a cascaded multi-scale downsampling layer and a stacked spatial-spectral mixing expert Mamba block; the spatial-spectral mixing expert Mamba block dynamically models and fuses the spatial dimension and the spectral dimension of the hyperspectral image through a spatial mixing expert submodule and a spectral sharing expert submodule; the decoder performs feature fusion and upsampling on the spatial-spectral features to generate the abundance prediction map, and generates the non-negative endmember matrix from the deepest feature after upsampling. The application can improve the unmixing precision of the hyperspectral image.

Description

Technical Field

[0001] This application belongs to the field of hyperspectral remote sensing image processing technology, specifically relating to a hyperspectral demixing method, computer equipment, and readable storage medium. Background Technology

[0002] Hyperspectral images, while acquiring spatial information about ground features, can record the spectral reflectance of hundreds of consecutive narrow bands, providing a rich data foundation for fine-grained classification, target detection, and quantitative analysis of ground features. However, due to limitations in sensor spatial resolution and the complexity of the Earth's surface, many pixels in the images contain mixed spectra of multiple ground features, creating the "mixed pixel" problem. Hyperspectral unmixing aims to decompose each mixed pixel into a set of basic ground feature spectra, i.e., endmembers, and their corresponding proportions, i.e., abundance. This is a crucial step in achieving quantitative extraction of hyperspectral information.

[0003] The State Space Model (SSM), especially the Mamba architecture, provides a new technical approach for efficient long sequence modeling required for hyperspectral unmixing. Mamba, through a data-dependent selective scanning mechanism, can achieve global context modeling in linear complexity, overcoming the inherent limitations of CNNs and Transformers.

[0004] However, existing Mamba-based hyperspectral unmixing methods generally suffer from the following shortcomings: the single-branch structure makes it difficult to decouple spectral and spatial features, and they usually use a uniform serialization method to process spatial and spectral information simultaneously, without performing decoupled and refined modeling of spatial context and spectral curves. Summary of the Invention

[0005] The purpose of this application is to provide a hyperspectral demixing method, computer device, and readable storage medium that can improve the accuracy of demixing hyperspectral images.

[0006] To solve the above-mentioned technical problems, this application is implemented as follows: In a first aspect, embodiments of this application provide a hyperspectral demixing method, the method comprising: Obtain the hyperspectral image of the object to be unmixed; Based on the hyperspectral image to be unmixed, the hyperspectral unmixing network model is invoked to obtain the abundance prediction map and the non-negative prediction endmember matrix. The hyperspectral unmixing network model includes an encoder and a decoder. The encoder extracts multi-scale enhanced spatial-spectral features from the hyperspectral image to be unmixed based on cascaded multi-scale downsampling layers and stacked spatial-spectral mixing expert Mamba blocks. The spatial-spectral mixing expert Mamba blocks dynamically model and collaboratively fuse the spatial and spectral dimensions of the hyperspectral image to be unmixed through spatial mixing expert submodules and spectral sharing expert submodules, respectively. The decoder performs feature fusion and upsampling on the multi-scale enhanced spatial-spectral features to generate an abundance prediction map, and generates a non-negative endmember matrix from the deepest upsampled features.

[0007] In a second aspect, embodiments of this application provide a computer device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0008] Thirdly, embodiments of this application provide a computer-readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0009] Fourthly, embodiments of this application also provide a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0010] In this embodiment, the Mamba MoE (Mamba Hybrid Expert Model) is introduced into the field of hyperspectral unmixing. Specifically, a spatial-spectral hybrid expert Mamba architecture, comprising a spectral sharing expert and a spatial mixing expert, is constructed to achieve decoupled modeling of spatial context and spectral features. The spectral sharing expert captures long-range spectral dependencies using bidirectional selective state-space scanning, while the spatial mixing expert adaptively perceives the spatial morphology of ground features through dynamic routing and multi-directional scanning. This effectively solves the problems of spatial-spectral information coupling and weak characterization of irregular ground features in traditional methods, significantly improving unmixing accuracy. Attached Figure Description

[0011] Figure 1 This is a flowchart illustrating a hyperspectral demixing method provided in some embodiments of this application; Figure 2 This is a schematic diagram of the overall structure of the hyperspectral unmixing network model provided in some embodiments of this application; Figure 3 This is a schematic diagram of the internal structure of the Mamba block, a spatial spectrum mixing expert, provided in some embodiments of this application; Figure 4These are endmember visualization results of the unmixing method and comparison method provided by some embodiments of this application on the Samson dataset; Figure 5 These are abundance visualization results of the unmixing methods provided by some embodiments of this application on the Samson dataset; Figure 6 These are endmember visualization results of the unmixing method and comparison method provided by some embodiments of this application on the Jasper Ridge dataset; Figure 7 These are abundance visualization results of the unmixing methods provided by some embodiments of this application on the Jasper Ridge dataset; Figure 8 These are internal structural diagrams of a computer device provided in some embodiments of this application. Detailed Implementation

[0012] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0013] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0014] In one exemplary embodiment, this application proposes a hyperspectral demixing method. The hyperspectral demixing method provided by this application will be described in detail below with reference to the accompanying drawings and specific embodiments and application scenarios.

[0015] Reference Figure 1 The method includes steps 102-104. Wherein: Step 102: Obtain the hyperspectral image to be unmixed.

[0016] In this embodiment, the hyperspectral image data to be unmixed is denoted as... ,in, and These represent the spatial height and spatial width of the image, respectively. This indicates the number of spectral bands. The hyperspectral image can originate from an airborne hyperspectral imager, a spaceborne hyperspectral imager, or a ground-based hyperspectral imaging device.

[0017] Step 104: Based on the hyperspectral image to be unmixed, call the hyperspectral unmixing network model to obtain the abundance prediction map and the non-negative prediction endmember matrix.

[0018] In some embodiments, the hyperspectral image data to be demixed is input into a trained hyperspectral demixing network model. The encoder extracts multi-scale spatial-spectral joint enhancement features by downsampling at multiple scales and stacking Mixture of Experts Mamba Blocks (MMoE Blocks). The decoder progressively upsamples in a bottom-up manner and performs skip-connection fusion with the enhancement feature maps at the corresponding scales. The abundance prediction head outputs a target abundance map that satisfies the constraints of non-negativity and sum to 1. At the same time, the endmember generation head generates a globally shared target endmember matrix from the highest resolution fused features, thus completing the hyperspectral image demixing task.

[0019] Specifically, such as Figure 2 As shown, the hyperspectral unmixing network model includes an encoder and a decoder. The encoder extracts multi-scale enhanced spatial-spectral features from the hyperspectral image to be unmixed based on cascaded multi-scale downsampling layers and stacked MMoE Blocks. The MMoE Blocks dynamically model and collaboratively fuse the spatial and spectral dimensions of the hyperspectral image to be unmixed through a spatial mixing expert submodule and a spectral sharing expert submodule, respectively. The decoder performs feature fusion and upsampling on the multi-scale enhanced spatial-spectral features to generate an abundance prediction map, and generates a non-negative endmember matrix from the deepest upsampled feature.

[0020] In some embodiments, such as Figure 3 As shown, the cascaded multi-scale downsampling layer is used to extract multi-scale initial features; the stacked MMoE Block performs spatial-spectral joint enhancement on the initial features at each scale, outputting enhanced multi-scale spatial-spectral features. The MMoE Block comprises a first-layer normalization unit, a Spectral-Spatial Mixture of Experts (SSMoE) module, a second-layer normalization unit, and a feedforward neural network, connected sequentially. The input of the first-layer normalization unit is added to the output of the SSMoE module using a learnable first jump scaling factor, and the input of the second-layer normalization unit is added to the output of the feedforward neural network using a learnable second jump scaling factor, forming a double-residual structure. The SSMoE module is the core module of the MMoE Block.

[0021] The feedforward neural network includes a first fully connected layer, an activation function layer, and a second fully connected layer connected in sequence. The input features of the feedforward neural network are mapped to the hidden feature space by the first fully connected layer, and after being nonlinearly transformed by the activation function layer, they are mapped back to the original feature dimension by the second fully connected layer.

[0022] The multi-scale downsampling layer comprises three cascaded downsampling blocks, each downsampling block containing a convolutional kernel with a size of [missing information]. The two-dimensional convolutional layer, group normalization layer, modified linear unit ReLU activation function, and window are as follows: The average pooling layer; the multi-scale initial features are obtained through the following recursive process, i.e., formula (1): in, For index variables, , , The hyperspectral image to be demixed; For the first The initial feature map output by the downsampling block. This represents the two-dimensional convolutional layer. This indicates the group normalization layer. This represents the ReLU activation function. This represents the average pooling layer. It can be understood that the resulting layer... , , Its spatial dimensions decrease step by step, while the number of channels increases step by step.

[0023] The feedforward neural network is a multilayer perceptron (MLP), and the MMoEBlock uses the following formula (2) to... Perform spatial-spectral joint enhancement: in, for Spatial spectral features enhanced by the MMoE Block; Presentation layer normalization operation; This refers to the SSMoE module; This indicates an instance normalization operation; This refers to the MLP; , These represent the first jump scaling factor and the second jump scaling factor, respectively. This indicates a residual connection.

[0024] In some embodiments, the SSMoE module includes a parallel spectral sharing expert submodule and a spatial mixing expert submodule; the spectral sharing expert submodule is used to perform a bidirectional selective state-space scan along the spectral dimension of the output features of the first-layer normalization unit; the spatial mixing expert submodule is used to perform a selective state-space scan along the spatial dimension of the output features of the first-layer normalization unit through multi-directional spatial scanning and dynamic expert routing; the SSMoE module performs the following operations on its input features: The input feature map of the SSMoE module is processed by a convolution kernel with a size of [missing value]. The two-dimensional convolutional layer expands the number of channels to twice the original number of spectral channels and uniformly splits them into spatial branch features and spectral branch features along the channel dimension.

[0025] The dimension of the spectral branch feature is transposed from channel-first format to space-first format, sent to the spectral sharing expert submodule to obtain the spectral enhancement feature, and then transposed back to the channel-first format.

[0026] The spatial branching features are directly fed into the spatial hybrid expert submodule to obtain spatial enhancement features.

[0027] The spectral enhancement feature and the spatial enhancement feature are concatenated along the channel dimension, and then passed through a convolutional kernel with a size of [missing value]. A two-dimensional convolutional layer is used to compress the number of channels of the stitched features to the original number of spectral channels.

[0028] The concatenated features after channel compression are added pixel-by-pixel to the input feature map of the SSMoE module to achieve residual connection.

[0029] In some embodiments, the spectral sharing expert submodule performs the following operations on the spectral branch features of the spatially optimized format: The spectral branch features of the spatially optimized format are flattened along the spatial dimension and reshaped into a two-dimensional sequence; wherein each row of the two-dimensional sequence corresponds to a spatial pixel, the sequence length is the number of spectral bands, and the feature at each position is a scalar or a vector extended to the internal dimension by linear projection.

[0030] Local context aggregation is performed along the sequence length using one-dimensional convolution, followed by SiLU nonlinear activation to construct forward and reverse sequences, respectively. The forward and reverse sequences are then concatenated along the sequence batch dimension to obtain a bidirectional sequence.

[0031] Based on the learnable first linear projection, the first time-varying parameters required for the first selective state-space model are dynamically generated from the bidirectional sequence; wherein, the first time-varying parameters include a first time-varying state transition matrix, a first input projection matrix, and a first output projection matrix.

[0032] A selective scan operation is performed on the bidirectional sequence, and the underlying state evolution mechanism of this operation satisfies the following formulas (3) and (4): Among them, the bidirectional sequence of the first Input features at each sequence position; The first in the bidirectional sequence Hidden states at each sequence position; Represents element-wise product; , , The first and second parts of the bidirectional sequence are respectively the first and second parts. The first time-varying state transition matrix, the first input projection matrix, and the first output projection matrix for each sequence position; The first jump vector; The first in the bidirectional sequence The result of selective scanning of sequence positions.

[0033] The Through the learnable first system matrix The first time-varying state transition matrix, after being input adaptively, is discretized to obtain the following; and stated The first fundamental input projection matrix can be learned. and the first basic output projection matrix Each is generated by adaptive modulation after being subjected to linear projection of the input.

[0034] The results of the selective scan are split into forward passage results and reverse passage results.

[0035] After reversing the reverse passage result along the sequence length direction, it is added element-wise to the forward passage result, and then passed through layer normalization and linear projection layers to restore the original number of spectral channels, thereby obtaining the spectral enhancement feature.

[0036] In some embodiments, the spatial hybrid expert submodule includes multiple parallel spatial experts and a gating router for dynamic expert selection. The multiple parallel spatial experts and the gating router perform the following operations on the spatial branch features: The gated router performs global average pooling on the spatial branch features, compressing the spatial branch features of each channel into a scalar, and obtains the original scores of each spatial expert through a linear layer without a bias term. Then, it generates the routing weights of each spatial expert through a normalization function, as shown in formula (5) below: in, The route weight; This represents the normalization function. Let be the linear transformation weight matrix of the gated router; The spatial branching feature; Indicates the above The global average pooling operation is performed. Indicates the Mathematical symbols and tensor shapes, This represents the number of samples in the current batch. This represents the total number of space experts.

[0037] Multiple scanning direction representations are generated for the spatial branch features to obtain... Group orientation features, among which, The Group orientation features include at least: Versions that maintain the original row and column arrangement, versions that transpose the row and column arrangement, versions that flip the original row and column arrangement along the spatial dimension, and versions that flip the transpose of the row and column arrangement along the spatial dimension.

[0038] Each spatial expert is assigned to process one set of directional features and outputs expert features after state space scanning and gating enhancement.

[0039] The expert features are weighted and summed, and then the residuals of the sums are added to the spatial branch features to obtain the spatial enhancement features.

[0040] In some embodiments, during the training phase, all expert outputs are summed by routing weights; during the inference phase, the top k experts with the highest weights are selected, calculated, renormalized, and then weighted, and finally added to the residual of the input of the spatial hybrid expert submodule to obtain the spatial augmented features.

[0041] In some embodiments, the space experts perform the following operations on the corresponding directional features: The corresponding directional features are linearly projected along the channel dimension to split the corresponding directional features into spatial principal features and spatial gated features along the channel direction, wherein the number of channels of the spatial principal features is less than the number of channels of the original spectrum. The spatial main feature dimension is transposed to channel dimension priority, and local spatial patterns are extracted by grouped depthwise separable convolution and SiLU activation function, and then flattened into a one-dimensional sequence along the spatial dimension.

[0042] The second time-varying parameters required to generate a selective state-space model from the one-dimensional sequence based on learnable linear projection; wherein the second time-varying parameters include a second time-varying state transition matrix, a second input projection matrix, and a second output projection matrix; Based on the second time-varying parameter, and the learnable second system matrix and second jump vector, a selective scanning operation is performed on the one-dimensional sequence to obtain the global spatial context sequence.

[0043] The global spatial context sequence is reshaped into the original two-dimensional spatial form. After layer normalization, it is multiplied element-wise with the spatial gated features processed by the SiLU activation function. The number of channels is then restored to the original number of spectral channels through a linear projection layer to obtain the output feature map of the spatial expert.

[0044] In this embodiment, the linear complexity selective scanning and hybrid expert dynamic computation mechanism based on Mamba significantly reduces the computational complexity and memory usage of the model while maintaining high unmixing accuracy. It has the ability to process high-resolution hyperspectral images in real time and has good engineering application value.

[0045] In some embodiments, the decoder's fusion of the multi-scale enhanced spatial spectral features, the generation of the abundance prediction map and the non-negative endmember matrix are achieved through the following operations: From the third enhanced feature map at the deepest scale Begin, the above The first fused feature is obtained by inputting it into the corresponding feature fusion block. The feature fusion block consists of two identical residual convolutional units, each residual convolutional unit including a ReLU activation function connected in sequence. Convolution, ReLU activation and Convolution is performed, and the input and output of the residual convolution unit are added pixel by pixel.

[0046] Regarding the Perform bilinear upsampling to the second enhanced feature map The same spatial dimensions, and compared with those processed by the first residual convolution unit. The features are then added together and then passed through a second residual convolutional unit to obtain the second fused feature. .

[0047] Regarding the Perform bilinear upsampling to the first enhanced feature map The same spatial dimensions, and compared with those processed by the third residual convolution unit. The features are then added together and passed through the fourth residual convolutional unit to obtain the third fused feature. .

[0048] The After bilinear upsampling to the original input space size, the data is input to the abundance prediction head and normalized to obtain the abundance prediction map. The normalization operation can be implemented using the Softmax function, which is performed along the channel dimension to ensure that the abundance of a single pixel is non-negative and that its sum is 1.

[0049] The following formula (6) is used to describe the Perform global average pooling along the spatial dimension to obtain the global context vector. : in, For the Spatial height, For the Space width, The first in the current batch one sample , Used to index different samples in a batch.

[0050] The Input an endmember generator head to obtain log odds; wherein, the endmember generator head is composed of a fully connected layer, a ReLU activation function, and a cascaded fully connected layer.

[0051] The log-probability is passed through the Softplus function and averaged over the batch dimension to obtain a globally shared non-negative endmember matrix.

[0052] In the above embodiments, a bottom-up multi-level fusion and skip connection structure is adopted in the decoding stage to enhance the spatial detail recovery capability; at the same time, a globally shared non-negative endmember matrix is ​​generated from the fused high-resolution features, which conforms to the physical meaning of the linear mixture model and improves the interpretability of endmembers and reconstruction quality.

[0053] In some embodiments, the method further includes: Flatten the spatial dimensions of the abundance prediction map to obtain the abundance array.

[0054] Perform a batch matrix multiplication operation between the abundance array and the globally shared non-negative endmember matrix to obtain a flattened reconstructed spectral array.

[0055] The reconstructed spectral array is rearranged according to the spatial dimensions of the hyperspectral image to obtain a reconstructed hyperspectral image with a shape that is completely consistent with the hyperspectral image.

[0056] In some embodiments, during the training phase of the hyperspectral unmixing network model, the total loss of the multi-constraint loss function of the hyperspectral unmixing network model is obtained by weighted summation of spectral reconstruction loss, abundance physical constraint loss, abundance sparsity loss, abundance entropy loss, and endmember regularization loss through adjustable weight coefficients, as shown in formulas (7) to (12) below: The total loss Calculation formula (7): in, These are preset weighting coefficients used to adjust the relative contribution of each loss term.

[0057] The spectral reconstruction loss Calculation formula (8): in, Indicates the total number of pixels; This indicates the batch size of the hyperspectral images; Indicates the number of spectral bands; and These represent the height and width of the space, respectively. Represents the inverse cosine function; This refers to the reconstructed hyperspectral image; This represents the hyperspectral image; and The reconstructed hyperspectral image and the hyperspectral image respectively represent the first... spectral vector of pixels; A minimum constant preset to prevent the denominator from being zero; The Euclidean norm of a vector; Let Frobenius norm denote the matrix.

[0058] Abundance Physical Constraint Loss Calculation formula (9): in, The number of endmembers represents the hyperparameter of the hyperspectral unmixing network model. This represents the abundance prediction map; The abundance prediction graph represents the first... The abundance distribution matrix corresponding to each endmember; This indicates taking the maximum value. It can be understood that the abundance prediction graph includes the abundance distribution matrix corresponding to each endmember; the non-negative endmember matrix includes the spectral band vector corresponding to each endmember.

[0059] Abundance sparsity loss Calculation formula (10): in, This represents the operation of raising the absolute value of each element to the power of 0.5.

[0060] Ender regularization loss Calculation formula (11): in, Represents the first element in the non-negative endmember matrix. The terminal in the first The value of the band; The hyperspectral image indicates that the hyperspectral image is in the first... Spatial pixel mean of the band.

[0061] Abundance entropy loss calculation formula (12): in, In the first A sample in space Belongs to the first The abundance ratio of each endmember; To prevent small constants from logarithmic singularities.

[0062] This embodiment designs a multi-constraint composite loss function that integrates spectral reconstruction, abundance physical constraints, sparsity, abundance entropy, and endmember regularization. This ensures that the unmixing results strictly satisfy the physical laws of nonnegativity and summation to 1 under the linear mixture model. Furthermore, the sparsity of abundance is enhanced through entropy constraints, thereby improving the reliability of endmembers and abundance maps.

[0063] The following comparative experiments verify the effectiveness of the hyperspectral unmixing network model proposed in the embodiments of this application: This comparative experiment uses two publicly available datasets, Samson and Jasper Ridge, which are commonly used in the field of hyperspectral unmixing, for validation.

[0064] The Samson dataset was acquired by the SAMSON sensor. The original image spatial resolution is 952×952 pixels, the spectral coverage ranges from 401nm to 889nm, it contains 156 consecutive bands, and the spectral interval is 3.13nm.

[0065] This comparative experiment selected a 95×95 pixel sub-region as the experimental object. This sub-region contains three typical endmembers: soil, trees, and water, with a corresponding number of 3 endmembers.

[0066] The Jasper Ridge dataset was acquired by the AVIRIS hyperspectral sensor in the Jasper Ridge area of ​​California, USA.

[0067] The original image has a spatial resolution of 512×614 pixels, a spectral coverage range of 380nm to 2500nm, and contains 224 spectral channels with a spectral resolution of approximately 9.46nm. To reduce computational overhead and eliminate the effects of strong water vapor absorption and atmospheric interference, channels 1-3, 108-112, 154-166, and 220-224 were removed in this comparative experiment, retaining 198 effective channels. Subsequently, a 100×100 pixel sub-image was cropped from the (105,269) coordinate position of the original image as experimental data. This sub-region contains four typical endmembers: water, soil, trees, and roads, with a corresponding endmember count of 4.

[0068] This comparative experiment uses two evaluation metrics commonly used in the field of hyperspectral unmixing to quantitatively assess the model performance: (1) Spectral Angle Distance (SAD): This measure is used to measure the spectral similarity between the extracted endmembers and the true endmembers. The smaller the SAD value, the higher the accuracy of endmember extraction.

[0069] (2) Root Mean Square Error (RMSE): This measures the pixel-by-pixel deviation between the estimated abundance and the true abundance. The smaller the RMSE value, the higher the accuracy of the abundance estimation.

[0070] To verify the effectiveness of the embodiments of this application, the following representative comparison algorithms were selected: Deep Autoencoder Network (DAEN), Cycle-Consistent Unmixing Network (CyCUNet), Abundance-Guided Attention Network (A2SN), Multi-stage Convolutional Autoencoder Network (MuCAEU), Spatial-Spectral Adaptive Fusion (SSAF), and Cascaded Spatial-Spectral Mamba Unmixing Network (UNMamba).

[0071] The model training settings are as follows: In the Samson dataset, the Adam optimizer is used with an initial learning rate of 5e-4 and a weight decay factor of 1e-4. The weight coefficients are... The values ​​are 1.0, 0.1, 1e-6, 0.02, and 1e-2, respectively, with 500 training rounds.

[0072] Under these conditions, to comprehensively evaluate the unmixing performance and stability of the model, all experiments were repeated 10 times, and the final results were averaged. The endmember SAD evaluation indexes of the unmixing method provided in this embodiment and the comparative experimental methods are shown in Table 1 below.

[0073] Table 1: Comparison of different methods for endmember SAD in the Samson dataset.

[0074] Among them, the unmixed endmembers are: Soil (soil), Tree (tree), Water (water), and mSAD (mean spectral angular distance), which is used to average the SAD of the corresponding column endmember spectra.

[0075] The abundance RMSE is shown in Table 2 below: Table 2: Comparison of different methods for RMSE abundance in the Samson dataset.

[0076] Where mRMSE is the mean RMSE, which is used to average the RMSE of the abundance of the corresponding column endmembers.

[0077] In the Jasper Ridge dataset, the Adam optimizer is used with an initial learning rate of 5e-4 and a weight decay factor of 1e-4. The weight coefficients are... The values ​​were 1.0, 0.1, 1e-3, 0.01, and 1e-5, with 500 training epochs. Ten repeated experiments were conducted under these conditions, and the final result was the average. The endmember SAD evaluation metrics for the unmixing method provided in this embodiment and the comparative experimental method are shown in Table 3 below: Table 3: Comparison of different methods for endmember SAD on the Jasper Ridge dataset.

[0078] The abundance RMSE is shown in Table 4 below: Table 4: Comparison of different methods for RMSE of abundance in the Jasper Ridge dataset It should be noted that the bolded numbers in Tables 1 to 4 are the best for this row, while the underlined numbers are the second best for this row.

[0079] Figure 4 and Figure 5 The images show the endmember spectra and abundance comparisons between the hyperspectral unmixing method (OURS, ours) provided in this application and other methods, respectively, for the Samson dataset. In the endmember spectra, the black line represents the true reference endmember spectrum, and the red line represents the predicted endmember spectrum of each method. In the abundance images, GT (Ground Turth, true value) represents the true abundance reference image.

[0080] Figure 6 and Figure 7 The images show the endmember spectra and abundance comparisons between the Jasper Ridge dataset and various methods, respectively. In the endmember spectra, the black line represents the true reference endmember spectrum, and the red line represents the predicted endmember spectrum of the method. In the abundance images, GT represents the true abundance reference map.

[0081] From Tables 1 to 4 above and Figures 4 to 7 As can be seen, the endmember spectra extracted by OURS on the Samson and Jasper Ridge datasets are in high agreement with the real spectra. The abundance maps outperform the comparison methods in terms of ground cover edge preservation and detail restoration, demonstrating good physical consistency and spatial interpretability.

[0082] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0083] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 8 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a hyperspectral demixing method.

[0084] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0085] In one embodiment, a computer-readable storage medium is provided, on which a program or instructions are stored, which, when executed by a processor, implement the steps in the above-described method embodiments.

[0086] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0087] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0088] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0089] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A hyperspectral unmixing method, characterized in that, The hyperspectral demixing method includes: Obtain the hyperspectral image of the object to be unmixed; Based on the hyperspectral image to be unmixed, the hyperspectral unmixing network model is invoked to obtain the abundance prediction map and the non-negative prediction endmember matrix. The hyperspectral unmixing network model includes an encoder and a decoder. The encoder extracts multi-scale enhanced spatial-spectral features from the hyperspectral image to be unmixed based on cascaded multi-scale downsampling layers and stacked spatial-spectral mixing expert Mamba blocks. The spatial-spectral mixing expert Mamba blocks dynamically model and collaboratively fuse the spatial and spectral dimensions of the hyperspectral image to be unmixed through spatial mixing expert submodules and spectral sharing expert submodules, respectively. The decoder performs feature fusion and upsampling on the multi-scale enhanced spatial-spectral features to generate an abundance prediction map, and generates a non-negative endmember matrix from the deepest upsampled features.

2. The method according to claim 1, characterized in that, The cascaded multi-scale downsampling layers are used to extract multi-scale initial features; the stacked spatial-spectral mixing expert Mamba blocks perform spatial-spectral joint enhancement on the initial features at each scale, outputting enhanced multi-scale spatial-spectral features. The spatial-spectral mixing expert Mamba block includes a first-layer normalization unit, a spatial-spectral mixing expert SSMoE module, a second-layer normalization unit, and a feedforward neural network connected in sequence. The input of the first-layer normalization unit is added to the output of the SSMoE module through a learnable first jump scaling factor, and the input of the second-layer normalization unit is added to the output of the feedforward neural network through a learnable second jump scaling factor, forming a double residual structure. The feedforward neural network includes a first fully connected layer, an activation function layer, and a second fully connected layer connected in sequence; wherein, the input features of the feedforward neural network are mapped to the hidden layer feature space by the first fully connected layer, and after being nonlinearly transformed by the activation function layer, are mapped back to the original feature dimension by the second fully connected layer; The multi-scale downsampling layer comprises three cascaded downsampling blocks, each downsampling block containing a convolutional kernel with a size of [missing information]. The two-dimensional convolutional layer, group normalization layer, modified linear unit ReLU activation function, and window are as follows: The average pooling layer; the multi-scale initial features are obtained through the following recursive process: in, For index variables, , , The hyperspectral image to be demixed; For the first The initial feature map output by the downsampling block. This represents the two-dimensional convolutional layer. This indicates the group normalization layer. This represents the ReLU activation function. This refers to the average pooling layer; The feedforward neural network is a multilayer perceptron (MLP), and the spatial-spectral mixing expert Mamba block is configured using the following formula. Perform spatial-spectral joint enhancement: in, for Spatial spectral features enhanced by the aforementioned spatial spectral mixing expert Mamba block; Presentation layer normalization operation; This refers to the SSMoE module; This indicates an instance normalization operation; This refers to the MLP; , These represent the first jump scaling factor and the second jump scaling factor, respectively. This indicates a residual connection.

3. The method according to claim 2, characterized in that, The SSMoE module includes a parallel spectral sharing expert submodule and a spatial mixing expert submodule; the spectral sharing expert submodule is used to perform a bidirectional selective state-space scan along the spectral dimension of the output features of the first layer normalization unit. The spatial hybrid expert submodule is used to perform selective state space scanning along the spatial dimension of the output features of the first-layer normalization unit through multi-directional spatial scanning and dynamic expert routing; the SSMoE module performs the following operations on its input features: The input feature map of the SSMoE module is processed by a convolution kernel with a size of [missing value]. The two-dimensional convolutional layer expands the number of channels to twice the original number of spectral channels and uniformly splits them into spatial branch features and spectral branch features along the channel dimension; The dimension of the spectral branch feature is transposed from channel-first format to space-first format, sent to the spectral sharing expert submodule to obtain the spectral enhancement feature, and then transposed back to the channel-first format. The spatial branching features are directly fed into the spatial hybrid expert submodule to obtain spatial enhancement features; The spectral enhancement feature and the spatial enhancement feature are concatenated along the channel dimension, and then passed through a convolutional kernel with a size of [missing value]. Two-dimensional convolutional layers are used to compress the number of channels of the stitched features to the original number of spectral channels; The concatenated features after channel compression are added pixel-by-pixel to the input feature map of the SSMoE module to achieve residual connection.

4. The method according to claim 3, characterized in that, The spectral sharing expert submodule performs the following operations on the spectral branch features of the spatially optimized format: The spectral branch features of the spatially optimized format are flattened along the spatial dimension and reshaped into a two-dimensional sequence; wherein each row of the two-dimensional sequence corresponds to a spatial pixel, the sequence length is the number of spectral bands, and the feature at each position is a scalar or a vector after being extended to the internal dimension by linear projection. Local context aggregation is performed along the sequence length using one-dimensional convolution, and after SiLU nonlinear activation, a forward sequence and a reverse sequence are constructed respectively. The forward sequence and the reverse sequence are then concatenated along the sequence batch dimension to obtain a bidirectional sequence. Based on the learnable first linear projection, the first time-varying parameters required for the first selective state-space model are dynamically generated from the bidirectional sequence; wherein, the first time-varying parameters include a first time-varying state transition matrix, a first input projection matrix, and a first output projection matrix; A selective scan operation is performed on the bidirectional sequence, and the underlying state evolution mechanism of this operation satisfies the following formula: in, The first in the bidirectional sequence Input features at each sequence position; The first in the bidirectional sequence Hidden states at each sequence position; Represents element-wise product; , , The first and second parts of the bidirectional sequence are respectively the first and second parts. The first time-varying state transition matrix, the first input projection matrix, and the first output projection matrix for each sequence position; The first jump vector; The first in the bidirectional sequence The result of selective scanning of sequence positions; The Through the learnable first system matrix The first time-varying state transition matrix, after being input adaptively, is discretized to obtain the following; and stated The first fundamental input projection matrix can be learned. and the first basic output projection matrix Each is generated after adaptive modulation via the linear projection of the input; The results of the selective scan are divided into forward passage results and reverse passage results; After reversing the reverse passage result along the sequence length direction, it is added element-wise to the forward passage result, and then passed through layer normalization and linear projection layers to restore the original number of spectral channels, thereby obtaining the spectral enhancement feature.

5. The method according to claim 3, characterized in that, The spatial hybrid expert submodule comprises multiple parallel spatial experts and a gating router for dynamic expert selection. The multiple parallel spatial experts and the gating router perform the following operations on the spatial branch features: The gated router performs global average pooling on the spatial branch features, compressing the spatial branch features of each channel into a scalar, and obtains the raw scores of each spatial expert through a linear layer without a bias term. Then, it generates the routing weights of each spatial expert through a normalization function, as shown in the following formula: in, The route weight; This represents the normalization function. Let be the linear transformation weight matrix of the gated router; The spatial branching feature; Indicates the above The global average pooling operation is performed. Indicates the Mathematical symbols and tensor shapes, This represents the number of samples in the current batch. The total number of space experts; Multiple scanning direction representations are generated for the spatial branch features to obtain... Group orientation features, among which, The Group orientation features include at least: Versions that preserve the original row and column arrangement, versions that transpose the row and column arrangement, versions that flip the original row and column arrangement along the spatial dimension, and versions that flip the transpose of the row and column arrangement along the spatial dimension. Each spatial expert is assigned to process one set of directional features and outputs expert features after state space scanning and gating enhancement. The expert features are weighted and summed, and then the residuals of the sums are added to the spatial branch features to obtain the spatial enhancement features.

6. The method according to claim 5, characterized in that, Each of the aforementioned space experts performs the following operations on the corresponding directional features: The corresponding directional features are linearly projected along the channel dimension to split the corresponding directional features into spatial principal features and spatial gated features along the channel direction, wherein the number of channels of the spatial principal features is less than the number of channels of the original spectrum. The spatial main feature is transposed to channel dimension priority, and local spatial patterns are extracted by grouped depthwise separable convolution and SiLU activation function, and then flattened into a one-dimensional sequence along the spatial dimension. The second time-varying parameters required to generate a selective state-space model from the one-dimensional sequence based on learnable linear projection; wherein the second time-varying parameters include a second time-varying state transition matrix, a second input projection matrix, and a second output projection matrix; Based on the second time-varying parameter, and the learnable second system matrix and second jump vector, a selective scanning operation is performed on the one-dimensional sequence to obtain the global spatial context sequence; The global spatial context sequence is reshaped into the original two-dimensional spatial form. After layer normalization, it is multiplied element-wise with the spatial gated features processed by the SiLU activation function. The number of channels is then restored to the original number of spectral channels through a linear projection layer to obtain the output feature map of the spatial expert.

7. The method according to claim 1, characterized in that, The decoder's fusion of the multi-scale enhanced spatial spectral features, and the generation of the abundance prediction map and non-negative endmember matrix, are achieved through the following operations: From the third enhanced feature map at the deepest scale Begin, the above The first fused feature is obtained by inputting it into the corresponding feature fusion block. The feature fusion block consists of two identical residual convolutional units, each residual convolutional unit including a ReLU activation function connected in sequence. Convolution, ReLU activation and Convolution is performed, and the input and output of the residual convolution unit are added pixel by pixel; Regarding the Perform bilinear upsampling to the second enhanced feature map The same spatial dimensions, and compared with those processed by the first residual convolution unit. The features are then added together and then passed through a second residual convolutional unit to obtain the second fused feature. ; Regarding the Perform bilinear upsampling to the first enhanced feature map The same spatial dimensions, and compared with those processed by the third residual convolution unit. The features are then added together and passed through the fourth residual convolutional unit to obtain the third fused feature. ; The After bilinear upsampling to the original input spatial size, the data is input to the abundance prediction head and normalization is performed to obtain the abundance prediction map. Regarding the Perform global average pooling along the spatial dimension to obtain the global context vector. : in, For the Spatial height, For the Space width, The first in the current batch one sample , Used to index different samples in a batch; The Input an endmember generator head to obtain log odds; wherein, the endmember generator head is composed of a fully connected layer, a ReLU activation function, and a cascaded fully connected layer; The log-probability is passed through the Softplus function and averaged over the batch dimension to obtain a globally shared non-negative endmember matrix.

8. The method according to claim 7, characterized in that, The method further includes: Flatten the spatial dimensions of the abundance prediction map to obtain an abundance array; Perform a batch matrix multiplication operation between the abundance array and the globally shared non-negative endmember matrix to obtain a flattened reconstructed spectral array. The reconstructed spectral array is rearranged according to the spatial dimensions of the hyperspectral image to obtain a reconstructed hyperspectral image with a shape that is completely consistent with the hyperspectral image; During the training phase of the hyperspectral unmixing network model, the total loss of the multi-constraint loss function of the hyperspectral unmixing network model is obtained by weighting and summing the spectral reconstruction loss, abundance physical constraint loss, abundance sparsity loss, abundance entropy loss, and endmember regularization loss using adjustable weight coefficients, as shown in the following formula: The total loss Calculation formula: in, These are preset weighting coefficients used to adjust the relative contribution of each loss term; The spectral reconstruction loss Calculation formula: in, Indicates the total number of pixels; This indicates the batch size of the hyperspectral images; Indicates the number of spectral bands; and These represent the height and width of the space, respectively. Represents the inverse cosine function; This refers to the reconstructed hyperspectral image; This represents the hyperspectral image; and The reconstructed hyperspectral image and the hyperspectral image respectively represent the first... spectral vector of pixels; A minimum constant preset to prevent the denominator from being zero; The Euclidean norm of a vector; Denotes the Frobenius norm of a matrix; Abundance Physical Constraint Loss Calculation formula: in, Indicates the number of endmembers; This represents the abundance prediction map; The abundance prediction graph represents the first... The abundance distribution matrix corresponding to each endmember; This indicates taking the maximum value; Abundance sparsity loss Calculation formula: in, This represents the operation of raising the absolute value of each element to the power of 0.

5. Ender regularization loss Calculation formula: in, Represents the first element in the non-negative endmember matrix. The terminal in the first The value of the band; The hyperspectral image indicates that the hyperspectral image is in the first... Spatial pixel mean of the band; Abundance entropy loss calculation formula: in, In the first A sample in space Belongs to the first The abundance ratio of each endmember; To prevent small constants from logarithmic singularities.

9. A computer device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the hyperspectral demixing method as described in any one of claims 1-8.

10. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the hyperspectral demixing method as described in any one of claims 1-8.

Images