Hyperspectral Image Sparse Unmixing Method, Device, Electronic Device and Medium

Abstract

The present invention provides a hyperspectral image sparse unmixing method, apparatus, electronic device and medium. The method includes: extracting features from an original hyperspectral image to obtain multi-scale spatial features and spectral features; splicing the multi-scale spatial features and spectral features to obtain an original spatial-spectral joint feature; decoupling the features of the original spatial-spectral joint feature to obtain an enhanced spatial-spectral joint feature; normalizing the enhanced spatial-spectral joint feature according to a preset weight matrix to obtain a preliminary abundance matrix; iteratively correcting the preliminary abundance matrix and the preset weight matrix through a loss function in combination with a neural network to obtain a sparse abundance matrix and an endmember matrix; and reconstructing the sparse abundance matrix and the endmember matrix to obtain a reconstructed hyperspectral image. The present invention realizes a double improvement in the accuracy and robustness of hyperspectral image sparse unmixing through the design of a multi-scale spatial-spectral joint autoencoder network and a regularization processing strategy.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular, to a hyperspectral image sparse unmixing method, apparatus, electronic device, and medium. Background Art

[0002] Hyperspectral image technology, as an advanced imaging means, uses a spectrometer to image a target object within a wide and continuous spectral range, capable of capturing and recording rich spectral information. This technology provides reliable data support for multiple key fields such as ground object recognition, environmental monitoring, and resource exploration. In practical applications, a single pixel in a hyperspectral image may simultaneously contain spectral information of multiple ground objects, increasing the difficulty of image interpretation and ground object classification. Therefore, hyperspectral image unmixing technology has emerged. The core objective of this technology is to accurately separate the pure spectral signals of different ground objects and their spatial distributions from mixed pixels, thereby achieving more accurate ground object classification and recognition.

[0003] However, in the process of unmixing a hyperspectral image through a spectral unmixing autoencoder, it mainly relies on a single pixel or the neighborhood of a pixel as the input, resulting in a low accuracy of unmixing the hyperspectral image. Summary of the Invention

[0004] In view of this, embodiments of the present disclosure provide a hyperspectral image sparse unmixing method, apparatus, electronic device, and medium, which at least partially solve the problems existing in the prior art.

[0005] In a first aspect, embodiments of the present disclosure provide a hyperspectral image sparse unmixing method, including:

[0006] Performing feature extraction on an original hyperspectral image to obtain multi-scale spatial features and spectral features;

[0007] Stitching the multi-scale spatial features and the spectral features to obtain an original spatial-spectral joint feature;

[0008] Performing feature decoupling on the original spatial-spectral joint feature to obtain an enhanced spatial-spectral joint feature;

[0009] Normalizing the enhanced spatial-spectral joint feature according to a pre-set weight matrix to obtain a preliminary abundance matrix;

[0010] Through a pre-set loss function, in combination with a pre-set neural network, iteratively correcting the preliminary abundance matrix and the weight matrix to obtain a sparse abundance matrix and an endmember matrix, where the loss function includes: a reweighted collaborative sparsity regular term and a reweighted spectral total variation regular term;

[0011] Obtaining a reconstructed hyperspectral image according to the sparse abundance matrix and the endmember matrix;

[0012] Performing feature extraction on the original hyperspectral image to obtain multi-scale spatial features, including:

[0013] Performing max pooling, strip pooling, and average pooling on the original hyperspectral image to obtain a maximized result, a striped result, and an averaged result respectively;

[0014] Adding the maximized result, the striped result, and the averaged result to obtain a fused feature;

[0015] Performing convolution and activation function processing on the fused feature to obtain the multi-scale spatial features;

[0016] The loss function includes: spectral reconstruction error L RE and the reweighted collaborative sparse regularization term R co-spare and the reweighted spectral total variation regularization term R TV ;

[0017] The formula of the loss function is:

[0018] L total = L RE + λ1R co-spare + λ2R TV ,

[0019] where λ1 and λ2 are regularization term coefficients;

[0020] where x i represents the true spectral vector of the i-th pixel; represents the true spectral vector of the i-th pixel; n represents the total number of pixels; · represents vector dot product operation; ||·|| represents the L2 norm of the vector;

[0021] where W 1,x is the weight matrix, A x is the preliminary abundance matrix, represents element-wise multiplication, || || 2,1 i.e., L 2,1 norm;

[0022] where H(E) is the difference operation on the preliminary abundance matrix E, W2 is the weight matrix, represents element-wise multiplication, ||·|| 1,1 represents the l 1,1 norm.

[0023] Optionally, performing feature extraction on the original hyperspectral image to obtain spectral features, including:

[0024] Perform two-dimensional convolution, activation function processing, batch normalization, and spectral self-attention weight calculation on the original hyperspectral image in sequence to obtain the spectral features.

[0025] Optionally, the splicing of the multi-scale spatial features and the spectral features to obtain the original spatial-spectral joint features includes: splicing the spectral features and the multi-scale spatial features along the channel dimension to obtain the original spatial-spectral joint features.

[0026] Optionally, the feature decoupling of the original spatial-spectral joint features to obtain the enhanced spatial-spectral joint features includes: generating a channel weight matrix, performing pixel-wise multiplication on the channel weight matrix and the original spatial-spectral joint features to obtain the weighted spatial-spectral joint features;

[0027] Perform cross-attention perception enhancement processing on the weighted spatial-spectral joint features to obtain the enhanced spatial-spectral joint features.

[0028] Optionally, through a pre-set loss function, in combination with a pre-set neural network, iteratively correct the preliminary abundance matrix and the weight matrix to obtain a sparse abundance matrix and an endmember matrix, including: inputting the original hyperspectral image into the neural network, combining the pre-set network parameters corresponding to the neural network, and performing unsupervised training on the neural network through the loss function to obtain the trained neural network;

[0029] Input the enhanced spatial-spectral joint features into the fully connected layer and normalization layer of the trained neural network, and in combination with the reweighted collaborative sparse regularization term in the loss function, generate the sparse abundance matrix;

[0030] Extract the weight matrix corresponding to the decoder in the trained neural network to obtain the endmember matrix.

[0031] In a second aspect, an embodiment of the present disclosure provides a hyperspectral image sparse unmixing device, including:

[0032] An extraction module for extracting features from the original hyperspectral image to obtain multi-scale spatial features and spectral features;

[0033] A splicing module for splicing the multi-scale spatial features and the spectral features to obtain the original spatial-spectral joint features;

[0034] A decoupling module for performing feature decoupling on the original spatial-spectral joint features to obtain the enhanced spatial-spectral joint features;

[0035] A normalization module, which is used to normalize the enhanced spatial-spectral joint features according to a preset weight matrix to obtain a preliminary abundance matrix;

[0036] A training module, which is used to iteratively correct the preliminary abundance matrix and the weight matrix through a preset loss function in combination with a preset neural network to obtain a sparse abundance matrix and an endmember matrix. The loss function includes: a reweighted collaborative sparsity regularization term and a reweighted spectral total variation regularization term;

[0037] A reconstruction module, which is used to reconstruct the sparse abundance matrix and the endmember to obtain a reconstructed hyperspectral image;

[0038] The extraction module is specifically used to perform max pooling, stripe pooling, and average pooling on the original hyperspectral image to obtain a maximized result, a striped result, and an averaged result respectively; add the maximized result, the striped result, and the averaged result to obtain a fused feature; perform convolution and activation function processing on the fused feature to obtain the multi-scale spatial feature;

[0039] The loss function includes: a spectral reconstruction error L RE , the reweighted collaborative sparsity regularization term R co-spare and the reweighted spectral total variation regularization term R TV ;

[0040] The formula of the loss function is:

[0041] L total = L RE + λ1R co-spare + λ2R TV ,

[0042] where λ1 and λ2 are regularization term coefficients;

[0043] where x i represents the true spectral vector of the i-th pixel; represents the true spectral vector of the i-th pixel; n represents the total number of pixels; · represents the vector dot product operation; ||·|| represents the L2 norm of the vector;

[0044] where W 1,x is the weight matrix, A x is the preliminary abundance matrix, represents element-wise multiplication, || || 2,1 i.e., the L 2,1 norm;

[0045] Where H(E) is the difference operation of the preliminary abundance matrix E, W2 is the weight matrix, represents element-by-element multiplication, ||·|| 1,1 Indicates l 1，1 Norm.

[0046] In a third aspect, an embodiment of the present disclosure provides an electronic device, the electronic device comprising:

[0047] at least one processor; and a memory communicatively coupled to the at least one processor;

[0048] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the hyperspectral image sparse unmixing method as described in any one of the first aspects.

[0049] In a fourth aspect, an embodiment of the present disclosure provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the hyperspectral image sparse unmixing method as described in any one of the first aspects.

[0050] In summary, compared with the prior art, this embodiment has the following advantages:

[0051] 1. Enhanced feature extraction and fusion capabilities: The hyperspectral image sparse unmixing method proposed in this embodiment fully considers and effectively utilizes the three-dimensional properties of hyperspectral images, namely spatial and spectral properties, through the design of parallel multi-scale pooling spatial feature extraction and spectral self-attention processing. After the features output by the dual processing process are blended, the information receiving domain of the model is further broadened through the HAFB mechanism, and the feature weights of the two dimensions (space and spectrum) are adjusted, thereby enhancing the global expression ability of the features. This design enables the model to more effectively connect contextual information with information between the spectrum and space, enriches the level and details of the features, and provides a more accurate and comprehensive feature basis for subsequent endmember extraction and abundance estimation.

[0052] 2. Improve the accuracy and robustness of unmixing: In this embodiment, by calculating the spatial homogeneity of each endmember in the hyperspectral image, a loss function is constructed that includes a reweighted collaborative sparse regularization term and a reweighted spectral total variation regularization term. The reweighted collaborative sparse regularization term utilizes the spatial similarity of abundances within local regions of the hyperspectral image. Through superpixel segmentation and the reweighted factor matrix, it imposes row sparsity constraints on the abundance matrix, enabling adjacent pixels to share the same set of endmembers and strengthening local sparsity, thereby significantly improving the unmixing accuracy. At the same time, the reweighted spectral total variation regularization term imposes a difference constraint on the spectral dimension of the non-linear parameters. Through difference operations and the reweighted factor matrix, it ensures the smoothness of the non-linear parameters in both spectral and spatial dimensions, effectively suppressing noise and overfitting problems, and further enhancing the robustness of the model.

[0053] 3. Integrate spatial-spectral information and optimize endmember extraction and abundance estimation: The method of this embodiment not only integrates the spatial and spectral information of the hyperspectral image, but also significantly improves the accuracy of endmember extraction and abundance estimation through fine feature extraction and regularization processing. This optimization enables the method to perform excellently when dealing with complex hyperspectral image data, providing more accurate and reliable technical support for fields such as remote sensing image processing, environmental monitoring, and geological exploration.

[0054] In summary, through the multi-scale spatio-spectral joint autoencoder network design and regularization processing strategy in this embodiment, the dual improvement of the accuracy and robustness of hyperspectral image sparse unmixing is achieved, with significant technical advantages and application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] To make the objectives, technical solutions, and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below in conjunction with specific embodiments and the accompanying drawings. Herein, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but not to limit the present invention.

[0056] The term "including" and its variations used herein represent open inclusion, that is, "including but not limited to". Unless specifically stated, the term "or" means "and / or". The term "based on" means "at least partially based on". The terms "an example embodiment" and "an embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first", "second", etc. may refer to different or the same objects. There may also be other explicit and implicit definitions below.

[0057] Figure 1 It is a schematic flow diagram of a hyperspectral image sparse unmixing method provided by an embodiment of the present application;

[0058] Figure 2Schematic diagram for extracting multi-scale pooling spatial features provided by an embodiment of the present application;

[0059] Figure 3 Schematic diagram of the process of a spectral self-attention mechanism provided by an embodiment of the present application;

[0060] Figure 4 Schematic diagram of the process of a hybrid attention perception mechanism provided by an embodiment of the present application;

[0061] Figure 5 Schematic diagram of the result of abundance estimation obtained on a synthetic dataset provided by an embodiment of the present application;

[0062] Figure 6 Schematic diagram of the result of abundance estimation obtained on the Samson dataset provided by an embodiment of the present application;

[0063] Figure 7 Schematic diagram of the result of abundance estimation obtained on the Jasper Ridge dataset provided by an embodiment of the present application;

[0064] Figure 8 Schematic diagram of the structure of a hyperspectral image sparse unmixing device provided by an embodiment of the present application;

[0065] Figure 9 Exemplary structural diagram of a device capable of implementing the method according to an embodiment of the present invention provided by an embodiment of the present application. Detailed implementation manners

[0066] The following describes the embodiments of the present disclosure in detail with reference to the accompanying drawings.

[0067] The following illustrates the embodiments of the present disclosure through specific specific examples. Those skilled in the art can easily understand other advantages and effects of the present disclosure from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments. The present disclosure can also be implemented or applied through other different specific implementation manners. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that, without conflict, the following embodiments and the features in the embodiments can be combined with each other. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present disclosure without creative efforts belong to the scope of protection of the present disclosure.

[0068] Note that the following describes various aspects of embodiments within the scope of the appended claims. It should be apparent that the aspects described herein can be embodied in a wide variety of forms, and any specific structure and / or function described herein is illustrative only. Based on this disclosure, those skilled in the art should understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects described herein can be used to implement a device and / or practice a method. Additionally, this device can be implemented and this method can be practiced using other structures and / or functionality in addition to one or more of the aspects described herein.

[0069] It should also be noted that the diagrams provided in the following embodiments only illustrate the basic concept of the present disclosure schematically. Only the components related to the present disclosure are shown in the diagrams, rather than being drawn according to the number, shape, and size of the components in actual implementation. The type, quantity, and ratio of each component in actual implementation can be arbitrarily changed, and the component layout type may also be more complex.

[0070] In addition, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that the aspects described herein can be practiced without these specific details.

[0071] Please refer to Figure 1 , Figure 1 which is a schematic flowchart of a hyperspectral image sparse unmixing method provided for an embodiment of the present application and can be executed by an electronic device, such as by one or more processors within the electronic device, to implement the following steps:

[0072] S101, perform feature extraction on the original hyperspectral image to obtain multi-scale spatial features and spectral features.

[0073] Among them, the operation of performing feature extraction on the original hyperspectral image may include: three parallel pooling operations and a pooling spatial feature fusion processing operation, and the three parallel pooling operations may be max pooling, strip pooling, and average pooling respectively.

[0074] Optionally, in the process of performing feature extraction on the original hyperspectral image to obtain multi-scale spatial features, the electronic device may first perform max pooling, strip pooling, and average pooling on the original hyperspectral image to obtain a maximized result, a striped result, and an averaged result respectively, and add the maximized result, the striped result, and the averaged result to obtain a fused feature, and then perform convolution and activation function processing on the fused feature to obtain multi-scale spatial features.

[0075] See Figure 2 , Figure 2Schematic diagram for extracting multi-scale pooling spatial features provided by an embodiment of the present application, as shown in Figure 2 follows: The first pooling operation is max pooling Maxpool, which is used to capture the maximum value features in the image, and can highlight the significant details and edges in the image. The expression of max pooling y m is shown in formula (1):

[0076] y m = Maxpool(F input )(1)

[0077] where F input is the original hyperspectral image.

[0078] The second pooling operation is stripe pooling. By performing pooling operations on the input hyperspectral image in the horizontal and vertical directions, preliminary features are obtained. Subsequently, through corresponding convolutions in different directions, an expansion operation is carried out to upsample the features back to the original resolution, obtaining complete stripe features. These stripe features help to extract the integrity of the hyperspectral image in a certain direction and can fully reflect the consistency of spatial and spectral information. The expression of horizontal pooling is shown in formula (2), and the expression of vertical pooling is shown in formula (3):

[0079]

[0080] where i represents the i-th row, j represents the j-th column; W represents the width (column) of the image, H represents the height (row) of the image, h represents the height direction, and w represents the width direction. is the pixel radiation value of the i-th row and j-th column of the original hyperspectral image.

[0081] Formula (2) represents taking the average of all pixel radiation values in the i-th row; formula (3) represents taking the average of all pixel radiation values in the j-th column.

[0082] The third pooling operation is average pooling Averagepool, which takes the average of the pixel radiation values in each pooling window. Average pooling can smooth the features in the hyperspectral image, reduce the influence of noise, and at the same time retain the overall trend, and can effectively extract global spatial features. The expression of average pooling y a is shown in formula (4):

[0083] y a = Averagepool(F input )(4)

[0084] where F input is the original hyperspectral image.

[0085] Correspondingly, after obtaining the maximized result, the striped result, and the averaged result, a fusion processing operation can be performed on the maximized result, the striped result, and the averaged result, and then convolution and activation function processing are performed on the fused features, so as to obtain multi-scale spatial features, so that in subsequent steps, spatial information of different scales can be captured, the perception ability of complex ground object distributions can be enhanced, and more comprehensive spatial feature support can be provided for subsequent spatial-spectral joint feature fusion and unmixing.

[0086] Optionally, the above-mentioned maximized result, striped result, and averaged result can be fused through an addition operation to obtain a fused feature F pool , and then convolution and activation function processing are performed on the fused feature to obtain multi-scale spatial features. For example, 1×1 convolution and sigmoid activation function processing can be used to obtain multi-scale spatial feature F spa .

[0087] Similarly, in the process of extracting spectral features from the original hyperspectral image, two-dimensional convolution, activation function processing, batch normalization, and spectral self-attention weight calculation can be sequentially performed on the original hyperspectral image to obtain spectral features.

[0088] See Figure 3 , Figure 3 which is a schematic flowchart of a spectral self-attention mechanism (spatial attention module, SAM) provided by an embodiment of the present application. As Figure 3 shown, the spectral self-attention mechanism process includes: two cascaded two-dimensional convolution layer processes (preferably, 1×1 two-dimensional convolution) and two spectral self-attention mechanism processes.

[0089] Specifically, after the first 1×1 convolution process, batch normalization (BN) processing and SAM processing are cascaded immediately. Similarly, the second 1×1 convolution process is sequentially connected to the LeakyReLU activation function, BN processing, and SAM processing.

[0090] For example, in the design of the spectral self-attention mechanism, the stride of all convolution processes is set to 1, and zero padding is used as the padding strategy to ensure the constancy of the spatial dimension. For the LeakyReLU activation function, the negative slope parameter of the activation function can be set to 0.2.

[0091] In addition, the output dimension of the first convolution process is set to a fixed value (48), while the output dimension of the second convolution process can be flexibly adjusted according to the number of endmembers in the specific dataset, that is, according to the number of types of pure substances in the dataset. It should be noted that as the implementation of the self-attention mechanism, the SAM processing process does not change the dimension of the input data.

[0092] Therefore, the final output of the spectral feature extraction process is the spectral feature F spe , and the number of channels is exactly equal to the number of endmembers C in the dataset.

[0093] S102, Concatenate the multi-scale spatial features and the spectral features to obtain the original spatial-spectral joint features.

[0094] Specifically, the spectral self-attention mechanism processes by adjusting the attention weights on the spectral feature channels, thereby generating the spectral feature F spe . Subsequently, these spectral spatial features F spe are concatenated with the multi-scale spatial features F spa in the channel dimension by performing a concatenate operation, in this way, the information of the two is fused, and finally the original spatial-spectral joint feature F spa-spe is generated.

[0095] Optionally, in the process of concatenating the multi-scale spatial features and the spectral features to obtain the original spatial-spectral joint features, the spectral features and the multi-scale spatial features can be concatenated in the channel dimension to obtain the original spatial-spectral joint features.

[0096] For example, the concatenate function tf.concat in the tensorflow framework can be used to concatenate the two features in the channel dimension.

[0097] S103, Decouple the features of the original spatial-spectral joint features to obtain the enhanced spatial-spectral joint features.

[0098] Although the concatenation operation in step S102 above integrates different-scale features containing context information, the simple concatenation method fails to fully explore the complementarity between these features, resulting in a significant decrease in the accuracy of unmixing.

[0099] Therefore, a hybrid attention fusion based (HAFB) mechanism needs to be introduced. The HAFB mechanism can assign different weights to different channels, thereby effectively highlighting important features and suppressing secondary features, realizing the efficient and accurate fusion of multi-scale features, and then obtaining a more comprehensive and accurate feature representation.

[0100] Optionally, in the process of decoupling the features of the original empty-spectrum joint features to obtain enhanced empty-spectrum joint features, a channel weight matrix can be generated, and the channel weight matrix and the original empty-spectrum joint features are multiplied pixel by pixel to obtain weighted empty-spectrum joint features, and then the weighted empty-spectrum joint features are subjected to cross-attention perception enhancement processing to obtain enhanced empty-spectrum joint features.

[0101] Specifically, refer to Figure 4 , Figure 4 which is a schematic flowchart of a hybrid attention perception mechanism provided by an embodiment of the present application. The HAFB mechanism is a mechanism designed specifically for processing empty-spectrum joint features and can include: a channel attention fusion stage and an attention-based perception enhancement stage, a total of two core stages.

[0102] As Figure 4 shown, in the channel attention fusion stage (stage 1), a channel weight matrix can be generated using deep learning. First, the input original empty-spectrum joint feature F spa-spe is processed through a pooling layer to extract global information and reduce the amount of computation. Then, the output of the pooling layer is passed to a fully connected layer to further fuse and transform the information, generating an intermediate feature vector, which is the result output by the fully connected layer. Finally, the intermediate feature vector is non-linearly transformed through a sigmoid activation function to generate a channel weight matrix W c between 0 and 1, and this matrix is used to represent the importance of each channel for the original empty-spectrum joint features.

[0103] To apply these weights to the input original empty-spectrum joint features, a pixel-by-pixel multiplication operation can be performed, that is, the original empty-spectrum joint feature value of each channel is multiplied by the weight corresponding to the original empty-spectrum joint feature, so as to obtain weighted empty-spectrum joint features.

[0104] The goal of the attention-based perception enhancement stage (stage 2) is to further enhance the global dependence between the empty-spectrum joint features. As Figure 4 shown, the electronic device can introduce two cross-attention processes. These two cross-attention processes process different dimensions of the feature map (for example, the horizontal and vertical directions) respectively and capture the correlations between different positions.

[0105] In this way, HAFB can identify and enhance the dependencies between pixels that are most critical for the overall feature representation. Finally, the outputs of the two cross-attention processes are combined to form enhanced empty-spectrum joint features. By splicing the multi-scale spatial features and spectral features of the extracted original hyperspectral image and inputting them into the HAFB mechanism for feature decoupling and enhancement, more accurate and rich enhanced empty-spectrum joint features can be obtained.

[0106] Optionally, as Figure 4 shown, first, input the original empty-spectrum joint feature F spa-spe into the HAFB mechanism and go through the channel attention fusion stage. In this stage, through a pooling layer, a fully connected layer, a ReLU activation function, another fully connected layer, and a sigmoid activation function in sequence, calculate the channel weight matrix W c .

[0107] Subsequently, multiply the weight matrix W c element-wise with the input original empty-spectrum joint feature F spa-spe , and input the result F c into a 1×1 convolutional layer, and set the number of output channels of this convolutional layer to the number of endmembers. In each round of cross-attention processing in the attention-based perception enhancement stage, only the cross region in Figure 4 is involved.

[0108] Since the attention mechanism in the second round is executed on the features generated in the first round, equivalent global attention can be achieved at the end of stage 2, which will establish effective global dependencies between pixels in the original empty-spectrum joint feature, and can enhance the empty-spectrum feature based on global spatial attention to obtain the enhanced empty-spectrum joint feature.

[0109] Because the attention mechanism does not change the spatial structure and the number of channels of the data, the finally generated enhanced empty-spectrum joint feature has an unchanged spatial shape and the number of channels is the number of endmembers.

[0110] S104. Normalize the enhanced empty-spectrum joint feature according to the pre-set weight matrix to obtain the preliminary abundance matrix.

[0111] In the fields of hyperspectral image processing, remote sensing data analysis, etc., the empty-spectrum joint feature is an important information source. It combines spatial features and spectral features and can describe the characteristics of ground objects more comprehensively. In order to extract useful information from these features, normalization processing is usually required and an abundance matrix is generated.

[0112] The abundance matrix reflects the distribution of different ground object components in the image, while the endmember matrix represents the basic spectral features of these components. Through the abundance matrix and the endmember matrix, tasks such as ground object classification and anomaly detection can be achieved.

[0113] Optionally, the normalization layer performs a normalization operation on the enhanced spatial-spectral joint features. The role of the normalization layer is to scale and translate the enhanced spatial-spectral joint features so that the enhanced spatial-spectral joint features conform to a certain distribution (such as a normal distribution), thereby eliminating the dimensional difference and numerical range difference between different enhanced spatial-spectral joint features. The normalization layer may adopt various methods, such as the Z-score normalization method, to process the enhanced spatial-spectral joint features.

[0114] The enhanced spatial-spectral joint features processed by the normalization layer are used to generate a preliminary abundance matrix. The preliminary abundance matrix is a two-dimensional array, where each element represents the abundance (i.e., proportion) of the corresponding ground object component in the image. The methods for generating the preliminary abundance matrix may include non-negative matrix factorization (NMF) and sparse representation, etc. The goal of these methods for generating the preliminary abundance matrix is to decompose the normalized enhanced spatial-spectral joint features into a series of endmember components and corresponding abundance values.

[0115] The decoder is usually a component in a deep learning model, which is used to map the enhanced spatial-spectral joint features back to the original data space or generate a new data representation. The pre-set weight matrix usually contains important information about the endmember components. Each row of the pre-set weight matrix may represent the spectral features of an endmember component.

[0116] S105, through a pre-set loss function, combined with a pre-set neural network, iteratively correct the preliminary abundance matrix and the pre-set weight matrix to obtain a sparse abundance matrix and an endmember matrix.

[0117] Specifically, the loss function is a metric that measures the difference between the prediction results of the neural network and the actual data. The neural network can continuously adjust the preliminary abundance matrix and the pre-set weight matrix through its non-linear fitting ability to minimize the loss function.

[0118] In the process of iteratively correcting the preliminary abundance matrix and the pre-set weight matrix, use the preliminary abundance matrix and the pre-set weight matrix as the starting point, and calculate the loss function value for the preliminary abundance matrix and the pre-set weight matrix input into the neural network.

[0119] Correspondingly, according to the gradient information of the loss function, the weights and biases of the neural network can be adjusted through the backpropagation algorithm, and at the same time, the element values of the current abundance matrix and the current weight matrix can also be directly adjusted. Moreover, according to the gradient information obtained by backpropagation, a certain optimization algorithm (such as gradient descent, Adam, etc.) can be used to update the abundance matrix and the weight matrix.

[0120] Repeat the above process until the value of the loss function converges to a smaller value, or reaches the preset number of iterations, completing the iterative correction of the initial abundance matrix and the preset weight matrix to obtain the sparse abundance matrix and the endmember matrix.

[0121] It should be noted that the weight matrix after the iteration is completed is the endmember matrix. Among them, the loss function can include: the reweighted collaborative sparse regularization term and the reweighted spectral total variation regularization term.

[0122] Optionally, in the process of iteratively correcting the initial abundance matrix and the preset weight matrix to obtain the sparse abundance matrix and the endmember matrix, the original hyperspectral image can be input into the neural network, combined with the network parameters preset corresponding to the neural network, and the neural network is unsupervised trained through the loss function to obtain the trained neural network; the enhanced spatial-spectral joint features are input into the fully connected layer and the normalization layer of the trained neural network, and combined with the reweighted collaborative sparse regularization term in the loss function to generate the sparse abundance matrix; according to the weight matrix corresponding to the decoder in the trained neural network, the endmember matrix is generated.

[0123] For example, the loss function can include: the spectral reconstruction error L RE , the reweighted collaborative sparse regularization term R co-spare and the reweighted spectral total variation regularization term R TV .

[0124] Correspondingly, the formula of the loss function can be:

[0125] L total = L RE + λ1R co-spare + λ2R TV (5)

[0126] Among them, λ1 and λ2 are regularization term coefficients.

[0127] Moreover Among them, x i represents the true spectral vector of the i-th pixel; represents the true spectral vector of the i-th pixel; n represents the total number of pixels; · represents the vector dot product operation; ||·|| represents the L2 norm of the vector.

[0128] In addition, Among them, W 1,x is the weight matrix, A x is the initial abundance matrix, represents element-by-element multiplication;

[0129] Furthermore, Among them, H(E) is the difference operation on the initial abundance matrix E, and W2 is the weight matrix. Indicates element-by-element multiplication.

[0130] Furthermore, L RE , R co-spare are calculated as shown in Formula (6) and Formula (7) respectively:

[0131]

[0132] where n is the total number of endmembers, X is the number of hyperspectral image pixels, x i and represent the original hyperspectral pixel and the reconstructed hyperspectral pixel of the i-th pixel respectively; ° represents the Hadamard product (element-by-element multiplication); A is the abundance matrix; W 1,k is the reweighting factor matrix used to adjust sparsity, and the elements of W 1,k W 1,k,i,j are updated in each iteration, and the update formula is as shown in Formula (8):

[0133]

[0134] where δ is a small positive number used to prevent the denominator from being zero.

[0135] || || 2，1 That is, the L 2,1 norm, which can be expressed as Formula (9):

[0136]

[0137] where a ij represents the abundance value of the j-th endmember in the i-th pixel, and n represents the number of pixels.

[0138] This abundance sparse loss function aims to suppress redundant abundances by sparsifying the abundance matrix A. Since each row of A T corresponds to each column of A, this actually encourages some columns in A to become close to zero, thereby reducing the influence of these columns during the optimization process. This sparsity constraint helps to improve the robustness of endmember and abundance estimation, especially when dealing with noisy data.

[0139] R TV can be calculated as shown in Formula (10):

[0140]

[0141] where E is the abundance matrix; H represents the difference operation on E; W2 is the reweighting factor matrix used to penalize non-zero terms; ||·|| 1，1 represents l 1，1The norm makes the matrix after differentiation sparse; the element \(W\) of the reweighted factor matrix \(W_2\) 2,i,j is updated in each iteration, and the update formula is formula (11):

[0142]

[0143] where \(\delta\) is also a small positive number used to prevent the denominator from being zero.

[0144] S106. Obtain the reconstructed hyperspectral image according to the sparse abundance matrix and the endmember matrix.

[0145] In deep learning, a neural network can extract features of input data through convolution operations. The 1x1 convolution is a special convolution operation with a convolution kernel size of 1x1. This convolution operation does not change the spatial dimensions (height and width) of the input data, but can change the number of channels of the data (i.e., the number of features).

[0146] In this embodiment, the number of convolution kernels is set to the number of spectral channels of the dataset, and the number of convolution kernels of the 1x1 convolution layer is set to be the same as the number of spectral channels in the dataset. This means that each convolution kernel corresponds to a specific spectral channel or feature, and through this convolution kernel, the neural network can learn how to extract the features corresponding to each spectrum from the input data.

[0147] It should be noted that the number of spectral channels of the dataset is defined as the number of spectral bands of each pixel in the hyperspectral image. For example, the synthetic dataset has 200 bands, and the Samson dataset has 156 bands.

[0148] After the neural network training is completed, the weight matrix of this convolution layer is used as the result of extracting the endmembers. During the training process, the neural network continuously adjusts the weights of the convolution kernels through the backpropagation algorithm to minimize the loss function, thereby optimizing the performance of the model.

[0149] Moreover, the weight matrix of the 1x1 convolution layer (i.e., the weights of each convolution kernel) will contain information about the spectral features extracted from the input data. These weight matrices are regarded as the extracted endmember results. Finally, the decoder reconstructs the sparse abundance matrix and the endmember matrix to obtain the reconstructed hyperspectral image. The unmixing task of the hyperspectral image is completed.

[0150] It should be noted that according to the sparse abundance matrix, the distribution ratio of the endmembers of the reconstructed hyperspectral image in space is obtained, while according to the endmember matrix, the spectral information of the reconstructed hyperspectral image is obtained.

[0151] It should also be noted that the reconstructed hyperspectral image obtained according to the sparse abundance matrix and the endmember matrix is obtained by multiplying the two matrices of the sparse abundance matrix and the endmember matrix.

[0152] In this implementation, the hyperspectral image sparse unmixing method provided in this embodiment was also verified by simulation. Specifically as follows:

[0153] The embodiments of the present invention were carried out in the python 3.10.0 environment, using the TensorFlow 2.10.0 deep learning framework. The network was trained on the NVIDIA GeForce RTX 4090 GPU and Core TM the Core i9-13900KF processor. The detailed hyperparameter settings of the model include: the stride of the convolutional layer is set to 1, and the same padding strategy is adopted to keep the spatial shape unchanged. The negative slope of the LeakyReLU activation function is set to 0.2, and the dropout rate of the Dropout layer is also set to 0.2 to prevent gradient disappearance and overfitting. The final model is trained by the Adam optimizer, and the initial learning rate is set to 0.005. During the training process, the batch size is set to 32, and the number of training epochs is set to 50 to achieve the best performance.

[0154] Experimental data and comparative experiments:

[0155] The synthetic data used in the embodiments of the present invention is an image of 60×60 pixels. Five different endmembers are selected from the ASTER spectral library for this image, covering 200 bands from 0.4 microns to 14 microns. The abundance values of the materials are generated according to the Dirichlet distribution to simulate complex real-world mixing situations. To evaluate the robustness of the present invention, Gaussian white noise with a signal-to-noise ratio of 20 db is also introduced into this dataset.

[0156] The real datasets used in the embodiments of the present invention are the Samson and Jasper Ridge datasets; the Samson dataset has an image size of 95×95, a wavelength range from 380 to 2500 nanometers, and 156 channels. This method includes three typical surface coverings of soil, trees, and water as endmembers; the Jasper Ridge dataset has an image size of 100×100 pixels, a wavelength range from 400 to 2500 nanometers, and 192 bands, including four endmembers of road, soil, water, and trees.

[0157] The hyperspectral sparse unmixing method (Our) provided in this application was experimentally compared with 5 advanced deep learning-based hyperspectral unmixing methods to demonstrate the experimental effects of endmember extraction and abundance estimation of the present invention. All comparative methods set relevant hyperparameters according to the steps described in the original papers.

[0158] It should be noted that the five advanced and equally deep learning-based hyperspectral unmixing methods are: 2DCNN (2D convolutional neural unmixing network), DAEU (deep autoencoder network), Deep Trans (deep Transformer network), TANET (two-stream autoencoder network), and CyCU-Net (cycle-consistent unmixing network).

[0159] To fully evaluate the robustness and effectiveness of the model in the unmixing task, the embodiments of the present invention adopt two evaluation metrics: spectral angle distance (SAD) and root mean square error (RMSE); the lower the SAD value, the closer the extracted endmember is to the true endmember; if the RMSE value is lower, it means that the estimated abundance is closer to the true abundance, and their calculation methods are shown in formulas (12) and (13) respectively:

[0160]

[0161] where, e j represents the true endmember vector of the j-th endmember, represents the extracted endmember vector of the j-th endmember. a i is the true abundance of the j-th pixel, represents the corresponding estimated abundance value.

[0162] Table 1 shows the experimental results of the 20db synthetic dataset.

[0163] Table 1

[0164]

[0165] As can be seen from Table 1, except for the asphalt endmember, the other endmembers of the embodiments of the present application have achieved the best results, and the average SAD obtained is also the smallest. At the same time, in terms of abundance, the present invention is also the best among the vast majority of endmembers, achieving the best average RMSE; Figure 5 The abundance maps and endmember comparison curves of all methods on the synthetic data are shown respectively. It can be observed that the abundance maps of the method provided by the embodiments of the present application are visually closest to the ground truth (GT) in asphalt, concrete, limestone, conifer, and basalt respectively.

[0166] Table 2 shows the experimental results of the Samson dataset.

[0167] Table 2

[0168]

[0169] Table 2 shows the experimental results of different methods on the Samson dataset. As can be seen from the table, the embodiment of the present application achieved the best SAD results in all end members. In terms of abundance, except for the soil end member, other end members also achieved the best results and the smallest average RMSE value. Figure 6 The visualization results of abundance estimation and endmember extraction on the Samson dataset are shown respectively. It can be seen that the method provided in the embodiment of the present application is very close to the true value (GT) in soil (Soil), tree (Tree), and water (Water).

[0170] Table 3 shows the experimental results of the Jasper Ridge dataset.

[0171] Table 3

[0172]

[0173] Table 3 shows the experimental results of various methods on the Jasper Ridge dataset. From the data in the table, it can be seen that the embodiment of the present application not only achieved the best average SAD value and average RMSE value, but also achieved the best or suboptimal results in the endmember extraction and abundance estimation results of a single endmember. Figure 7 The visualization results of abundance estimation and end member extraction are shown respectively. Figure 7 It can be seen that the abundance map estimated by the method provided in the embodiment of the present application has the highest restoration degree in road (Road), soil (Soil), tree (Tree), and water (Water), and is extremely close to the true value (GT).

[0174] In summary, compared with the prior art, this embodiment has the following advantages:

[0175] 1. Enhanced feature extraction and fusion capabilities: The sparse unmixing method for hyperspectral images proposed in this embodiment fully considers and effectively utilizes the three-dimensional properties of hyperspectral images, namely spatial and spectral properties, through the design of parallel multi-scale pooling spatial feature processing and spectral self-attention processing. After the features output by the dual processing process are blended, the information receiving domain of the model is further broadened through the HAFB mechanism, and the feature weights of the two dimensions (space and spectrum) are adjusted, thereby enhancing the global expression ability of the features. This design enables the model to more effectively connect contextual information with information between the spectrum and space, enriches the level and details of the features, and provides a more accurate and comprehensive feature basis for subsequent end member extraction and abundance estimation.

[0176] 2. Improve the accuracy and robustness of unmixing: In this embodiment, by calculating the spatial homogeneity of each endmember in the hyperspectral image, a loss function is constructed that includes a reweighted collaborative sparse regularization term and a reweighted spectral total variation regularization term. The reweighted collaborative sparse regularization term utilizes the spatial similarity of abundances within local regions of the hyperspectral image. Through superpixel segmentation and the reweighted factor matrix, a row sparsity constraint is imposed on the abundance matrix, enabling adjacent pixels to share the same set of endmembers and strengthening local sparsity, thus significantly improving the unmixing accuracy. At the same time, the reweighted spectral total variation regularization term imposes a difference constraint on the spectral dimension of the non-linear parameters. Through difference operations and the reweighted factor matrix, the smoothness of the non-linear parameters in both the spectral and spatial domains is ensured, effectively suppressing noise and overfitting problems and further enhancing the robustness of the model.

[0177] 3. Integrate spatial-spectral information and optimize endmember extraction and abundance estimation: The method of this embodiment not only integrates the spatial and spectral information of the hyperspectral image but also significantly improves the accuracy of endmember extraction and abundance estimation through fine feature extraction and regularization processing. This optimization enables the method to perform excellently when dealing with complex hyperspectral image data, providing more accurate and reliable technical support for fields such as remote sensing image processing, environmental monitoring, and geological exploration.

[0178] In summary, through the multi-scale spatial-spectral joint autoencoder network design and regularization processing strategy in this embodiment, the dual improvements in the accuracy and robustness of hyperspectral image sparse unmixing are achieved, with significant technical advantages and application prospects.

[0179] The second embodiment of the present invention provides a hyperspectral image sparse unmixing device. Refer to Figure 8 , Figure 8 which is a schematic structural diagram of a hyperspectral image sparse unmixing device provided by an embodiment of the present application. It includes:

[0180] An extraction module 810 for extracting features from the original hyperspectral image to obtain multi-scale spatial features and spectral features;

[0181] A splicing module 820 for splicing the multi-scale spatial features and the spectral features to obtain the original spatial-spectral joint features;

[0182] A decoupling module 830 for decoupling the features of the original spatial-spectral joint features to obtain enhanced spatial-spectral joint features;

[0183] A normalization module 840 for normalizing the enhanced spatial-spectral joint features according to a pre-set weight matrix to obtain a preliminary abundance matrix;

[0184] The training module 850 is used to iteratively correct the preliminary abundance matrix and the preset weight matrix by using a preset loss function in combination with a preset neural network to obtain a sparse abundance matrix and an end member matrix, wherein the loss function includes: a reweighted collaborative sparse regularization term and a reweighted spectral total variation regularization term;

[0185] The reconstruction module 860 is used to reconstruct the sparse abundance matrix to obtain a reconstructed hyperspectral image.

[0186] Optionally, the extraction module performs maximum pooling, stripe pooling and average pooling on the original hyperspectral image to obtain a maximization result, a stripe result and an average result, respectively; the maximization result, the stripe result and the average result are added to obtain a fusion feature; and the fusion feature is processed by convolution and activation function to obtain a multi-scale spatial feature.

[0187] Optionally, the splicing module splices the spectral features and the multi-scale spatial features according to the channel dimension to obtain the original spatial-spectral joint features.

[0188] Optionally, the decoupling module performs two-dimensional convolution, activation function processing, batch normalization, and spectral self-attention weight calculation on the original hyperspectral image in sequence to obtain the spectral features.

[0189] Optionally, the decoupling module generates a channel weight matrix, multiplies the channel weight matrix and the original spatial-spectral joint features pixel by pixel, and obtains weighted spatial-spectral joint features; and performs cross-attention perception enhancement processing on the weighted spatial-spectral joint features to obtain enhanced spatial-spectral joint features.

[0190] Optionally, a training module inputs the original hyperspectral data into the neural network, combines the pre-set network parameters corresponding to the neural network, performs unsupervised training on the neural network through a loss function, and obtains a trained neural network; inputs the enhanced spatial-spectral joint features into the fully connected layer and normalization layer of the trained neural network, and combines the reweighted collaborative sparse regularization term in the loss function to generate a sparse abundance matrix; generates an end member matrix according to the weight matrix corresponding to the decoder in the trained neural network.

[0191] In summary, the sparse unmixing device for hyperspectral images provided in the embodiment of the present application fully considers and effectively utilizes the three-dimensional properties of hyperspectral images, namely, spatial and spectral properties. After blending the output features, the feature weights of the two dimensions (spatial and spectral) are adjusted, thereby enhancing the global expression ability of the features. This design enables the sparse unmixing device for hyperspectral images to more effectively connect contextual information with spectral and spatial information, enrich the level and details of the features, and provide a more accurate and comprehensive feature basis for subsequent end member extraction and abundance estimation.

[0192] The hyperspectral image sparse unmixing device provided in this embodiment constructs a loss function containing a reweighted collaborative sparse regularization term and a reweighted spectral total variation regularization term by calculating the spatial homogeneity of each endmember of the hyperspectral image. The reweighted collaborative sparse regularization term utilizes the spatial similarity of abundances within the local region of the hyperspectral image, imposes row sparsity constraints on the abundance matrix through superpixel segmentation and the reweighted factor matrix, enables adjacent pixels to share the same set of endmembers, and strengthens the local sparsity, thereby significantly improving the unmixing accuracy. At the same time, the reweighted spectral total variation regularization term imposes a difference constraint on the spectral dimension of the nonlinear parameters, and ensures the smoothness of the nonlinear parameters in both spectral and spatial domains through difference operations and the reweighted factor matrix, effectively suppressing noise and overfitting problems, and further enhancing the robustness of the model.

[0193] The hyperspectral image sparse unmixing device provided in this embodiment not only integrates the spatial and spectral information of the hyperspectral image, but also significantly improves the accuracy of endmember extraction and abundance estimation through fine feature extraction and regularization processing. This optimization enables the method to perform excellently when dealing with complex hyperspectral image data, providing more accurate and reliable technical support for fields such as remote sensing image processing, environmental monitoring, and geological exploration.

[0194] The third embodiment of the present invention also provides an electronic device, which includes:

[0195] At least one processor; and, a memory communicatively connected to the at least one processor; wherein,

[0196] The memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to execute the hyperspectral image sparse unmixing method of any of the foregoing embodiments.

[0197] The fourth embodiment of the present invention also provides a non-transitory computer-readable storage medium, which stores computer instructions for causing the computer to execute the hyperspectral image sparse unmixing method of any of the foregoing embodiments.

[0198] The fifth embodiment of the present invention also provides a computer program product, which includes a computing program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, and when the program instructions are executed by a computer, the computer is enabled to execute the hyperspectral image sparse unmixing method of any of the foregoing embodiments.

[0199] The sixth embodiment of the present invention also provides a computer program, which includes program instructions, and when the program instructions are executed by a computer, the computer is enabled to execute the hyperspectral image sparse unmixing method of any of the foregoing embodiments.

[0200] Figure 9 This is a schematic diagram of a device 1000 that can implement the method of the embodiments of the present application or realize the embodiments of the present invention. In some embodiments, it may include more or fewer devices than shown in the figure. In some embodiments, it can be implemented using a single or multiple devices. In some embodiments, it can be implemented using cloud or distributed devices.

[0201] As Figure 9 shown, the device 1000 includes a processor 1001, which can perform various appropriate operations and processes according to the programs and / or data stored in the read-only memory (ROM) 1002 and / or the programs and / or data loaded from the storage section 1008 into the random access memory (RAM) 1003. The processor 1001 can be a multi-core processor or can include multiple processors. In some embodiments, the processor 1001 can include a general main processor and one or more special coprocessors, such as a central processing unit (CPU), a graphics processing unit (GPU), a neural network processing unit (NPU), a digital signal processor (DSP), and so on. In the RAM 1003, various programs and data required for the operation of the device 1000 are also stored. The processor 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. The input / output (I / O) interface 1005 is also connected to the bus 1004.

[0202] The above-mentioned processor and memory are jointly used to execute the programs stored in the memory, and when the programs are executed by a computer, they can implement the methods, steps, or functions described in the above embodiments.

[0203] The following components are connected to the I / O interface 1005: an input section 1006 including a keyboard, a mouse, a touch screen, etc.; an output section 1007 including, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card, a modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed, so that the computer program read from it can be installed into the storage section 1008 as needed. Figure 9 Only some components are schematically shown, and it does not mean that the device 1000 only includes Figure 9 the components shown.

[0204] The systems, devices, modules, or units illustrated in the above embodiments can be implemented by a computer or its associated components. The computer can be, for example, a mobile terminal, a smart phone, a personal computer, a laptop computer, an in-vehicle human-machine interaction device, a personal digital assistant, a media player, a navigation device, a game console, a tablet computer, a wearable device, a smart TV, an Internet of Things system, a smart home, an industrial computer, a server, or a combination thereof.

[0205] Although not shown, in an embodiment of the present invention, a computer-readable storage medium is provided, on which a computer program / instructions are stored. When the computer program / instructions are executed by a processor, the hyperspectral image sparse unmixing method described in Embodiment 1 is implemented.

[0206] The storage medium in the embodiments of the present invention includes permanent and non-permanent, removable and non-removable articles that can implement information storage by any method or technology. Examples of the storage medium include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD), or other optical storage, magnetic cassette tapes, magnetic disk storage, or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device.

[0207] Although not shown, an embodiment of the present invention also provides a computer program product, including: a computer program / instructions, which when executed by a processor, implement the hyperspectral image sparse unmixing method described in Embodiment 1.

[0208] The methods, programs, systems, devices, etc. in the embodiments of the present invention can be executed or implemented in a single or multiple networked computers, and can also be practiced in a distributed computing environment. In the embodiments of this specification, in these distributed computing environments, tasks can be executed by remote processing devices connected through a communication network.

[0209] Those skilled in the art should understand that the embodiments of this specification can be provided as a method, a system, or a computer program product. Therefore, those skilled in the art can think that the implementation of the functional modules / units or controllers and related method steps illustrated in the above embodiments can be achieved in a software, hardware, or a combination of software and hardware manner.

[0210] Unless otherwise specified, the operations or steps of the methods and procedures described according to the embodiments of the present invention do not have to be performed in a particular order and still achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0211] In this document, multiple embodiments of the present invention have been described. For the sake of brevity, the descriptions of each embodiment are not exhaustive, and the same or similar features or parts among the various embodiments may be omitted. In this document, "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean applicable to at least one embodiment or example according to the present invention, rather than all embodiments. The above terms do not necessarily refer to the same embodiment or example. Without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0212] The exemplary systems and methods of the present invention have been specifically shown and described with reference to the above embodiments, which are only examples of the best mode for implementing the systems and methods. Those skilled in the art can understand that various changes can be made to the embodiments of the systems and methods described herein when implementing the systems and / or methods without departing from the spirit and scope of the present invention defined in the appended claims.

Claims

1. A hyperspectral image sparse unmixing method, characterized in that Including: Performing feature extraction on the original hyperspectral image to obtain multi-scale spatial features and spectral features; Stitching the multi-scale spatial features and the spectral features to obtain the original spatial-spectral joint features; Performing feature decoupling on the original spatial-spectral joint features to obtain enhanced spatial-spectral joint features; Normalizing the enhanced spatial-spectral joint features according to a pre-set weight matrix to obtain a preliminary abundance matrix; Through a pre-set loss function, in combination with a pre-set neural network, iteratively correcting the preliminary abundance matrix and the weight matrix to obtain a sparse abundance matrix and an endmember matrix, the loss function including: a reweighted collaborative sparsity regularization term and a reweighted spectral total variation regularization term; Obtaining the reconstructed hyperspectral image according to the sparse abundance matrix and the endmember matrix; The performing feature extraction on the original hyperspectral image to obtain multi-scale spatial features includes: Performing max pooling, strip pooling, and average pooling on the original hyperspectral image to respectively obtain a maximized result, a striped result, and an averaged result; Adding the maximized result, the striped result, and the averaged result to obtain a fused feature; Performing convolution and activation function processing on the fused feature to obtain the multi-scale spatial features; The loss function includes: spectral reconstruction error L RE , the reweighted collaborative sparse regularization term R co-spare and the reweighted spectral total variation regularization term R TV ; The formula of the loss function is: L total = L RE + λ1R co-spare + λ2R TV , where λ1 and λ2 are regularization term coefficients; where x i represents the true spectral vector of the i-th pixel; represents the true spectral vector of the i-th pixel; n represents the total number of pixels; · represents the vector dot product operation; ||·|| represents the L2 norm of the vector; where W 1,x is the weight matrix, A x is the preliminary abundance matrix, ° represents element-by-element multiplication, and ||·|| 2,1 i.e., L 2,1 norm; where H(E) is the differential operation on the preliminary abundance matrix E, W2 is the weight matrix, ° represents element-by-element multiplication, and ||·|| 1,1 denotes l 1,1 norm.

2. The hyperspectral image sparse unmixing method according to claim 1, characterized in that The performing feature extraction on the original hyperspectral image to obtain spectral features includes: Successively performing two-dimensional convolution, activation function processing, batch normalization, and spectral self-attention weight calculation on the original hyperspectral image to obtain the spectral features.

3. The hyperspectral image sparse unmixing method according to claim 1, characterized in that, The stitching the multi-scale spatial features and the spectral features to obtain the original spatial-spectral joint features includes: Stitching the spectral features and the multi-scale spatial features along the channel dimension to obtain the original spatial-spectral joint features.

4. The hyperspectral image sparse unmixing method according to claim 1, wherein The performing feature decoupling on the original spatial-spectral joint features to obtain enhanced spatial-spectral joint features includes: Generating a channel weight matrix, multiplying the channel weight matrix and the original spatial-spectral joint features pixel by pixel to obtain a weighted spatial-spectral joint feature; Performing cross-cross attention perception enhancement processing on the weighted spatial-spectral joint feature to obtain the enhanced spatial-spectral joint features.

5. The hyperspectral image sparse unmixing method according to any one of claims 1 to 4, characterized in that, The iteratively correcting the preliminary abundance matrix and the weight matrix through a pre-set loss function, in combination with a pre-set neural network, to obtain a sparse abundance matrix and an endmember matrix includes: Inputting the original hyperspectral image into the neural network, in combination with pre-set network parameters corresponding to the neural network, and performing unsupervised training on the neural network through the loss function to obtain a trained neural network; Inputting the enhanced spatial-spectral joint features into the fully connected layer and the normalization layer of the trained neural network, and in combination with the reweighted collaborative sparsity regularization term in the loss function, generating the sparse abundance matrix; Extracting the weight matrix corresponding to the decoder in the trained neural network to obtain the endmember matrix.

6. A hyperspectral image sparse unmixing device, characterized in that, Including: An extraction module for performing feature extraction on the original hyperspectral image to obtain multi-scale spatial features and spectral features; A splicing module, configured to splice the multi-scale spatial features and the spectral features to obtain original spatial-spectral joint features; A decoupling module, configured to perform feature decoupling on the original spatial-spectral joint features to obtain enhanced spatial-spectral joint features; A normalization module, configured to normalize the enhanced spatial-spectral joint features according to a preset weight matrix to obtain a preliminary abundance matrix; A training module, configured to iteratively correct the preliminary abundance matrix and the weight matrix through a preset loss function in combination with a preset neural network to obtain a sparse abundance matrix and an endmember matrix, where the loss function includes: a reweighted collaborative sparsity regularization term and a reweighted spectral total variation regularization term; A reconstruction module, configured to reconstruct the sparse abundance matrix and the endmember matrix to obtain a reconstructed hyperspectral image; The extraction module is specifically configured to perform max pooling, strip pooling, and average pooling on the original hyperspectral image to respectively obtain a maximized result, a striped result, and an averaged result; add the maximized result, the striped result, and the averaged result to obtain a fused feature; perform convolution and activation function processing on the fused feature to obtain the multi-scale spatial features; The loss function includes: spectral reconstruction error L RE , the reweighted collaborative sparse regularization term R co-spare and the reweighted spectral total variation regularization term R TV ; The formula of the loss function is: L total = L RE + λ1R co-spare + λ2R TV , where λ1 and λ2 are regularization term coefficients; where x i represents the true spectral vector of the i-th pixel; represents the true spectral vector of the i-th pixel; n represents the total number of pixels; · represents the vector dot product operation; ||·|| represents the L2 norm of the vector; where W 1,x is the weight matrix, A x is the preliminary abundance matrix, ° represents element-by-element multiplication, and || || 2,1 i.e., L 2,1 norm; where H(E) is the difference operation on the preliminary abundance matrix E, W2 is the weight matrix, ° represents element-by-element multiplication, and ||·|| 1，1 denotes the 1，1 l-norm.

7. An electronic device, characterized in that, The electronic device includes: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the hyperspectral image sparse unmixing method according to any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer instructions for causing a computer to execute the hyperspectral image sparse unmixing method according to any one of claims 1 to 5.

Images