Hyperspectral image sparse unmixing method and device, electronic equipment and medium

Abstract

The invention provides a hyperspectral image sparse unmixing method and device, electronic equipment and a medium, and the method comprises the steps: carrying out the feature extraction of an original hyperspectral image, and obtaining a multi-scale spatial feature and a spectral feature; splicing the multi-scale spatial features and the spectral features to obtain original spatial-spectral joint features; performing feature decoupling on the original spatial-spectral joint features to obtain enhanced spatial-spectral joint features; according to a preset weight matrix, normalizing the enhanced spatial-spectral joint features to obtain a preliminary abundance matrix; iteratively correcting the initial abundance matrix and a preset weight matrix through a loss function in combination with a neural network to obtain a sparse abundance matrix and an end member matrix; and reconstructing the sparse abundance matrix and the end member matrix to obtain a reconstructed hyperspectral image. According to the invention, through the multi-scale spatial spectrum combined self-encoder network design and the regularization processing strategy, double improvement of the precision and robustness of sparse unmixing of the hyperspectral image is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method, device, electronic equipment and medium for sparse unmixing of hyperspectral images. Background Art

[0002] Hyperspectral imaging technology, as an advanced imaging method, uses a spectrometer to image the target object in a wide and continuous spectral range, and can capture and record rich spectral information. This technology provides reliable data support for many key fields such as object identification, environmental monitoring, and resource exploration. In practical applications, a pixel of a hyperspectral image may contain spectral information of multiple objects at the same time, which increases the difficulty of image interpretation and object classification. Therefore, hyperspectral image unmixing technology came into being. The core goal of this technology is to accurately separate the pure spectral signals and spatial distribution of different objects from mixed pixels, so as to achieve more accurate object classification and identification.

[0003] However, in the process of unmixing the hyperspectral image by the spectral unmixing autoencoder, it mainly relies on a single pixel or pixel neighborhood as input, resulting in low accuracy of unmixing the hyperspectral image. Summary of the invention

[0004] In view of this, the embodiments of the present disclosure provide a method, device, electronic device and medium for sparse unmixing of hyperspectral images, which at least partially solve the problems existing in the prior art.

[0005] In a first aspect, an embodiment of the present disclosure provides a hyperspectral image sparse unmixing method, comprising: Extract features from the original hyperspectral image to obtain multi-scale spatial features and spectral features; splicing the multi-scale spatial features and the spectral features to obtain original spatial-spectral joint features; Performing feature decoupling on the original spatial-spectral joint feature to obtain an enhanced spatial-spectral joint feature; Normalizing the enhanced spatial spectrum joint features according to a preset weight matrix to obtain a preliminary abundance matrix; The preliminary abundance matrix and the weight matrix are iteratively corrected by a preset loss function in combination with a preset neural network to obtain a sparse abundance matrix and an end member matrix, wherein the loss function includes: a reweighted collaborative sparse regularization term and a reweighted spectral total variation regularization term; A reconstructed hyperspectral image is obtained according to the sparse abundance matrix and the endmember matrix.

[0006] Optionally, the extracting features from the original hyperspectral image to obtain multi-scale spatial features includes: performing maximum pooling, stripe pooling and average pooling on the original hyperspectral image to obtain a maximization result, a stripe result and an average result, respectively; Adding the maximization result, the striping result and the averaging result to obtain a fusion feature; The fused features are subjected to convolution and activation function processing to obtain the multi-scale spatial features.

[0007] Optionally, the extracting features from the original hyperspectral image to obtain spectral features includes: The original hyperspectral image is sequentially subjected to two-dimensional convolution, activation function processing, batch normalization and spectral self-attention weight calculation to obtain the spectral feature.

[0008] Optionally, the splicing of the multi-scale spatial features and the spectral features to obtain an original spatial-spectral joint feature includes: splicing the spectral features and the multi-scale spatial features according to a channel dimension to obtain the original spatial-spectral joint feature.

[0009] Optionally, the performing feature decoupling on the original spatial-spectral joint feature to obtain the enhanced spatial-spectral joint feature includes: generating a channel weight matrix, and performing pixel-by-pixel multiplication of the channel weight matrix and the original spatial-spectral joint feature to obtain a weighted spatial-spectral joint feature; The weighted spatial-spectral joint features are subjected to cross-attention perception enhancement processing to obtain the enhanced spatial-spectral joint features.

[0010] Optionally, the iterative correction of the preliminary abundance matrix and the weight matrix by a preset loss function in combination with a preset neural network to obtain a sparse abundance matrix and an endmember matrix includes: inputting the original hyperspectral image into the neural network, combining with preset network parameters corresponding to the neural network, performing unsupervised training on the neural network by the loss function to obtain a trained neural network; Inputting the enhanced spatial-spectral joint features into the fully connected layer and the normalization layer of the trained neural network, and combining the reweighted collaborative sparse regularization term in the loss function to generate the sparse abundance matrix; The weight matrix corresponding to the decoder in the trained neural network is extracted to obtain the end member matrix.

[0011] Optionally, the loss function includes: spectral reconstruction error , the reweighted collaborative sparse regularization term and the reweighted spectral total variation regularization term ; The formula of the loss function is: , in, and is the regularization term coefficient; ,in, Indicates The true spectral vector of pixels; Indicates The true spectral vector of pixels; Indicates the total number of pixels; Represents vector dot product operation; Represents a vector norm; ,in is the weight matrix, is the preliminary abundance matrix, Indicates element-by-element multiplication; ,in For the preliminary abundance matrix The difference operation, is the weight matrix, Represents element-by-element multiplication.

[0012] In a second aspect, the present disclosure provides a hyperspectral image sparse unmixing device, comprising: An extraction module is used to extract features from the original hyperspectral image to obtain multi-scale spatial features and spectral features; A splicing module, used for splicing the multi-scale spatial features and the spectral features to obtain original spatial-spectral joint features; A decoupling module, used for performing feature decoupling on the original spatial-spectral joint feature to obtain an enhanced spatial-spectral joint feature; A normalization module, used for normalizing the enhanced spatial spectrum joint features according to a preset weight matrix to obtain a preliminary abundance matrix; A training module, for iteratively correcting the preliminary abundance matrix and the weight matrix through a preset loss function in combination with a preset neural network to obtain a sparse abundance matrix and an end member matrix, wherein the loss function includes: a reweighted collaborative sparse regularization term and a reweighted spectral total variation regularization term; A reconstruction module is used to reconstruct the sparse abundance matrix and the end members to obtain a reconstructed hyperspectral image.

[0013] In a third aspect, an embodiment of the present disclosure provides an electronic device, the electronic device comprising: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the hyperspectral image sparse unmixing method as described in any one of the first aspects.

[0014] In a fourth aspect, an embodiment of the present disclosure provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the hyperspectral image sparse unmixing method as described in any one of the first aspects.

[0015] In summary, compared with the prior art, this embodiment has the following advantages: 1. Enhanced feature extraction and fusion capabilities: The sparse unmixing method for hyperspectral images proposed in this embodiment fully considers and effectively utilizes the three-dimensional properties of hyperspectral images, namely spatial and spectral properties, through the design of parallel multi-scale pooling spatial feature extraction and spectral self-attention processing. After the features output by the dual processing process are blended, the information receiving domain of the model is further broadened through the HAFB mechanism, and the feature weights of the two dimensions (space and spectrum) are adjusted, thereby enhancing the global expression ability of the features. This design enables the model to more effectively connect contextual information with information between the spectrum and space, enriches the level and details of the features, and provides a more accurate and comprehensive feature basis for subsequent endmember extraction and abundance estimation.

[0016] 2. Improve the accuracy and robustness of unmixing: This embodiment constructs a loss function including a reweighted collaborative sparse regularization term and a reweighted spectral total variation regularization term by calculating the spatial homogeneity of each end member of the hyperspectral image. The reweighted collaborative sparse regularization term utilizes the spatial similarity of abundance in local areas of the hyperspectral image, and imposes row sparsity constraints on the abundance matrix through superpixel segmentation and reweighting factor matrix, so that adjacent pixels can share the same set of end members and strengthen local sparsity, thereby significantly improving the accuracy of unmixing. At the same time, the reweighted spectral total variation regularization term imposes differential constraints on the spectral dimension of the nonlinear parameters, and ensures the smoothness of the nonlinear parameters in the spectrum and space through differential operations and reweighting factor matrices, effectively suppressing noise and overfitting problems, and further enhancing the robustness of the model.

[0017] 3. Integrate spatial-spectral information to optimize endmember extraction and abundance estimation: The method of this embodiment not only integrates the spatial and spectral information of the hyperspectral image, but also significantly improves the accuracy of endmember extraction and abundance estimation through fine feature extraction and regularization processing. This optimization enables the method to perform well in processing complex hyperspectral image data, providing more accurate and reliable technical support for remote sensing image processing, environmental monitoring, geological exploration and other fields.

[0018] In summary, this embodiment achieves a dual improvement in the accuracy and robustness of sparse unmixing of hyperspectral images through multi-scale spatial-spectral joint autoencoder network design and regularization processing strategy, which has significant technical advantages and application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific implementation methods and drawings. Here, the exemplary implementation methods of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0020] As used herein, the term "including" and its variations mean open inclusion, i.e., "including but not limited to". Unless otherwise stated, the term "or" means "and / or". The term "based on" means "based at least in part on". The terms "an example embodiment" and "an embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0021] Figure 1 A schematic diagram of a process for sparse unmixing of hyperspectral images provided in an embodiment of the present application; Figure 2 A schematic diagram of extracting multi-scale pooling spatial features provided in an embodiment of the present application; Figure 3 A schematic diagram of a process flow of a spectral self-attention mechanism provided in an embodiment of the present application; Figure 4 A flow chart of a hybrid attention perception mechanism provided in an embodiment of the present application; Figure 5 A schematic diagram of an abundance estimation result obtained on a synthetic data set provided in an embodiment of the present application; Figure 6 A schematic diagram of an abundance estimation result obtained on a Samson data set provided in an embodiment of the present application; Figure 7A schematic diagram of an abundance estimation result obtained on a Jasper Ridge dataset provided in an embodiment of the present application; Figure 8 A schematic diagram of the structure of a hyperspectral image sparse unmixing device provided in an embodiment of the present application; Fig. 9 An exemplary structural diagram of a device capable of implementing a method according to an embodiment of the present invention is provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0023] The following describes the embodiments of the present disclosure through specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present disclosure.

[0024] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present disclosure, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein may be used to implement this device and / or practice this method.

[0025] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. The drawings only show components related to the present disclosure rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0026] Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects described may be practiced without these specific details.

[0027] See also Figure 1 , Figure 1 A flow chart of a hyperspectral image sparse unmixing method provided in an embodiment of the present application can be executed by an electronic device, such as by one or more processors in the electronic device, to implement the following steps: S101, extracting features from the original hyperspectral image to obtain multi-scale spatial features and spectral features.

[0028] The feature extraction operation on the original hyperspectral image may include: three parallel pooling operations and a pooling space feature fusion processing operation, and the three parallel pooling operations may be maximum pooling, stripe pooling and average pooling respectively.

[0029] Optionally, in the process of extracting features from the original hyperspectral image to obtain multi-scale spatial features, the electronic device can first perform maximum pooling, stripe pooling and average pooling on the original hyperspectral image to obtain maximization results, stripe results and average results, respectively, and add the maximization results, stripe results and average results to obtain fused features, and then perform convolution and activation function processing on the fused features to obtain multi-scale spatial features.

[0030] See also Figure 2 , Figure 2 A schematic diagram of extracting multi-scale pooling spatial features provided in an embodiment of the present application is shown in FIG. Figure 2 Shown: The first pooling operation is the maximum pooling , used to capture the maximum value features in the image, which can highlight the significant details and edges in the image, and the maximum pooling The expression of is shown in formula (1): (1) in, is the original hyperspectral image.

[0031] The second pooling operation is stripe pooling, which obtains preliminary features by performing pooling operations on the input hyperspectral image in the horizontal and vertical directions, and then performs expansion operations through corresponding convolutions in different directions to upsample the features to restore them to the original resolution and obtain complete stripe features. This stripe feature helps to extract the integrity of the hyperspectral image in a certain direction and can fully reflect the consistency of spatial and spectral information. Horizontal pooling The expression of is shown in formula (2), vertical pooling The expression of is shown in formula (3): (2) (3) in, i Representative i row, j represents the j Columns; W represents the width of the image (columns), H represents the height of the image (rows), h represents the height direction, and w represents the width direction. The original hyperspectral image i Row, No. j The pixel radiance value of the column.

[0032] Formula (2) represents the i The radiation values ​​of all pixels in the row are averaged; formula (3) means that the radiation values ​​of all pixels in the jth column are averaged.

[0033] The third pooling operation is average pooling , which takes the average value of the pixel radiation value of each pooling window. Average pooling can smooth the features in the hyperspectral image, reduce the influence of noise, and retain the overall trend. It can effectively extract global spatial features. The expression of is shown in formula (4): (4) in, is the original hyperspectral image.

[0034] Correspondingly, after obtaining the maximization results, striping results and averaging results, the maximization results, striping results and averaging results can be fused, and then the fused features can be convolved and activated to obtain multi-scale spatial features. In this way, in subsequent steps, spatial information of different scales can be captured, the perception of complex land distribution can be enhanced, and more comprehensive spatial feature support can be provided for subsequent spatial-spectral joint feature fusion and unmixing.

[0035] Optionally, the above maximization results, striping results and averaging results can be fused by adding operations to obtain fused features , then perform convolution and activation function processing on the fused features to obtain multi-scale spatial features. For example, you can use 1×1 convolution and sigmoid activation function processing to obtain multi-scale spatial features .

[0036] Similarly, in the process of extracting features from the original hyperspectral image to obtain spectral features, the original hyperspectral image can be subjected to two-dimensional convolution, activation function processing, batch normalization, and spectral self-attention weight calculation in sequence to obtain the spectral features.

[0037] See also Figure 3 , Figure 3 A schematic diagram of a spectral self-attention mechanism (SAM) provided in an embodiment of the present application. Figure 3 As shown, the spectral self-attention mechanism process includes: two serially connected two-dimensional convolution layer processing (preferably, 1×1 two-dimensional convolution) and two spectral self-attention mechanism processing.

[0038] Specifically, the first 1×1 convolution is followed by batch normalization (BN) and SAM. Similarly, the second 1×1 convolution is followed by LeakyReLU activation, BN, and SAM.

[0039] For example, in the design of the spectral self-attention mechanism, the step size of all convolution processes is set to 1, and zero padding is used as the padding strategy to ensure the constancy of the spatial dimension. For the LeakyReLU activation function, the negative slope parameter of the activation function can be set to 0.2.

[0040] In addition, the output dimension of the first convolution process is set to a fixed value (48), while the output dimension of the second convolution process can be flexibly adjusted according to the number of end members of the specific data set, that is, according to the number of types of pure substances in the data set. It is worth noting that as an implementation of the self-attention mechanism, the SAM processing process does not change the dimension of the input data.

[0041] Therefore, the final output of the spectral feature extraction process is the spectral feature , the number of channels is exactly equal to the number of endmembers in the data set .

[0042] S102, concatenating multi-scale spatial features and spectral features to obtain original spatial-spectral joint features.

[0043] Specifically, the spectral self-attention mechanism is processed by adjusting the attention weight on the spectral feature channel to generate the spectral feature Subsequently, these spectral spatial features Features and multi-scale spatial features Perform concatenation operation on the channel dimension to fuse the information of the two and finally generate the original spatial spectrum joint feature .

[0044] Optionally, in the process of splicing multi-scale spatial features and spectral features to obtain original spatial-spectral joint features, the spectral features and multi-scale spatial features can be spliced ​​according to the channel dimension to obtain the original spatial-spectral joint features.

[0045] For example, you can use the concatenate function tf.concat in the tensorflow framework to concatenate two features according to the channel dimension.

[0046] S103, performing feature decoupling on the original spatial-spectral joint feature to obtain an enhanced spatial-spectral joint feature.

[0047] Although the splicing operation in step S102 above integrates features of different scales containing contextual information, the simple splicing method fails to fully exploit the complementarity between these features, resulting in a significant decrease in the unmixing accuracy.

[0048] Therefore, it is necessary to introduce the hybrid attention fusion based (HAFB) mechanism. The HAFB mechanism can assign different weights to different channels, thereby effectively highlighting important features and suppressing secondary features, achieving efficient and accurate fusion of multi-scale features, and thus obtaining a more comprehensive and accurate feature representation.

[0049] Optionally, in the process of performing feature decoupling on the original spatial-spectral joint features to obtain enhanced spatial-spectral joint features, a channel weight matrix can be generated, and the channel weight matrix and the original spatial-spectral joint features can be multiplied pixel by pixel to obtain weighted spatial-spectral joint features, and then the weighted spatial-spectral joint features can be cross-attention perception enhancement processed to obtain enhanced spatial-spectral joint features.

[0050] Specifically, see Figure 4 , Figure 4 A flow chart of a hybrid attention perception mechanism provided in an embodiment of the present application. The HAFB mechanism is a mechanism designed specifically for processing spatial-spectral joint features, and may include: a channel attention fusion stage and an attention-based perception enhancement stage, a total of two core stages.

[0051] like Figure 4 As shown in Figure 1, in the channel attention fusion stage (stage 1), deep learning can be used to generate the channel weight matrix. First, the original spatial-spectral joint feature of the input The pooling layer is used to extract global information and reduce the amount of computation. Next, the output of the pooling layer is passed to the fully connected layer to further fuse and transform the information to generate an intermediate feature vector, which is the output of the fully connected layer. Finally, the intermediate feature vector is nonlinearly transformed by the sigmoid activation function to generate a channel weight matrix between 0 and 1. , this matrix is ​​used to represent the importance of each channel to the original spatial spectrum joint features.

[0052] In order to apply these weights to the input original spatial-spectral joint features, a pixel-by-pixel multiplication operation can be performed, that is, the original spatial-spectral joint feature value of each channel is multiplied by the weight corresponding to the original spatial-spectral joint feature to obtain the weighted spatial-spectral joint feature.

[0053] The goal of the attention-based perceptual enhancement stage (stage 2) is to further enhance the global dependencies between the spatial-spectral joint features. Figure 4 As shown, the electronic device can introduce two cross-attention processes. The two cross-attention processes process different dimensions (e.g., horizontal and vertical directions) of the feature map respectively and capture the correlation between different positions.

[0054] In this way, HAFB can identify and enhance the dependencies between pixels that are most critical to the overall feature representation. Finally, the outputs of the two cross-attention processes are merged to form the enhanced spatial-spectral joint features. By splicing the multi-scale spatial features and spectral features extracted from the original hyperspectral image and inputting them into the HAFB mechanism for feature decoupling and enhancement, more accurate and rich enhanced spatial-spectral joint features can be obtained.

[0055] Optional, such as Figure 4 As shown, first, the original empty spectrum is combined with the feature The input is sent to the HAFB mechanism and goes through the channel attention fusion stage. This stage passes through the pooling layer, the fully connected layer, the ReLU activation function, the fully connected layer again, and the sigmoid activation function to calculate the channel weight matrix. .

[0056] Then, the weight matrix Combine features with the original empty spectrum of the input Multiply each pixel by pixel and convert the result Input to a 1*1 convolutional layer, the number of output channels of the convolutional layer is set to the number of end members. In the attention-based perception enhancement stage, each round of cross-attention processing only involves Figure 4 The cross area in .

[0057] Since the second round of attention mechanism is performed on the features generated in the first round, equivalent global attention can be achieved at the end of stage 2, which will establish effective global dependencies between pixels in the original spatial-spectral joint features, and can enhance the spatial-spectral features based on global spatial attention to obtain enhanced spatial-spectral joint features.

[0058] Because the attention mechanism does not change the spatial structure and number of channels of the data, the resulting enhanced spatial-spectral joint feature , the spatial shape remains unchanged, and the number of channels is the number of end members.

[0059] S104, normalizing the enhanced spatial spectrum joint features according to a preset weight matrix to obtain a preliminary abundance matrix.

[0060] In the fields of hyperspectral image processing and remote sensing data analysis, spatial-spectral joint features are an important source of information. They combine spatial features and spectral features to more comprehensively describe the characteristics of the objects. In order to extract useful information from these features, it is usually necessary to perform normalization and generate an abundance matrix.

[0061] The abundance matrix reflects the distribution of different ground object components in the image, while the endmember matrix represents the basic spectral characteristics of these components. Through the abundance matrix and endmember matrix, tasks such as ground object classification and anomaly detection can be achieved.

[0062] Optionally, the normalization layer performs normalization on the enhanced spatial-spectral joint features. The function of the normalization layer is to scale and translate the enhanced spatial-spectral joint features so that the enhanced spatial-spectral joint features conform to a certain distribution (such as a normal distribution), thereby eliminating the dimensional differences and numerical range differences between different enhanced spatial-spectral joint features. The normalization layer may use a variety of methods, such as the Z-score normalization method, to process the enhanced spatial-spectral joint features.

[0063] The enhanced spatial-spectral joint features after normalization layer processing are used to generate a preliminary abundance matrix. The preliminary abundance matrix is ​​a two-dimensional array, in which each element represents the abundance (i.e., proportion) of the corresponding ground object component in the image. Methods for generating a preliminary abundance matrix may include non-negative matrix factorization (NMF) and sparse representation. The goal of these methods for generating preliminary abundance matrices is to decompose the normalized enhanced spatial-spectral joint features into a series of end-member components and corresponding abundance values.

[0064] The decoder is usually a component in a deep learning model that maps the enhanced spatial-spectral joint features back to the original data space or generates a new data representation. The pre-set weight matrix usually contains important information about the endmember components, and each row of the pre-set weight matrix may represent the spectral characteristics of an endmember component.

[0065] S105, iteratively correcting the preliminary abundance matrix and the preset weight matrix through a preset loss function in combination with a preset neural network to obtain a sparse abundance matrix and an endmember matrix.

[0066] Specifically, the loss function is a measure of the difference between the prediction results of the neural network and the actual data. The neural network can continuously adjust the preliminary abundance matrix and the pre-set weight matrix through nonlinear fitting capabilities to minimize the loss function.

[0067] In the process of iteratively correcting the preliminary abundance matrix and the preset weight matrix, the preliminary abundance matrix and the preset weight matrix are used as starting points to calculate the loss function value for the preliminary abundance matrix and the preset weight matrix input into the neural network.

[0068] Correspondingly, the weights and biases of the neural network can be adjusted through the back propagation algorithm according to the gradient information of the loss function, and the element values ​​of the current abundance matrix and the current weight matrix can also be directly adjusted. Moreover, the abundance matrix and the weight matrix can be updated using some optimization algorithm (such as gradient descent, Adam, etc.) according to the gradient information obtained by back propagation.

[0069] The above process is repeated until the loss function value converges to a smaller value or reaches a preset number of iterations, completing the iterative correction of the preliminary abundance matrix and the preset weight matrix to obtain the sparse abundance matrix and endmember matrix.

[0070] It should be noted that the weight matrix completed by iteration is the end member matrix. The loss function may include: a reweighted collaborative sparse regularization term and a reweighted spectral total variation regularization term.

[0071] Optionally, in the process of iteratively correcting the preliminary abundance matrix and the preset weight matrix to obtain a sparse abundance matrix and an endmember matrix, the original hyperspectral image can be input into the neural network, combined with the pre-set network parameters corresponding to the neural network, and the neural network can be unsupervisedly trained through a loss function to obtain a trained neural network; the enhanced spatial-spectral joint features are input into the fully connected layer and the normalization layer of the trained neural network, and combined with the reweighted collaborative sparse regularization term in the loss function to generate a sparse abundance matrix; and the endmember matrix is ​​generated according to the weight matrix corresponding to the decoder in the trained neural network.

[0072] For example, the loss function may include: spectral reconstruction error , reweighted collaborative sparse regularization term and reweighted spectral total variation regularization term .

[0073] Correspondingly, the formula of the loss function can be: (5) in, and is the regularization coefficient.

[0074] and ,in, Indicates The true spectral vector of pixels; Indicates The true spectral vector of pixels; Indicates the total number of pixels; Represents vector dot product operation; Represents a vector Norm.

[0075] in addition, ,in is the weight matrix, is the preliminary abundance matrix, Indicates element-by-element multiplication; also, ,in For the preliminary abundance matrix The difference operation, is the weight matrix, Represents element-by-element multiplication.

[0076] Further, , The calculation methods are shown in formula (6) and formula (7): in, is the total number of end members, is the number of pixels in the hyperspectral image, and Respectively represent Original hyperspectral pixels and reconstructed hyperspectral pixels of pixels; Represents the Hadamard product (element-by-element multiplication); is the abundance matrix; is the reweighting factor matrix used to adjust the sparsity, Elements Updated in each iteration, the update formula is shown in formula (8): (8) in is a small positive number used to prevent the denominator from being zero.

[0077] Right now norm, which can be expressed as formula (9): in, Representative Pixels The abundance value of the end member, Indicates the number of pixels.

[0078] The abundance sparse loss function aims to sparse the abundance matrix To achieve the suppression of redundant abundance. Each row corresponds to Each column of Some columns in become close to zero, reducing the influence of these columns in the optimization process. This sparsity constraint helps improve the robustness of endmember and abundance estimates, especially for noisy data.

[0079] The calculation method can be expressed as formula (10): in, is the abundance matrix; Express Differential operation; is the reweighting factor matrix used to penalize non-zero entries; express The norm makes the difference matrix sparse; the reweighting factor matrix Elements Updated in each iteration, the update formula is formula (11): (11) in Again, a small positive number to prevent the denominator from being zero.

[0080] S106, obtaining a reconstructed hyperspectral image according to the sparse abundance matrix and the endmember matrix.

[0081] In deep learning, neural networks can extract features of input data through convolution operations. 1x1 convolution is a special convolution operation with a convolution kernel size of 1x1. This convolution operation does not change the spatial dimensions (height and width) of the input data, but can change the number of channels of the data (that is, the number of features).

[0082] In this embodiment, the number of convolution kernels is set to the number of spectral channels of the data set, and the number of convolution kernels of the 1x1 convolution layer is set to be the same as the number of spectral channels in the data set. This means that each convolution kernel corresponds to a specific spectral channel or feature, and through this convolution kernel, the neural network can learn how to extract features corresponding to each spectrum from the input data.

[0083] It should be noted that the number of spectral channels of the dataset is defined as the number of spectral bands for each pixel in the hyperspectral image, for example, the synthetic dataset has 200 bands and the Samson dataset has 156 bands.

[0084] After the neural network training is completed, the weight matrix of the convolution layer is used as the end member extraction result. During the training process, the neural network continuously adjusts the weight of the convolution kernel through the back propagation algorithm to minimize the loss function and thus optimize the performance of the model.

[0085] Moreover, the weight matrix of the 1x1 convolution layer (i.e., the weight of each convolution kernel) will contain information about the spectral features extracted from the input data. These weight matrices are regarded as the extracted endmember results. Finally, the decoder reconstructs the sparse abundance matrix and the endmember matrix to obtain the reconstructed hyperspectral image. The unmixing task of the hyperspectral image is completed.

[0086] It should be noted that the distribution ratio of the endmembers of the reconstructed hyperspectral image in space is obtained according to the sparse abundance matrix, and the spectral information of the reconstructed hyperspectral image is obtained according to the endmember matrix.

[0087] It should also be noted that the reconstructed hyperspectral image is obtained according to the sparse abundance matrix and the endmember matrix, and the reconstructed hyperspectral image is obtained by multiplying the sparse abundance matrix and the endmember matrix.

[0088] In this implementation, the sparse unmixing method of the hyperspectral image provided in this embodiment is also simulated and verified. The details are as follows: The embodiments of the present invention are carried out in the Python 3.10.0 environment, using the TensorFlow 2.10.0 deep learning framework. The network is trained on an NVIDIA GeForce RTX 4090 GPU and an Intel® Core™ i9-13900KF processor. The detailed hyperparameter settings of the model include: the step size of the convolutional layer is set to 1, and the same padding strategy is adopted to keep the spatial shape unchanged. The negative slope of the LeakyReLU activation function is set to 0.2, and the dropout rate of the Dropout layer is also set to 0.2 to prevent the gradient from disappearing and prevent overfitting. The final model is trained by the Adam optimizer, and the initial learning rate is set to 0.005. During the training process, the batch size is set to 32 and the training round is set to 50 rounds to achieve optimal performance.

[0089] Experimental data and comparative experiments: The synthetic data used in the embodiment of the present invention is a 60×60 pixel image, which selects five different end members from the ASTER spectral library, covering 200 bands from 0.4 microns to 14 microns, and the abundance values ​​of the materials are generated according to the Dirichlet distribution to simulate the complex real-world mixing situation. In order to evaluate the robustness of the present invention, the data set also introduces Gaussian white noise with a signal-to-noise ratio of 20db.

[0090] The real data sets used in the embodiments of the present invention are the Samson and Jasper Ridge data sets; the image size of the Samson data set is 95×95, the wavelength range is from 380 to 2500 nanometers, and it has 156 channels. The method includes three typical surface covers, soil, trees, and water, as end members; the image size of the Jasper Ridge data set is 100×100 pixels, the wavelength range is from 400 to 2500 nanometers, and it has 192 bands, including four end members: roads, soil, water, and trees.

[0091] The hyperspectral sparse unmixing method (Our) provided in this application is experimentally compared with five advanced hyperspectral unmixing methods that are also based on deep learning to demonstrate the experimental effects of endmember extraction and abundance estimation of the present invention. All the compared methods use the steps described in the original paper to set the relevant hyperparameters.

[0092] It should be noted that the five advanced hyperspectral unmixing methods that are also based on deep learning are: 2DCNN (2D convolutional neural unmixing network), DAEU (deep autoencoder network), Deep Trans (deep Transformer network), TANET (two-stream autoencoder network), and CyCU-Net (cycle consistency unmixing network).

[0093] In order to fully evaluate the robustness and effectiveness of the model in the unmixing task, the embodiment of the present invention adopts two evaluation indicators: spectral angular distance (SAD) and root mean square error (RMSE); the lower the SAD value, the closer the extracted endmember is to the true endmember; if the RMSE value is lower, it means that the estimated abundance is closer to the true abundance. The calculation methods are shown in formula (12) and formula (13) respectively: (12) (13) in, represents the real endmember vector of the jth endmember, Represents the extracted endmember vector of the jth endmember. is the true abundance of the j-th pixel, represents the corresponding estimated abundance value.

[0094] Table 1 shows the experimental results of the 20db synthetic dataset.

[0095] Table 1 From Table 1, it can be seen that, except for the asphalt end member, the other end members of the embodiment of the present application have achieved the best results, and the average SAD obtained is also the smallest. At the same time, in terms of abundance, the present invention is also the best among the vast majority of end members, and has achieved the best average RMSE; Figure 5 The abundance maps and end-member comparison curves of all methods on synthetic data are shown respectively. It can be concluded from observation that the abundance maps of the method provided in the embodiment of the present application are visually closest to the true value (GT) in asphalt, concrete, limestone, conifer and basalt.

[0096] Table 2 shows the experimental results of the Samson dataset.

[0097] Table 2 Table 2 shows the experimental results of different methods on the Samson dataset. As can be seen from the table, the embodiment of the present application achieved the best SAD results in all end members. In terms of abundance, except for the soil end member, other end members also achieved the best results and the smallest average RMSE value. Figure 6 The visualization results of abundance estimation and endmember extraction on the Samson dataset are shown respectively. It can be seen that the method provided in the embodiment of the present application is very close to the true value (GT) in soil (Soil), tree (Tree), and water (Water).

[0098] Table 3 shows the experimental results of the Jasper Ridge dataset.

[0099] Table 3 Table 3 shows the experimental results of various methods on the Jasper Ridge dataset. From the data in the table, it can be seen that the embodiment of the present application not only achieved the best average SAD value and average RMSE value, but also achieved the best or suboptimal results in the endmember extraction and abundance estimation results of a single endmember. Figure 7 The visualization results of abundance estimation and end member extraction are shown respectively. Figure 7It can be seen that the abundance map estimated by the method provided in the embodiment of the present application has the highest restoration degree in road (Road), soil (Soil), tree (Tree), and water (Water), and is extremely close to the true value (GT).

[0100] In summary, compared with the prior art, this embodiment has the following advantages: 1. Enhanced feature extraction and fusion capabilities: The sparse unmixing method for hyperspectral images proposed in this embodiment fully considers and effectively utilizes the three-dimensional properties of hyperspectral images, namely spatial and spectral properties, through the design of parallel multi-scale pooling spatial feature processing and spectral self-attention processing. After the features output by the dual processing process are blended, the information receiving domain of the model is further broadened through the HAFB mechanism, and the feature weights of the two dimensions (space and spectrum) are adjusted, thereby enhancing the global expression ability of the features. This design enables the model to more effectively connect contextual information with information between the spectrum and space, enriches the level and details of the features, and provides a more accurate and comprehensive feature basis for subsequent end member extraction and abundance estimation.

[0101] 2. Improve the accuracy and robustness of unmixing: This embodiment constructs a loss function including a reweighted collaborative sparse regularization term and a reweighted spectral total variation regularization term by calculating the spatial homogeneity of each end member of the hyperspectral image. The reweighted collaborative sparse regularization term utilizes the spatial similarity of abundance in local areas of the hyperspectral image, and imposes row sparsity constraints on the abundance matrix through superpixel segmentation and reweighting factor matrix, so that adjacent pixels can share the same set of end members and strengthen local sparsity, thereby significantly improving the accuracy of unmixing. At the same time, the reweighted spectral total variation regularization term imposes differential constraints on the spectral dimension of the nonlinear parameters, and ensures the smoothness of the nonlinear parameters in the spectrum and space through differential operations and reweighting factor matrices, effectively suppressing noise and overfitting problems, and further enhancing the robustness of the model.

[0102] 3. Integrate spatial-spectral information to optimize endmember extraction and abundance estimation: The method of this embodiment not only integrates the spatial and spectral information of the hyperspectral image, but also significantly improves the accuracy of endmember extraction and abundance estimation through fine feature extraction and regularization processing. This optimization enables the method to perform well in processing complex hyperspectral image data, providing more accurate and reliable technical support for remote sensing image processing, environmental monitoring, geological exploration and other fields.

[0103] In summary, this embodiment achieves a dual improvement in the accuracy and robustness of sparse unmixing of hyperspectral images through multi-scale spatial-spectral joint autoencoder network design and regularization processing strategy, which has significant technical advantages and application prospects.

[0104] The second embodiment of the present invention provides a hyperspectral image sparse unmixing device, see Figure 8 , Figure 8 A schematic diagram of the structure of a hyperspectral image sparse unmixing device provided in an embodiment of the present application includes: An extraction module 810 is used to extract features from the original hyperspectral image to obtain multi-scale spatial features and spectral features; A splicing module 820 is used to splice the multi-scale spatial feature and the spectral feature to obtain an original spatial-spectral joint feature; A decoupling module 830 is used to perform feature decoupling on the original spatial-spectral joint feature to obtain an enhanced spatial-spectral joint feature; A normalization module 840 is used to normalize the enhanced spatial spectrum joint feature according to a preset weight matrix to obtain a preliminary abundance matrix; The training module 850 is used to iteratively correct the preliminary abundance matrix and the preset weight matrix by using a preset loss function in combination with a preset neural network to obtain a sparse abundance matrix and an end member matrix, wherein the loss function includes: a reweighted collaborative sparse regularization term and a reweighted spectral total variation regularization term; The reconstruction module 860 is used to reconstruct the sparse abundance matrix to obtain a reconstructed hyperspectral image.

[0105] Optionally, the extraction module performs maximum pooling, stripe pooling and average pooling on the original hyperspectral image to obtain a maximization result, a stripe result and an average result, respectively; the maximization result, the stripe result and the average result are added to obtain a fusion feature; and the fusion feature is processed by convolution and activation function to obtain a multi-scale spatial feature.

[0106] Optionally, the splicing module splices the spectral features and the multi-scale spatial features according to the channel dimension to obtain the original spatial-spectral joint features.

[0107] Optionally, the decoupling module performs two-dimensional convolution, activation function processing, batch normalization, and spectral self-attention weight calculation on the original hyperspectral image in sequence to obtain the spectral features.

[0108] Optionally, the decoupling module generates a channel weight matrix, multiplies the channel weight matrix and the original spatial-spectral joint features pixel by pixel, and obtains weighted spatial-spectral joint features; and performs cross-attention perception enhancement processing on the weighted spatial-spectral joint features to obtain enhanced spatial-spectral joint features.

[0109] Optionally, a training module inputs the original hyperspectral data into the neural network, combines the pre-set network parameters corresponding to the neural network, performs unsupervised training on the neural network through a loss function, and obtains a trained neural network; inputs the enhanced spatial-spectral joint features into the fully connected layer and normalization layer of the trained neural network, and combines the reweighted collaborative sparse regularization term in the loss function to generate a sparse abundance matrix; generates an end member matrix according to the weight matrix corresponding to the decoder in the trained neural network.

[0110] In summary, the sparse unmixing device for hyperspectral images provided in the embodiment of the present application fully considers and effectively utilizes the three-dimensional properties of hyperspectral images, namely, spatial and spectral properties. After blending the output features, the feature weights of the two dimensions (spatial and spectral) are adjusted, thereby enhancing the global expression ability of the features. This design enables the sparse unmixing device for hyperspectral images to more effectively connect contextual information with spectral and spatial information, enrich the level and details of the features, and provide a more accurate and comprehensive feature basis for subsequent end member extraction and abundance estimation.

[0111] The hyperspectral image sparse unmixing device provided in this embodiment calculates the spatial homogeneity of each end member of the hyperspectral image, and constructs a loss function including a reweighted collaborative sparse regularization term and a reweighted spectral total variation regularization term. The reweighted collaborative sparse regularization term utilizes the spatial similarity of abundance in the local area of ​​the hyperspectral image, and imposes row sparsity constraints on the abundance matrix through superpixel segmentation and reweighting factor matrix, so that adjacent pixels can share the same set of end members, and strengthens local sparsity, thereby significantly improving the unmixing accuracy. At the same time, the reweighted spectral total variation regularization term imposes differential constraints on the spectral dimension of the nonlinear parameters, and ensures the smoothness of the nonlinear parameters in the spectrum and space through differential operations and reweighting factor matrices, effectively suppressing noise and overfitting problems, and further enhancing the robustness of the model.

[0112] The hyperspectral image sparse unmixing device provided in this embodiment not only integrates the spatial and spectral information of the hyperspectral image, but also significantly improves the accuracy of endmember extraction and abundance estimation through fine feature extraction and regularization processing. This optimization enables the method to perform well in processing complex hyperspectral image data, providing more accurate and reliable technical support for remote sensing image processing, environmental monitoring, geological exploration and other fields.

[0113] A third embodiment of the present invention further provides an electronic device, the electronic device comprising: at least one processor; and a memory in communication with the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the hyperspectral image sparse unmixing method of any of the aforementioned embodiments.

[0114] The fourth embodiment of the present invention further provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable the computer to execute the hyperspectral image sparse unmixing method of any of the aforementioned embodiments.

[0115] The fifth embodiment of the present invention further provides a computer program product, which includes a computing program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the hyperspectral image sparse unmixing method of any of the aforementioned embodiments.

[0116] The sixth embodiment of the present invention further provides a computer program, which includes program instructions. When the program instructions are executed by a computer, the computer is enabled to execute the hyperspectral image sparse unmixing method of any of the above embodiments.

[0117] Fig. 9 A schematic diagram of a method or device 1000 that can implement an embodiment of the present invention or implement an embodiment of the present invention provided in the present application may include more or fewer devices than shown in the figure in some embodiments. In some embodiments, it can be implemented using a single or multiple devices. In some embodiments, it can be implemented using cloud or distributed devices.

[0118] like Fig. 9 As shown, the device 1000 includes a processor 1001, which can perform various appropriate operations and processes according to the program and / or data stored in the read-only memory (ROM) 1002 or the program and / or data loaded from the storage part 1008 to the random access memory (RAM) 1003. Processor 1001 can be a multi-core processor or include multiple processors. In some embodiments, processor 1001 can include a general main processor and one or more special coprocessors, such as a central processing unit (CPU), a graphics processing unit (GPU), a neural network processor (NPU), a digital signal processor (DSP), etc. In RAM 1003, various programs and data required for the operation of device 1000 are also stored. Processor 1001, ROM 1002 and RAM 1003 are connected to each other via bus 1004. Input / output (I / O) interface 1005 is also connected to bus 1004.

[0119] The processor and the memory are used together to execute the program stored in the memory. When the program is executed by the computer, the methods, steps or functions described in the above embodiments can be implemented.

[0120] The following components are connected to the I / O interface 1005: an input section 1006 including a keyboard, a mouse, a touch screen, etc.; an output section 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card, a modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed, so that a computer program read therefrom is installed into the storage section 1008 as needed. Fig. 9 Only some components are schematically shown in the figure, which does not mean that the device 1000 only includes Fig. 9 Components shown.

[0121] The systems, devices, modules or units described in the above embodiments may be implemented by a computer or its associated components. The computer may be, for example, a mobile terminal, a smart phone, a personal computer, a laptop computer, a vehicle-mounted human-computer interaction device, a personal digital assistant, a media player, a navigation device, a game console, a tablet computer, a wearable device, a smart TV, an Internet of Things system, a smart home, an industrial computer, a server or a combination thereof.

[0122] Although not shown, in an embodiment of the present invention, a computer-readable storage medium is provided, on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the hyperspectral image sparse unmixing method described in Example 1 is implemented.

[0123] Storage media in embodiments of the present invention include permanent and non-permanent, removable and non-removable items that can be used to store information by any method or technology. Examples of storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0124] Although not shown, an embodiment of the present invention further provides a computer program product, including: a computer program / instruction, which implements the hyperspectral image sparse unmixing method described in Example 1 when executed by a processor.

[0125] The methods, programs, systems, devices, etc. of the embodiments of the present invention may be executed or implemented in a single or multiple networked computers, or may be practiced in a distributed computing environment. In the embodiments of this specification, in these distributed computing environments, tasks may be performed by remote processing devices connected via a communication network.

[0126] Those skilled in the art should understand that the embodiments of the present specification may be provided as methods, systems or computer program products. Therefore, those skilled in the art may imagine that the functional modules / units or controllers and related method steps described in the above embodiments may be implemented in software, hardware or a combination of software / hardware.

[0127] Unless explicitly stated, the actions or steps of the methods, programs, and embodiments of the present invention do not have to be performed in a specific order and can still achieve the desired results. In some implementations, multitasking and parallel processing are also possible or may be advantageous.

[0128] In this article, multiple embodiments of the present invention are described, but for the sake of brevity, the description of each embodiment is not exhaustive, and the same or similar features or parts between the embodiments may be omitted. In this article, "one embodiment", "some embodiments", "example", "specific example", or "some examples" are intended to be applicable to at least one embodiment or example of the present invention, but not all embodiments. The above terms do not necessarily mean to refer to the same embodiment or example. In the absence of mutual contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples.

[0129] The exemplary systems and methods of the present invention have been specifically shown and described with reference to the above embodiments, which are merely examples of the best modes for implementing the present systems and methods. It will be appreciated by those skilled in the art that various changes may be made to the embodiments of the systems and methods described herein when implementing the present systems and / or methods without departing from the spirit and scope of the present invention as defined in the appended claims.

Claims

1. A sparse unmixing method for hyperspectral images, characterized in that: include: Extract features from the original hyperspectral image to obtain multi-scale spatial features and spectral features; splicing the multi-scale spatial features and the spectral features to obtain original spatial-spectral joint features; Performing feature decoupling on the original spatial-spectral joint feature to obtain an enhanced spatial-spectral joint feature; Normalizing the enhanced spatial spectrum joint features according to a preset weight matrix to obtain a preliminary abundance matrix; The preliminary abundance matrix and the weight matrix are iteratively corrected by a preset loss function in combination with a preset neural network to obtain a sparse abundance matrix and an end member matrix, wherein the loss function includes: a reweighted collaborative sparse regularization term and a reweighted spectral total variation regularization term; A reconstructed hyperspectral image is obtained according to the sparse abundance matrix and the endmember matrix.

2. The hyperspectral image sparse unmixing method according to claim 1, characterized in that: The feature extraction of the original hyperspectral image to obtain multi-scale spatial features includes: Performing maximum pooling, stripe pooling and average pooling on the original hyperspectral image to obtain a maximization result, a stripe result and an average result, respectively; Adding the maximization result, the striping result and the averaging result to obtain a fusion feature; The fused features are subjected to convolution and activation function processing to obtain the multi-scale spatial features.

3. The hyperspectral image sparse unmixing method according to claim 1, characterized in that: The feature extraction of the original hyperspectral image to obtain the spectral features includes: The original hyperspectral image is sequentially subjected to two-dimensional convolution, activation function processing, batch normalization and spectral self-attention weight calculation to obtain the spectral feature.

4. The hyperspectral image sparse unmixing method according to claim 1, characterized in that: The step of splicing the multi-scale spatial features and the spectral features to obtain the original spatial-spectral joint features includes: The spectral features and the multi-scale spatial features are spliced ​​according to the channel dimension to obtain the original spatial-spectral joint features.

5. The hyperspectral image sparse unmixing method according to claim 1, characterized in that: The step of performing feature decoupling on the original spatial-spectral joint feature to obtain an enhanced spatial-spectral joint feature includes: Generate a channel weight matrix, and multiply the channel weight matrix and the original spatial-spectral joint feature pixel by pixel to obtain a weighted spatial-spectral joint feature; The weighted spatial-spectral joint features are subjected to cross-attention perception enhancement processing to obtain the enhanced spatial-spectral joint features.

6. The hyperspectral image sparse unmixing method according to any one of claims 1 to 5, characterized in that: The method of iteratively correcting the preliminary abundance matrix and the weight matrix by using a preset loss function in combination with a preset neural network to obtain a sparse abundance matrix and an end member matrix includes: Inputting the original hyperspectral image into the neural network, combining the preset network parameters corresponding to the neural network, and performing unsupervised training on the neural network through the loss function to obtain a trained neural network; Inputting the enhanced spatial-spectral joint features into the fully connected layer and the normalization layer of the trained neural network, and combining the reweighted collaborative sparse regularization term in the loss function to generate the sparse abundance matrix; The weight matrix corresponding to the decoder in the trained neural network is extracted to obtain the end member matrix.

7. The hyperspectral image sparse unmixing method according to any one of claims 1 to 5, characterized in that: The loss function includes: spectral reconstruction error , the reweighted collaborative sparse regularization term and the reweighted spectral total variation regularization term ; The formula of the loss function is: ， in, and is the regularization term coefficient; ,in, Indicates The true spectral vector of pixels; Indicates The true spectral vector of pixels; Indicates the total number of pixels; Represents vector dot product operation; Represents a vector norm; ,in is the weight matrix, is the preliminary abundance matrix, Indicates element-by-element multiplication; ,in For the preliminary abundance matrix The difference operation, is the weight matrix, Represents element-by-element multiplication.

8. A hyperspectral image sparse unmixing device, characterized in that: include: An extraction module is used to extract features from the original hyperspectral image to obtain multi-scale spatial features and spectral features; A splicing module, used for splicing the multi-scale spatial features and the spectral features to obtain original spatial-spectral joint features; A decoupling module, used for performing feature decoupling on the original spatial-spectral joint feature to obtain an enhanced spatial-spectral joint feature; A normalization module, used for normalizing the enhanced spatial spectrum joint features according to a preset weight matrix to obtain a preliminary abundance matrix; A training module, for iteratively correcting the preliminary abundance matrix and the weight matrix through a preset loss function in combination with a preset neural network to obtain a sparse abundance matrix and an end member matrix, wherein the loss function includes: a reweighted collaborative sparse regularization term and a reweighted spectral total variation regularization term; The reconstruction module is used to reconstruct the sparse abundance matrix and the endmember matrix to obtain a reconstructed hyperspectral image.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the hyperspectral image sparse unmixing method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the hyperspectral image sparse unmixing method according to any one of claims 1 to 7.

Images