A hyperspectral feature extraction method, device and storage medium

Abstract

The present invention relates to a hyperspectral feature extraction method, device and computer storage medium. The hyperspectral feature extraction method based on multimodal attention coding proposed in the present invention realizes the effective extraction of two kinds of sequential information in pixel space data by constructing a spatial dimensional sequential feature extraction module, removes the influence of the sequential direction of the sequential feature extraction process on the performance through a bidirectional long short-term memory layer, and constructs different attention weight matrices for different sequential information to strengthen the important parts of the sequential information, realize high-performance fusion of information, and finally effectively improve the expression ability of the extracted features.

Description

Technical Field

[0001] The present invention relates to the field of remote sensing technology, and in particular to a method and device for extracting hyperspectral features, and a computer storage medium. Background Art

[0002] In the field of remote sensing, hyperspectral imaging technology is widely used in a variety of studies. Hyperspectral images contain extremely rich spatial information and spectral information, where spatial information refers to the specific spatial position information of a single pixel at each spectral wavelength, and spectral information refers to the continuous spectral curve composed of the spectral reflectance of a single pixel at each spectral wavelength. By conducting feature extraction research on hyperspectral images, embedded features containing important discriminant information can be obtained, reducing the redundant information of the image, and further improving the recognition accuracy in subsequent classification research. In the early days, hyperspectral image feature extraction methods mainly extracted the spectral features of pixels, and did not consider the differences between the position information of different pixels. Therefore, these methods were difficult to obtain good classification results. In recent years, with the rapid increase in computer computing power and the continuous deepening of deep learning research, a large number of neural network methods for this research field have been proposed one after another. These methods introduce the theoretical idea of ​​multi-sensor data fusion, construct mappings for spatial information and spectral information for extraction, and finally realize feature fusion. This type of method avoids the loss of a large amount of important hyperspectral information, thereby further improving the classification performance of related algorithms.

[0003] From the perspective of data diversity, hyperspectral feature extraction methods can generally be divided into feature extraction methods based on spectral information and feature extraction methods based on spatial-spectral information.

[0004] The hyperspectral feature extraction method based on spectral information mainly uses a single spectral curve in the hyperspectral image to construct the feature extraction model, and does not consider the spatial position of the pixel at the spatial level, which belongs to single-modal learning. The commonly used methods in the early days generally include principal component analysis (Principle Component Analysis, PCA), minimum noise separation (Minimum Noise Fraction, MNF), linear discriminant analysis (Linear Discriminant Analysis, LDA), etc. At the same time, in order to deal with nonlinear data, some manifold learning-based methods have also been proposed, such as Local Linear Embedding (Local Linear Embedding, LLE), Laplacian Eigenmaps (Laplacian Eigenmaps, LE), Multidimensional Scaling (Multidimensional Scaling, MDS), etc. Such methods generally only consider the important discriminant information inside the hyperspectral pixel to ensure the classification ability of the pixel. In recent years, with the continuous deepening of deep learning research, some deep network models have also been widely used in research related to hyperspectral image feature extraction, such as Auto-encoder (AE), Variational Auto-encoder (VAE), Long Short Term Memory (LSTM), etc. However, such methods do not consider the spatial position relationship of hyperspectral pixels, and therefore only describe the discriminant information in hyperspectral images in detail from the spectral level. They lack important spatial information and cannot give full play to the advantage of "image-spectrum integration" of hyperspectral images, that is, there is synergy between spatial information and spectral information in the image. At present, the mainstream hyperspectral feature extraction method in this research field is the feature extraction method based on spatial-spectral information.

[0005] Hyperspectral images are a typical three-dimensional cube data, containing two spatial dimensions and one spectral dimension. This kind of data interactively combines the local spatial information of the target object pixel with the unique spectral information at each wavelength, and reflects them together in the complete hyperspectral image. Therefore, the hyperspectral image has the important feature of "image-spectrum integration", that is, the spatial information and spectral information of the hyperspectral image are consistent and synergistic. At the same time, due to the complex and changeable shooting scenes of hyperspectral images and the information interference of the shooting environment, there will be "one object, different spectrum" and "one spectrum, different objects" phenomena in hyperspectral images, which will seriously affect the analysis results of the interference hyperspectral pixels. The spatial information of hyperspectral images refers to the information composed of local neighborhood pixels of a single pixel at the spatial level. By assuming that each spectral pixel has a certain contribution relationship or correlation relationship with its local neighborhood pixels, it is possible to learn the local space and then master the multiple discriminant information of the pixel in the real object. The current method generally uses a convolutional neural network to perform convolution operations on the information of the pixel in the local receptive field, obtain the spatial features of the pixel, and combine it with other features to achieve multimodal learning. However, this type of method has the following defects: ① Convolutional neural networks and other methods are usually used for feature extraction, but the important areas in the information are not strengthened, and the important parts of the local information cannot be highlighted. Instead, they are processed in a consistent manner, which makes it difficult to highlight the key points of the information; ② It is impossible to mine the unique sequential information in the spatial-spectral information, and the differences in sequential information in different sequential directions are not considered. When extracting the sequential information in the spatial-spectral information, the influence of the sequential direction on the extraction effect is not considered. ③ When using the attention mechanism, the construction method of the weight matrix is ​​too single. ④ The fusion method of spatial information and spectral information is too simple and relatively single, and it is unable to fully realize the fusion of modal information (spatial information and spectral information). Summary of the invention

[0006] Therefore, the technical problem to be solved by the present invention is to overcome the problem of low accuracy of hyperspectral feature extraction in the prior art.

[0007] In order to solve the above technical problems, the present invention provides a hyperspectral feature extraction method, comprising: selecting a hyperspectral image for normalization processing to obtain an original domain hyperspectral image X×Y×B, where X and Y are the spatial dimensions of the hyperspectral image at each wavelength, and B is the number of bands of the hyperspectral image, setting a neighborhood sliding window s, and an embedding dimension d, and selecting, for each hyperspectral pixel of the original domain hyperspectral image, a neighborhood pixel with a size of s×s around it as the spatial data of the pixel, to obtain a neighborhood hyperspectral spatial data set;

[0008] Inputting the original domain hyperspectral image and the neighborhood hyperspectral spatial dataset into a pre-trained hyperspectral deep network model;

[0009] The neighborhood hyperspectral spatial data set is input into a static spatial dimension sequential feature extraction module. After the neighborhood hyperspectral spatial data set passes through a size transformation layer, it is input into m static units, and the static units include a bidirectional long short-term memory layer, a first average pooling layer, and a static attention layer. The static attention layer is used to calculate the Euclidean weight matrix and the spectral weight matrix of the output result of the first average pooling layer, and normalization operations are performed respectively. The Euclidean weight matrix and the spectral weight matrix are bitwise multiplied to obtain a static weight matrix. The static weight matrix is ​​used to calculate the output and generate a static sequential feature image after passing through a second average pooling layer.

[0010] The neighborhood hyperspectral spatial data set is input into a dynamic spatial dimension sequential feature extraction module. After passing through the size transformation layer and the splicing layer, the neighborhood hyperspectral spatial data set is input into m dynamic units. The dynamic units include the bidirectional long short-term memory layer, the first average pooling layer and the dynamic attention layer. The dynamic attention layer is used to calculate the weight matrix of the output result of the first average pooling layer, and a normalization operation is performed to obtain a dynamic weight matrix. The dynamic weight matrix is ​​used to calculate the output and generate a dynamic sequential feature image after passing through the second average pooling layer.

[0011] Extracting the features of the original domain hyperspectral image using a spectral feature extraction module to obtain an original domain spectral feature image;

[0012] The static sequential feature image, the dynamic sequential feature image and the original domain spectral feature image are fused to obtain a final spectral feature image;

[0013] The final spectral characteristic image is decoded by using a decoding module to generate a target spectral characteristic image.

[0014] Preferably, the pre-trained hyperspectral deep network model includes:

[0015] According to the formula Γ=Γ R +Γ KL Construct the loss function:

[0016]

[0017] Wherein, ∑(·) is the sum of all the contents in the brackets, Indata is the original domain hyperspectral image, Outdata is the target spectral feature image, μ is the fused feature image, and δ is the third original domain spectral feature image;

[0018] The step size is 10 -3 The Adam optimizer is used to train the hyperspectral deep network model until the loss function converges.

[0019] Preferably, the step of inputting the neighborhood hyperspectral spatial data set into a static spatial dimension sequential feature extraction module comprises:

[0020] Input the X×Y spatial data of the neighborhood hyperspectral spatial data set with a size of 1×s×s×B into the size transformation layer, and output the data with a size of 1×s 2 ×B;

[0021] Convert X×Y to 1×s 2 The spatial data of ×B is input into the bidirectional long short-term memory layer of the static unit, and the output size is 1×s 2 ×2d;

[0022] Convert X×Y to 1×s 2 ×2d spatial data is input into the first average pooling layer of the static unit, and the output size is 1×s 2 ×d;

[0023] Convert X×Y to 1×s 2 The spatial data of ×d is input into the static attention layer of the static unit, and the size is transformed to obtain a matrix of size 1×s×s×d

[0024] For the matrix The X×Y spatial data is based on the formula For each sample p whose spatial data position is [i, j] (1≤i≤s, 1≤j≤s) i,j With the center pixel The Euclidean distances between the two are stored bit by bit to obtain the Euclidean weight matrix M with a size of X×Y and a size of 1×s×s. E ;

[0025] For X×Y spatial data, according to the formula For each sample p whose spatial data position is [i, j] i,j With center sample The spectral distances between the two are stored bit by bit to obtain the spectral weight matrix M with a size of X×Y and a size of 1×s×s. S ;

[0026] For each of the Euclidean weight matrices M E And each of the spectral weight matrices M S Perform normalization operation

[0027] Each of the Euclidean weight matrices is bitwise multiplied with the corresponding spectral weight matrix to obtain X×Y static weight matrices M=M E ×M S ;

[0028] Expand the dimensions of the static weight matrices M of X×Y sizes of 1×s×s to 1×s×s×1;

[0029] Calculate output X×Y Its size is 1×s×s×d, and the converted size is 1×s 2 ×d Input the second average pooling layer to generate the static sequential feature image Output 1 .

[0030] Preferably, the step of inputting the neighborhood hyperspectral spatial data set into a dynamic spatial dimension sequential feature extraction module comprises:

[0031] The X×Y spatial data of the neighborhood hyperspectral spatial data set with a size of 1×s×s×B are input into the size transformation layer, and the output size is

[0032] The X×Y dimensions are The spatial data is input into the concatenation layer, and the output size is

[0033] The X×Y dimensions are The spatial data is input into the bidirectional long short-term memory layer of the dynamic unit, and the output size is 1×k×2d;

[0034] Input X×Y spatial data of size 1×k×2d into the first average pooling layer of the static unit, and output a matrix H of size 1×k×d;

[0035] The matrix H is input into the dynamic attention layer of the dynamic unit, and the X×Y spatial data of the matrix H are calculated according to the formula Calculate the sample p of each spatial data position in [i, j] (1≤i≤s, 1≤j≤s) i,j The sum of the spectral distances from all samples is stored bit by bit to obtain X×Y dynamic weight matrices N of size 1×s×s, and each of the dynamic weight matrices N is normalized;

[0036] Expand the dimensions of the dynamic weight matrix N of X×Y size 1×s×s to 1×s×s×1;

[0037] Calculate and output X×Y H+N×H, whose size is 1×k×d, and input the second average pooling layer to generate the static sequential feature image Output 2 .

[0038] Preferably, the step of fusing the static sequential feature image, the dynamic sequential feature image and the original domain spectral feature image to obtain a final spectral feature image comprises:

[0039] Performing fast Fourier transform, bitwise multiplication and splicing operations on the static sequential feature image, the dynamic sequential feature image and the second original domain spectral feature image to obtain the fused feature image;

[0040] The first original-domain spectral feature image is input into a fully connected layer to obtain a third original-domain spectral feature image, and a final spectral feature image is generated using the third original-domain spectral feature image, the fused feature image, and a randomly generated noise matrix.

[0041] Preferably, the third original domain spectral feature map δ, the fused feature map μ and the randomly generated noise matrix γ are used to generate a final spectral feature map Output = μ + (δ×γ).

[0042] Preferably, decoding the spectral characteristic image to generate a target spectral characteristic image comprises:

[0043] Input the spectral feature image with a size of 1×2d into the first fully connected layer, and the output is X×Y×d;

[0044] Input X×Y×d into the remaining 2nd to n-1th fully connected layers, and the output is X×Y×d;

[0045] Input X×Y×d into the nth fully connected layer and output the target spectral feature image Outdata with a size of 1×B.

[0046] The present invention also provides a device for extracting hyperspectral features, comprising:

[0047] A preprocessing module is used to select a hyperspectral image for normalization processing to obtain an original domain hyperspectral image X×Y×B, where X and Y are the spatial dimensions of the hyperspectral image at each wavelength, B is the number of bands of the hyperspectral image, a neighborhood sliding window s is set, and the embedding dimension is d. For each hyperspectral pixel in the original domain hyperspectral image, a neighborhood pixel with a size of s×s is selected as the spatial data of the pixel to obtain a neighborhood hyperspectral spatial data set;

[0048] An input module, used for inputting the original domain hyperspectral image and the neighborhood hyperspectral spatial dataset into a pre-trained hyperspectral deep network model;

[0049] A static sequential feature extraction module, used to input the neighborhood hyperspectral spatial data set into m static units after passing through a size transformation layer, wherein the static unit includes a bidirectional long short-term memory layer, a first average pooling layer, and a static attention layer, and use the static attention layer to calculate the Euclidean weight matrix and the spectral weight matrix of the output result of the first average pooling layer, and perform normalization operations on them respectively, and multiply the Euclidean weight matrix and the spectral weight matrix bit by bit to obtain a static weight matrix, and use the static weight matrix to calculate the output and generate a static sequential feature image after passing through a second average pooling layer;

[0050] A dynamic sequential feature extraction module, used for inputting the neighborhood hyperspectral spatial data set into m dynamic units after passing through the size transformation layer and the splicing layer, wherein the dynamic unit includes the bidirectional long short-term memory layer, the first average pooling layer and the dynamic attention layer, using the dynamic attention layer to calculate the weight matrix of the output result of the first average pooling layer, and performing a normalization operation to obtain a dynamic weight matrix, using the dynamic weight matrix to calculate the output and generating a dynamic sequential feature image after passing through the second average pooling layer;

[0051] A spectral feature extraction module, used to extract the features of the original domain hyperspectral image to obtain an original domain spectral feature image;

[0052] A feature fusion and splicing module, used for fusing the static sequential feature image, the dynamic sequential feature image and the original domain spectral feature image to obtain a final spectral feature image;

[0053] The decoding module is used to use the decoding module to decode the final spectral characteristic image to generate a target spectral characteristic image.

[0054] The present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned method for extracting hyperspectral features are implemented.

[0055] The above technical solution of the present invention has the following advantages compared with the prior art:

[0056] ① The present invention constructs different weight matrices for the two types of sequential information in the spatial level, respectively, and realizes the characteristics of high adaptability matching between the method and the sample characteristics. The construction of the weight matrix is ​​more in line with the data characteristics of the hyperspectral image, and the important part of the sequential information in the spatial level is strengthened. ② The present invention uses a bidirectional long short-term memory network and an attention mechanism to extract features from the two types of sequential information in the spatial level, reducing the influence of the sequential direction in the sequential information on the final performance. ③ The present invention uses the sequential information in the spatial level to correct the spectral information, realizes the optimization of feature extraction, solves the problem of the relatively single expression form of pixel information, and realizes a multi-modal feature extraction method. ④ The present invention corrects and fuses the two sequential features with the spectral features, and fully realizes the fusion of modal information (spatial information and spectral information). The present invention proposes a hyperspectral feature extraction method based on multimodal attention coding. By constructing a spatial dimensional sequential feature extraction module, the effective extraction of two kinds of sequential information in pixel space data is realized. The influence of sequential direction on performance in the process of sequential feature extraction is reduced by a bidirectional long short-term memory layer. Different attention weight matrices are constructed for different sequential information to enhance the important parts of the sequential information, realize the full fusion of information, and finally effectively improve the expressive power of the extracted features. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below according to specific embodiments of the present invention in conjunction with the accompanying drawings, wherein:

[0058] Figure 1 It is the implementation flow chart of hyperspectral feature extraction;

[0059] Figure 2 is a schematic diagram of the static attention layer;

[0060] Figure 3 is a schematic diagram of the dynamic attention layer;

[0061] Figure 4 It is the experimental flow chart of hyperspectral feature extraction;

[0062] Figure 5 A structural block diagram of a hyperspectral feature extraction device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0063] The core of the present invention is to provide a method, device, equipment and computer storage medium for extracting hyperspectral features to improve the accuracy of spectral feature extraction.

[0064] In order to enable those skilled in the art to better understand the scheme of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0065] Please refer to Figure 1 , Figure 1 This is a flowchart for realizing the hyperspectral feature extraction provided by the present invention; the specific operation steps are as follows:

[0066] S101: Select a hyperspectral image and perform normalization processing to obtain an original domain hyperspectral image X×Y×B, where X and Y are the spatial dimensions of the hyperspectral image at each wavelength, and B is the number of bands of the hyperspectral image. Set a neighborhood sliding window s, and select neighborhood pixels with a size of s×s around each hyperspectral pixel of the original domain hyperspectral image as the spatial data of the pixel to obtain a neighborhood hyperspectral spatial data set;

[0067] The normalization formula is as follows:

[0068]

[0069] Among them, x min Represents the minimum value in the data, x max is the maximum value.

[0070] Set the neighborhood sliding window s to an odd number greater than 0, the embedding dimension to d, d is greater than 0, and the number of nodes in each network layer is d;

[0071] For edge hyperspectral pixels, zero padding is performed on the missing parts.

[0072] Get all X×Y spatial data of size 1×s×s×B as input sample one, i.e., the neighborhood hyperspectral spatial dataset, and get all X×Y hyperspectral pixels of size 1×B as input sample two, i.e., the original domain hyperspectral image, where the number of input sample one and input sample two is the same and corresponds one to one, and the input sample two with the same serial number is the center of input sample one.

[0073] Shuffle all X×Y input samples 1 and all X×Y input samples 2 in the same order, and select the input samples 1 and 2 whose sequence numbers are in the first 30% as training samples 1 and 2. If X×Y×30% is not an integer, round it off.

[0074] Small batch training samples one and small batch training samples two with the same position are selected from all X×Y×30% training samples one and all X×Y×30% training samples two, and are input into the neural network. The number of small batch samples is 512. The 512 small batch training samples one are used as the input of the spatial dimension sequential feature extraction module one and the spatial dimension sequential feature extraction module two, and the 512 small batch training samples two are used as the input of the spectral feature extraction module to train the deep network.

[0075] According to the formula Γ=Γ R +Γ KL Construct the loss function:

[0076]

[0077] Wherein, ∑(·) is the sum of all the contents in the brackets, Indata is the original domain hyperspectral image, Outdata is the target spectral feature image, μ is the fused feature image, δ is the third original domain spectral feature image, Γ R Ensures the consistency between the input and output information of the deep network, Γ KL It ensures the consistency of Gaussian distribution and embedded feature distribution;

[0078] The step size is 10 -3 The Adam optimizer is used to train the hyperspectral deep network model until the loss function converges (at least 1000 times of training are required).

[0079] S102: Inputting the original domain hyperspectral image and the neighborhood hyperspectral spatial dataset into a pre-trained hyperspectral deep network model;

[0080] A hyperspectral deep network model is constructed, which includes a static spatial dimension sequential feature extraction module, a dynamic spatial dimension sequential feature extraction module, a spectral feature extraction module and a decoding module.

[0081] S103: Input the neighborhood hyperspectral spatial data set into a static spatial dimension sequential feature extraction module, input the neighborhood hyperspectral spatial data set into m static units after passing through a size transformation layer, and the static unit includes a bidirectional long short-term memory layer, a first average pooling layer, and a static attention layer, and use the static attention layer to calculate the Euclidean weight matrix and the spectral weight matrix of the output result of the first average pooling layer, and perform normalization operations on them respectively, and multiply the Euclidean weight matrix and the spectral weight matrix bit by bit to obtain a static weight matrix, and use the static weight matrix to calculate the output and generate a static sequential feature image after passing through a second average pooling layer;

[0082] The size transformation layer is used to rearrange a multidimensional matrix of a certain dimension to construct a new matrix with the same number of elements but different dimensions, only changing the shape of the matrix. The average pooling layer reduces one or more dimensions of the matrix by calculating the average value, compressing the dimensions of one or more dimensions to the original Where α is the pooling coefficient.

[0083] The bidirectional long short-term memory layer is an existing model in the field of deep learning. It includes two sequential long short-term memory layers to reduce the impact of the sequential direction on the feature mapping results. The activation function used by the bidirectional long short-term memory layer is the Tanh activation function:

[0084] The input size of the static spatial sequential feature extraction module is 1×s×s×B. The module consists of a reshape layer, m static units, and the second average pooling layer. The input size of the reshape layer is 1×s×s×B, and the output size is 1×s 2 ×B, the input size of the first static unit is 1×s 2 ×B, the output size is 1×s 2 ×d, the input size of the remaining m-1 static units is 1×s 2 ×d, the output size is 1×s 2 ×d, the input size of the average pooling layer is 1×s 2 ×d, the pooling coefficient for the second dimension is s 2 , the output is Output 1 , whose size is 1×d.

[0085] A static unit consists of a bidirectional long short-term memory layer, the first average pooling layer, and a static attention layer. The number of nodes in the bidirectional long short-term memory layer is d, the input size is the same as the static unit it is in, and the output size is 1×s 2 ×2d. The input size of the average pooling layer is 1×s 2 ×2d, the pooling factor for the third dimension is 2, and the output size is 1×s 2 ×d, the input size of the static attention layer is 1×s 2 ×d, the output size is 1×s 2 ×d.

[0086] S104: inputting the neighborhood hyperspectral spatial data set into a dynamic spatial dimension sequential feature extraction module, inputting the neighborhood hyperspectral spatial data set into m dynamic units after passing through the size transformation layer and the splicing layer, wherein the dynamic unit includes the bidirectional long short-term memory layer, the first average pooling layer and the dynamic attention layer, using the dynamic attention layer to calculate the weight matrix of the output result of the first average pooling layer, and performing a normalization operation to obtain a dynamic weight matrix, using the dynamic weight matrix to calculate the output and generating a dynamic sequential feature image after passing through the second average pooling layer;

[0087] The input size of the dynamic spatial dimension sequential feature extraction module is 1×s×s×B. The module consists of a size transformation layer, a concatenation layer, m dynamic units and the second average pooling layer. The input size of the size transformation layer is 1×s×s×B, and the output size is The input size of the concatenation layer is The output size is The input size of the first dynamic unit is The output size is 1×k×d, the input size of the remaining m-1 dynamic units is 1×k×d, the output size is 1×k×d, the input size of the average pooling layer is 1×k×d, the pooling coefficient for the second dimension is k, and the output is Output 2 , whose size is 1×d.

[0088] A dynamic unit consists of a bidirectional long short-term memory layer, the first average pooling layer, and a dynamic attention layer. The number of nodes in the bidirectional long short-term memory layer is d, the input size is consistent with the dynamic unit it is in, and the output size is 1×k×2d. The input size of the average pooling layer is 1×k×2d, the pooling coefficient for the third dimension is 2, and the output size is 1×k×d. The input size of the dynamic attention layer is 1×k×d, and the output size is 1×k×d.

[0089] S105: extracting the features of the original domain hyperspectral image using a spectral feature extraction module to obtain an original domain spectral feature image;

[0090] The spectral feature extraction module consists of m fully connected layers. The activation function used in the fully connected layers is the Tanh activation function. The input size of the first fully connected layer is 1×B and the output size is 1×d. The input size of the remaining m-1 fully connected layers is 1×d and the output size is 1×d.

[0091] The original domain hyperspectral image X×Y×B is input into the first fully connected layer, and the output is X×Y×d;

[0092] Input X×Y×d into the remaining m-1 fully connected layers. The output of the m-1th fully connected layer is the first original domain spectral feature map Toutput, and the output of the mth fully connected layer is the second original domain spectral feature map Output 3 .

[0093] S106: fusing the static sequential feature image, the dynamic sequential feature image and the original domain spectral feature image to obtain a final spectral feature image;

[0094] S107: Using a decoding module to decode the final spectral feature image to generate a target spectral feature image.

[0095] The two types of spatial information of the present invention are two types of continuous information extracted by using the LSTM network, also called sequential information, which are mainly two types of continuity in spatial information.

[0096] The present invention constructs different weight matrices for the two types of sequential information in the spatial level, respectively, to achieve the characteristics of high adaptability matching between the method and the sample characteristics. The construction of the weight matrix is ​​more in line with the data characteristics of the hyperspectral image, and the important part of the sequential information in the spatial level is strengthened. The present invention uses a bidirectional long short-term memory network and an attention mechanism to extract features from the two types of sequential information in the spatial level, reducing the influence of the sequential direction in the sequential information on the final performance. The present invention uses the sequential information in the spatial level to correct the spectral information, optimizes the feature extraction, solves the problem of the relatively single expression form of pixel information, and realizes a multimodal feature extraction method. The present invention performs information correction and fusion of the two sequential features with the spectral features, fully realizing the fusion of modal information (spatial information and spectral information). The present invention proposes a hyperspectral feature extraction method based on multimodal attention coding. By constructing a spatial dimensional sequential feature extraction module, the effective extraction of two kinds of sequential information in pixel space data is realized. The influence of sequential direction on performance in the process of sequential feature extraction is reduced by a bidirectional long short-term memory layer. Different attention weight matrices are constructed for different sequential information to enhance the important parts of the sequential information, realize the full fusion of information, and finally effectively improve the expressive power of the extracted features.

[0097] Based on the above embodiments, this embodiment further describes steps S103 and S104 in detail as follows:

[0098] refer to Figure 2 :

[0099] S131: Input the X×Y spatial data of the neighborhood hyperspectral spatial data set with a size of 1×s×s×B into the size transformation layer, and output the data with a size of 1×s 2 ×B;

[0100] S132: X×Y size is 1×s 2 The spatial data of ×B is input into the bidirectional long short-term memory layer of the static unit, and the output size is 1×s 2 ×2d;

[0101] S133: Convert X×Y dimensions to 1×s 2 ×2d spatial data is input into the first average pooling layer of the static unit, and the output size is 1×s 2 ×d;

[0102] S134: Convert X×Y dimensions to 1×s 2 The spatial data of ×d is input into the static attention layer of the static unit, and the size is transformed to obtain a matrix of size 1×s×s×d

[0103] S135: For the matrix The X×Y spatial data is based on the formula For each sample p whose spatial data position is [i, j] (1≤i≤s, 1≤j≤s) i,j With the center pixel The Euclidean distances between the two are stored bit by bit to obtain the Euclidean weight matrix M with a size of X×Y and a size of 1×s×s. E ;

[0104] S136: For X×Y spatial data, according to the formula For each sample p whose spatial data position is [i, j] i,j With center sample The spectral distances between the two are stored bit by bit to obtain the spectral weight matrix M with a size of X×Y and a size of 1×s×s. S ;

[0105] Where |·|2 is the sum of the squares of the elements of the vector and then taking the square root.

[0106] S137: For each of the Euclidean weight matrices M E And each of the spectral weight matrices M S Perform normalization operation Among them, x min Represents the minimum value in the matrix, x max Represents the maximum value in the matrix;

[0107] S138: Perform bitwise multiplication operation on each of the Euclidean weight matrices and the corresponding spectral weight matrix to obtain X×Y static weight matrices M=M E ×M S ;

[0108] S139: Expand the dimensions of the static weight matrices M of X×Y sizes of 1×s×s to 1×s×s×1;

[0109] S1310: Calculate and output X×Y Its size is 1×s×s×d, and the converted size is 1×s 2 ×d Input the second average pooling layer to generate the static sequential feature image Output 1 .

[0110] refer to Figure 3 :

[0111] S141: Input the X×Y spatial data of the neighborhood hyperspectral spatial data set with a size of 1×s×s×B into the size transformation layer, and output the data with a size of

[0112] S142: Set the X×Y dimensions to The spatial data is input into the concatenation layer, and the output size is

[0113] S143: Set the X×Y dimensions to The spatial data is input into the bidirectional long short-term memory layer of the dynamic unit, and the output size is 1×k×2d;

[0114] S144: Input X×Y spatial data of size 1×k×2d into the first average pooling layer of the static unit, and output a matrix H of size 1×k×d;

[0115] S145: Input the matrix H into the dynamic attention layer of the dynamic unit, and perform the X×Y spatial data of the matrix H according to the formula Calculate the sample p of each spatial data position in [i, j] (1≤i≤s, 1≤j≤s) i,j The sum of the spectral distances from all samples is stored bit by bit to obtain X×Y dynamic weight matrices N of size 1×s×s, and each of the dynamic weight matrices N is normalized;

[0116] S146: Expand the dimension of the dynamic weight matrix N of X×Y sizes of 1×s×s to 1×s×s×1;

[0117] S147: Calculate and output X×Y H+N×H, whose size is 1×k×d, and input the second average pooling layer to generate the static sequential feature image Output 2 .

[0118] The extracting the feature of the original domain hyperspectral image to obtain the original domain spectral feature image comprises:

[0119] The original domain hyperspectral image X×Y×B is input into the first fully connected layer, and the output is X×Y×d;

[0120] Input X×Y×d into the remaining m-1 fully connected layers. The output of the m-1th fully connected layer is the first original domain spectral feature map Toutput, and the output of the mth fully connected layer is the second original domain spectral feature map Output 3 .

[0121] The hyperspectral feature extraction method provided by the present invention considers both the spectral distance and the Euclidean distance between pixels when constructing the weight matrix for the first type of sequential information in the spatial level, and adopts a compromise between the two to achieve the construction of the weight matrix; when constructing the weight matrix for the second type of sequential information in the spatial level, the average spectral distance between each small sample and the entire sequential sample is focused on, highlighting its local correlation in the data, and effectively improving the weight of important information.

[0122] Based on the above embodiments, this embodiment describes step S106 in detail as follows:

[0123] S161: The static sequential feature image Ouput 1 , the dynamic sequential feature image Output 2 And the second original domain spectral feature image Output 3 Perform fast Fourier transform and bitwise multiplication and splicing operations to obtain the fused feature image μ;

[0124] Output 1 、Output 2 and Output 3 Perform fast Fourier transform and get and right and Perform bitwise multiplication and get right and Perform bitwise multiplication and get right and Perform inverse fast Fourier transform to get Output 13 and Output 23 , both of which have a size of 1×d. They are concatenated using a concatenation layer to obtain an output μ of size 1×2d.

[0125] S162: Input the first original domain spectral feature image into a fully connected layer to obtain a third original domain spectral feature image, and use the third original domain spectral feature image δ, the fused feature image μ and the randomly generated noise matrix γ to generate a final spectral feature image Output = μ + (δ × γ);

[0126] γ is a randomly generated noise matrix that conforms to a Gaussian distribution and has a size of 1×2d.

[0127] Fast Fourier transform and inverse fast Fourier transform are commonly used definitions in the field of signal processing, referring to efficient and fast calculation methods of discrete Fourier transform and inverse discrete Fourier transform using computers; bit-wise multiplication refers to multiplying the elements in the same position of two matrices or vectors of the same shape to finally obtain matrices or vectors of the same shape.

[0128] The present invention fuses the two sequential features with the spectral features through information correction fusion, utilizes the characteristics of fast Fourier transform to achieve high efficiency, realizes high-performance fusion of information, and ultimately effectively improves the expression ability of the extracted features.

[0129] Based on the above embodiments, this embodiment extracts the spatial-spectral features in the hyperspectral image and uses them for subsequent classification research. The Indiana Forest Dataset (Indian Pines Dataset) is used as an example. The details are as follows:

[0130] refer to Figure 4 :

[0131] Input: The input hyperspectral image is an image of size 145×145×200, containing a total of 21,025 pixels, each pixel contains 200 bands, and the entire data set contains 16 valid categories and background noise categories. After removing the pixels belonging to the background noise category, a total of 10,366 valid pixels remain.

[0132] Parameter settings: the neighborhood sliding window size is 7, the number of units in the static spatial dimension sequential feature extraction module and the dynamic spatial dimension sequential feature extraction module are both 3, the number of network layers in the spectral feature extraction module is 3, the number of network layers in the decoding module is 3, and the embedding dimension is 40.

[0133] Neighborhood information selection: For each pixel, obtain neighborhood information of size 7×7×200.

[0134] Train this hyperspectral deep network model:

[0135] Using 10366 valid pixels, 10366 input samples 1 and 10366 input samples 2 can be obtained. These samples are randomly sorted and 30% of the samples are selected for training the deep network model. The number of small batch samples is 512, and only one small batch sample is used for each training. After the training is completed, all 10366 input samples 1 and all 10366 input samples 2 are input into the deep model for testing to obtain embedded features of size 10366×40, and finally classified using the SVM classifier. 10% of the samples are randomly selected to train the SVM classifier, and the remaining 90% of the samples are used for testing to finally obtain the classification results. The overall classification accuracy and average classification accuracy are used to evaluate the classification results. The overall classification result refers to the ratio of the number of correctly classified samples in all samples divided by the total number of samples. The average classification accuracy first calculates the ratio of the number of correctly classified samples in each class divided by the number of samples in that class, and calculates the average of each ratio.

[0136] The classification results obtained using the hyperspectral feature extraction method based on multimodal attention encoding and the hyperspectral feature extraction method based on ordinary variational auto-encoding proposed in this application (including an encoder and a decoder, wherein the encoder consists of 3 fully connected layers, the decoder consists of 3 fully connected layers, and the number of network layer nodes is the same as the method implemented in this application) are shown in the following table.

[0137]

[0138] As can be seen from the table, the method of the present application can better improve the classification performance of the embedded features, with fewer misclassified samples. It can be seen that the method of the present application has a strong feature extraction capability. Therefore, the method of the present application can effectively improve the classifiability and classification accuracy of the embedded features.

[0139] Please refer to Figure 5 , Figure 5 A structural block diagram of a hyperspectral feature extraction device provided by an embodiment of the present invention; the specific device may include:

[0140] The preprocessing module 100 is used to select a hyperspectral image for normalization processing to obtain an original domain hyperspectral image X×Y×B, where X and Y are the spatial dimensions of the hyperspectral image at each wavelength, and B is the number of bands of the hyperspectral image. A neighborhood sliding window s is set, and the embedding dimension is d. For each hyperspectral pixel of the original domain hyperspectral image, a neighborhood pixel with a size of s×s is selected as the spatial data of the pixel to obtain a neighborhood hyperspectral spatial data set.

[0141] An input module 200 is used to input the original domain hyperspectral image and the neighborhood hyperspectral spatial dataset into a pre-trained hyperspectral deep network model;

[0142] A static sequential feature extraction module 300 is used to input the neighborhood hyperspectral spatial data set into m static units after passing through a size transformation layer, wherein the static unit includes a bidirectional long short-term memory layer, a first average pooling layer, and a static attention layer, and the static attention layer is used to calculate the Euclidean weight matrix and the spectral weight matrix of the output result of the first average pooling layer, and normalize them respectively, and multiply the Euclidean weight matrix and the spectral weight matrix bit by bit to obtain a static weight matrix, and the static weight matrix is ​​used to calculate the output and generate a static sequential feature image after passing through a second average pooling layer;

[0143] A dynamic sequential feature extraction module 400 is used to input the neighborhood hyperspectral spatial data set into m dynamic units after passing through the size transformation layer and the splicing layer, wherein the dynamic unit includes the bidirectional long short-term memory layer, the first average pooling layer and the dynamic attention layer, and use the dynamic attention layer to calculate the weight matrix of the output result of the first average pooling layer, and perform a normalization operation to obtain a dynamic weight matrix, and use the dynamic weight matrix to calculate the output and generate a dynamic sequential feature image after passing through the second average pooling layer;

[0144] A spectral feature extraction module 500 is used to extract the features of the original domain hyperspectral image to obtain an original domain spectral feature image;

[0145] A feature fusion and splicing module 600 is used to fuse the static sequential feature image, the dynamic sequential feature image and the original domain spectral feature image to obtain a final spectral feature image;

[0146] The decoding module 700 is used to use a decoding module to decode the final spectral characteristic image to generate a target spectral characteristic image.

[0147] The hyperspectral feature extraction device of the present embodiment is used to implement the aforementioned hyperspectral feature extraction method. Therefore, the specific implementation methods of the hyperspectral feature extraction device can be seen in the embodiment part of the aforementioned hyperspectral feature extraction method. For example, the preprocessing module 100, the input module 200, the static sequential feature extraction module 300, the dynamic sequential feature extraction module 400, the spectral feature extraction module 500, the feature fusion and splicing module 600, and the decoding module 700 are respectively used to implement steps S101, S102, S103, S104, S105, S106 and S107 in the aforementioned hyperspectral feature extraction method. Therefore, its specific implementation methods can refer to the descriptions of the corresponding embodiments of each part and will not be repeated here.

[0148] A specific embodiment of the present invention further provides a device for extracting hyperspectral features, comprising: a memory for storing a computer program; and a processor for implementing the steps of the above-mentioned method for extracting hyperspectral features when executing the computer program.

[0149] A specific embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned method for extracting hyperspectral features are implemented.

[0150] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0151] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0152] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0153] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0154] Obviously, the above embodiments are merely examples for the purpose of clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the present invention.

Claims

1. A hyperspectral feature extraction method, characterized in that: include: A hyperspectral image is selected and normalized to obtain an original hyperspectral image X×Y×B, where X and Y are the spatial dimensions of the hyperspectral image at each wavelength, and B is the number of bands of the hyperspectral image. A neighborhood sliding window s is set, and the embedding dimension is d. For each hyperspectral pixel in the original hyperspectral image, a neighborhood pixel with a size of s×s is selected as the spatial data of the pixel to obtain a neighborhood hyperspectral spatial data set. Inputting the original domain hyperspectral image and the neighborhood hyperspectral spatial dataset into a pre-trained hyperspectral deep network model; The pre-trained hyperspectral deep network model includes: According to the formula Γ=Γ R +Γ KL Construct the loss function: Wherein, ∑(·) is the sum of all the contents in the brackets, Indata is the original domain hyperspectral image, Outdata is the target spectral feature image, μ is the fused feature image, and δ is the third original domain spectral feature image; The step size is 10. -3 The Adam optimizer trains the hyperspectral deep network model until the loss function converges; The neighborhood hyperspectral spatial data set is input into a static spatial dimension sequential feature extraction module. After the neighborhood hyperspectral spatial data set passes through a size transformation layer, it is input into m static units, and the static units include a bidirectional long short-term memory layer, a first average pooling layer, and a static attention layer. The static attention layer is used to calculate the Euclidean weight matrix and the spectral weight matrix of the output result of the first average pooling layer, and normalization operations are performed respectively. The Euclidean weight matrix and the spectral weight matrix are bitwise multiplied to obtain a static weight matrix. The static weight matrix is ​​used to calculate the output and generate a static sequential feature image after passing through a second average pooling layer. The neighborhood hyperspectral spatial data set is input into a dynamic spatial dimension sequential feature extraction module. After passing through the size transformation layer and the splicing layer, the neighborhood hyperspectral spatial data set is input into m dynamic units. The dynamic units include the bidirectional long short-term memory layer, the first average pooling layer and the dynamic attention layer. The dynamic attention layer is used to calculate the weight matrix of the output result of the first average pooling layer, and a normalization operation is performed to obtain a dynamic weight matrix. The dynamic weight matrix is ​​used to calculate the output and generate a dynamic sequential feature image after passing through the second average pooling layer. Extracting the features of the original domain hyperspectral image using a spectral feature extraction module to obtain an original domain spectral feature image; The static sequential feature image, the dynamic sequential feature image and the original domain spectral feature image are fused to obtain a final spectral feature image; The final spectral characteristic image is decoded by using a decoding module to generate a target spectral characteristic image.

2. The hyperspectral feature extraction method according to claim 1, characterized in that: The step of inputting the neighborhood hyperspectral spatial data set into the static spatial dimension sequential feature extraction module comprises: Input the X×Y spatial data of the neighborhood hyperspectral spatial data set with a size of 1×s×s×B into the size transformation layer, and output the data with a size of 1×s 2 ×B; Convert X×Y to 1×s 2 The spatial data of ×B is input into the bidirectional long short-term memory layer of the static unit, and the output size is 1×s 2 ×2d; Convert X×Y to 1×s 2 ×2d spatial data is input into the first average pooling layer of the static unit, and the output size is 1×s 2 ×d; Convert X×Y to 1×s 2 The spatial data of ×d is input into the static attention layer of the static unit, and the size is transformed to obtain a matrix of size 1×s×s×d For the matrix The X×Y spatial data is based on the formula For each sample p whose spatial data position is [i, j] i,j With the center pixel The Euclidean distances between the two are stored bit by bit to obtain the Euclidean weight matrix M with a size of X×Y and a size of 1×s×s. E ; Among them, 1≤i≤s, 1≤j≤s; For X×Y spatial data, according to the formula For each sample p whose spatial data position is [i, j] i，j With center sample The spectral distances between the two are stored bit by bit to obtain the spectral weight matrix M with a size of X×Y and a size of 1×s×s. S ; For each of the Euclidean weight matrices M E and each of the spectral weight matrices M S Perform normalization operation Each of the Euclidean weight matrices is bitwise multiplied with the corresponding spectral weight matrix to obtain X×Y static weight matrices M=M E ×M S ; Expand the dimensions of the static weight matrices M of X×Y sizes of 1×s×s to 1×s×s×1; Calculate output X×Y Its size is 1×s×s×d, and the converted size is 1×s 2 ×d Input the second average pooling layer to generate the static sequential feature image Output 1 .

3. The hyperspectral feature extraction method according to claim 1, characterized in that: The step of inputting the neighborhood hyperspectral spatial data set into the dynamic spatial dimension sequential feature extraction module comprises: The X×Y spatial data of the neighborhood hyperspectral spatial data set with a size of 1×s×s×B are input into the size transformation layer, and the output size is The X×Y dimensions are The spatial data is input into the concatenation layer, and the output size is The X×Y dimensions are The spatial data is input into the bidirectional long short-term memory layer of the dynamic unit, and the output size is 1×k×2d; Input X×Y spatial data of size 1×k×2d into the first average pooling layer of the static unit, and output a matrix H of size 1×k×d; The matrix H is input into the dynamic attention layer of the dynamic unit, and the X×Y spatial data of the matrix H are calculated according to the formula Calculate the sample p of each spatial data position [i, j] i，j The sum of the spectral distances from all samples is stored bit by bit to obtain X×Y dynamic weight matrices N of size 1×s×s, and each of the dynamic weight matrices N is normalized; wherein 1≤i≤s, 1≤j≤s; Expand the dimensions of the dynamic weight matrix N of X×Y sizes of 1×s×s to 1×s×s×1; Calculate and output X×Y H+N×H, whose size is 1×k×d, and input the second average pooling layer to generate the static sequential feature image Output 2 .

4. The hyperspectral feature extraction method according to claim 1, characterized in that: The extracting the feature of the original domain hyperspectral image to obtain the original domain spectral feature image comprises: Input the original domain hyperspectral image X×Y×B into the first fully connected layer of the spectral feature extraction module, and output X×Y×d; Input X×Y×d into the remaining m-1 fully connected layers. The output of the m-1th fully connected layer is the first original domain spectral feature map Toutput, and the output of the mth fully connected layer is the second original domain spectral feature map Output 3 .

5. The hyperspectral feature extraction method according to claim 4, characterized in that: The step of fusing the static sequential feature image, the dynamic sequential feature image and the original domain spectral feature image to obtain a final spectral feature image comprises: Performing fast Fourier transform, bitwise multiplication and splicing operations on the static sequential feature image, the dynamic sequential feature image and the second original domain spectral feature image to obtain the fused feature image; The first original-domain spectral feature image is input into a fully connected layer to obtain a third original-domain spectral feature image, and a final spectral feature image is generated using the third original-domain spectral feature image, the fused feature image, and a randomly generated noise matrix.

6. The hyperspectral feature extraction method according to claim 5, characterized in that: The third original domain spectral feature map δ, the fused feature map μ and the randomly generated noise matrix γ are used to generate a final spectral feature map Output = μ + (δ×γ).

7. The hyperspectral feature extraction method according to claim 5, characterized in that: The decoding of the spectral characteristic image to generate a target spectral characteristic image comprises: Input the spectral feature image with a size of 1×2d into the first fully connected layer of the decoding module, and output it as X×Y×d; Input X×Y×d into the remaining 2nd to n-1th fully connected layers, and the output is X×Y×d; Input X×Y×d into the nth fully connected layer and output the target spectral feature image Outdata with a size of 1×B.

8. A device for extracting hyperspectral features, characterized in that: include: A preprocessing module is used to select a hyperspectral image for normalization processing to obtain an original domain hyperspectral image X×Y×B, where X and Y are the spatial dimensions of the hyperspectral image at each wavelength, B is the number of bands of the hyperspectral image, a neighborhood sliding window s is set, and the embedding dimension is d. For each hyperspectral pixel in the original domain hyperspectral image, a neighborhood pixel with a size of s×s is selected as the spatial data of the pixel to obtain a neighborhood hyperspectral spatial data set; An input module, used for inputting the original domain hyperspectral image and the neighborhood hyperspectral spatial dataset into a pre-trained hyperspectral deep network model; The pre-trained hyperspectral deep network model includes: according to the formula Γ=Γ R +Γ KL Construct the loss function: Wherein, ∑(·) is the sum of all the contents in the brackets, Indata is the original domain hyperspectral image, Outdata is the target spectral feature image, μ is the fused feature image, and δ is the third original domain spectral feature image; The step size is 10. -3 The Adam optimizer trains the hyperspectral deep network model until the loss function converges; A static sequential feature extraction module, used to input the neighborhood hyperspectral spatial data set into m static units after passing through a size transformation layer, wherein the static unit includes a bidirectional long short-term memory layer, a first average pooling layer, and a static attention layer, and use the static attention layer to calculate the Euclidean weight matrix and the spectral weight matrix of the output result of the first average pooling layer, and perform normalization operations on them respectively, and multiply the Euclidean weight matrix and the spectral weight matrix bit by bit to obtain a static weight matrix, and use the static weight matrix to calculate the output and generate a static sequential feature image after passing through a second average pooling layer; A dynamic sequential feature extraction module, used for inputting the neighborhood hyperspectral spatial data set into m dynamic units after passing through the size transformation layer and the splicing layer, wherein the dynamic unit includes the bidirectional long short-term memory layer, the first average pooling layer and the dynamic attention layer, using the dynamic attention layer to calculate the weight matrix of the output result of the first average pooling layer, and performing a normalization operation to obtain a dynamic weight matrix, using the dynamic weight matrix to calculate the output and generating a dynamic sequential feature image after passing through the second average pooling layer; A spectral feature extraction module, used to extract the features of the original domain hyperspectral image to obtain an original domain spectral feature image; A feature fusion and splicing module, used for fusing the static sequential feature image, the dynamic sequential feature image and the original domain spectral feature image to obtain a final spectral feature image; The decoding module is used to use the decoding module to decode the final spectral characteristic image to generate a target spectral characteristic image.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for extracting hyperspectral features as claimed in any one of claims 1 to 7 are implemented.

Images