A hyperspectral image classification method

Abstract

The application belongs to the field of image classification, and particularly relates to a hyperspectral image classification method. The classification method innovatively proposes to use a Deep Maching optical flow method to extract motion features in a spectral dimension as classification features of the hyperspectral image, can extract relative motion between adjacent frames in the spectral dimension, uses the fact that a spectral vector of any pixel in the hyperspectral image has inherent motion properties and the motion consistency between spectral vectors of adjacent pixels, obtains spectral motion features with spatial and spectral properties, as a beneficial supplement to discriminative features for distinguishing ground object categories, so as to improve classification accuracy. And by comparing the classification effects of various existing classification methods, it is proved that the spectral motion features extracted by the application have good discriminability, are better than traditional classic feature extraction methods and deep learning-based feature extraction methods, and can achieve good classification performance under small sample conditions.

Description

Technical Field

[0001] This invention belongs to the field of image classification, specifically relating to a hyperspectral image classification method. Background Technology

[0002] Hyperspectral imagery provides nearly continuous spectral information of ground features, enabling more precise differentiation of feature categories within the same scene. However, with insufficient training samples, the high-dimensional nonlinearity and high correlation of hyperspectral imagery can lead to a severe "curse of dimensionality." Simultaneously, the spectral mixing problem in hyperspectral imagery poses a significant challenge to classification tasks. To mitigate these issues, early research on hyperspectral imagery classification focused on three main aspects: feature engineering, classification strategies, and post-processing of classification results. Feature engineering includes band selection and feature extraction. Band selection is a hyperspectral imagery dimensionality reduction method. Its principle is to select several bands of interest or a subset of features with high information content, low correlation, and good classification and recognition capabilities from the initial bands of the hyperspectral imagery based on certain criteria to achieve data dimensionality reduction. Band selection can select a small number of representative bands from the original data, preserving the original spectral structure while reducing data dimensionality, thus alleviating the "curse of dimensionality" problem to some extent.

[0003] Unlike band selection, feature extraction focuses on transforming the original hyperspectral image to better distinguish different ground features. Research on hyperspectral image feature extraction methods first focuses on the spectral dimension. Spectral feature extraction projects the original spectral vectors into a low-dimensional feature space according to prior criteria, optimizing the projected feature vectors under the selected criteria. While feature extraction from spectral vectors can improve classification accuracy to some extent, the phenomenon of "different spectra for the same object" exists in hyperspectral images, thus classifying solely based on spectral features has limitations. In a hyperspectral image, adjacent pixels have a high probability of belonging to the same class; therefore, introducing spatial information can help create a more accurate classification map.

[0004] To automatically learn and extract suitable features from hyperspectral data for classification tasks, existing technologies have introduced deep learning methods into hyperspectral image classification. These include 2D-CNN and 3D-CNN methods that utilize spatial spectral information to improve classification performance, and 3D-CAE methods that learn data features unsupervised from large amounts of unlabeled data. Attention mechanisms, adversarial learning, active learning, and architecture search strategies have also been applied to hyperspectral image classification. Meanwhile, researchers have explored the application of deep learning models such as autoencoders, deep belief networks, deep recurrent networks, and capsule networks to hyperspectral image classification. Although deep learning methods have made significant progress in hyperspectral image classification, they still suffer from high training complexity and difficulty adapting well to small sample sizes, resulting in insufficient discriminative power of the extracted hyperspectral image features and low classification accuracy. Summary of the Invention

[0005] The purpose of this invention is to provide a hyperspectral image classification method to solve the problems of insufficient distinguishability of hyperspectral image features and low classification accuracy in existing hyperspectral image classification methods.

[0006] To achieve the above objectives, the present invention provides a hyperspectral image classification method, comprising the following steps:

[0007] 1) Unfold the hyperspectral image to be classified along the spectral dimension to obtain several frames of single-spectral value images;

[0008] 2) Obtain spectral flow motion information for each frame of single-spectral value image using the Deep Maching method;

[0009] 3) The spectral motion information of all single-spectral images is fused to obtain the spectral motion features of the hyperspectral image to be classified. Then, the spectral motion features are fused with the original spectral information of the hyperspectral image to be classified to obtain the spectral fusion features used for classification.

[0010] This innovative classification method proposes to use the Deep Maching optical flow method to extract motion features in the spectral dimension as classification features of hyperspectral images. It can extract the relative motion between adjacent frames in the spectral dimension. By utilizing the inherent motion attributes of the spectral vector of any pixel in the hyperspectral image and the motion consistency implied by the spectral vectors between adjacent pixels, it obtains spectral motion features that simultaneously possess spatial and spectral attributes. This serves as a useful supplement to the discriminative features for distinguishing land cover categories, thereby improving classification accuracy.

[0011] Furthermore, the process of obtaining the spectral flow motion information of each frame of single-spectral value image using the Deep Maching method is as follows:

[0012] ① Divide one frame in each adjacent frame monospectral image pair into sub-image blocks according to the minimum image size without overlap;

[0013] ② Obtain the response of another frame image in an adjacent frame monospectral image pair with respect to each sub-image block of that frame image; then stitch any four adjacent sub-image blocks together to obtain different parent image blocks, and obtain the response of the corresponding parent image block through the response of the sub-image; the response of the sub-image block refers to the similarity between each sub-image block in the other frame image and the sub-image block of the current frame image respectively, and the higher the similarity, the larger the response value; the sub-image block in the other frame image refers to any sub-image block in the other frame image with the same size as the sub-image block of the current frame image;

[0014] ③ Take the obtained parent image block as a higher-level sub-image block, repeat the process in ② and iterate until the obtained parent image block is the original single spectral value image before division. Find the local response maximum value of sub-image blocks at different levels, and then gradually obtain the fine pixel-level correspondence from the top to the bottom as the matching result.

[0015] ④ The matching results are incorporated into the optimization framework of minimizing optical flow energy to obtain the spectral flow motion information of each frame of single-spectral value image.

[0016] Furthermore, the response of the parent image block of a single-spectral image in an adjacent single-spectral image pair is obtained by the maximum similarity weighted average of the responses of the corresponding child image blocks, specifically as follows:

[0017]

[0018] Among them I 1 ,I 2 For adjacent frame single-spectral value image pairs, C N,p (p')I 1 The response is to the parent image patch; For I 1 The response of each child image patch to the parent image patch; p' is I 2 The current layer and the image I in this frame 1 The size of the parent image block is the same as the center of the parent image block, I 2 p is another frame in an adjacent single-spectral value image pair; p is I 1 The center of the sub-image patch; θ is I 1 The set of centers m' of each sub-image patch, N = M × 2 t t is the number of iterations, M is the minimum image size; and p+S N，i This represents the center location of the low-level response map, while S N，i This represents the drift of the center position of the lower-level response relative to the higher-level response.

[0019] Furthermore, the optimization objective of the optimization framework for minimizing optical flow energy is:

[0020]

[0021] in, Ω represents spectral flow motion information; I represents the image pair of adjacent frames with single spectral values. 1 ,I 2 The space where it is located, E D For data items, E S For the smoothing term, E M For matching items;

[0022] The data items are:

[0023]

[0024] The smoothing term is:

[0025]

[0026] The matching item is:

[0027] E M =cφψ(||w-w'|| 2 )

[0028] in, Let be the matching vector field; b is the binomial term b(x); φ(x) is the penalty term; δ and γ are weight coefficients; Where M0 is a tensor; Here, c is the normalized tensor; c is the total number of frames in the single-spectral image; and the penalty term... Where s 2 The parameters are substituted; ε and i are both constant terms.

[0029] During minimization, the optimization framework in the Brox optical flow method is used to minimize the optimization objective.

[0030] Furthermore, to achieve better optimization results, ε = 0.001.

[0031] Furthermore, in order to maintain good transitivity between responses at different levels, after obtaining the response of the parent image patch in each iteration, a nonlinear exponential transformation correction is performed on the response.

[0032] Furthermore, the minimum image size is related to the size of the hyperspectral image to be classified and the required classification accuracy.

[0033] If the minimum image size is too large, it may lead to excessive interference from weakly correlated matching information, resulting in low matching efficiency. If the minimum image size is too small, it may lose crucial correlated matching information, reducing matching accuracy. The fineness of the matching affects the actual matching effect, which in turn affects the contribution of the matching term in the subsequent energy minimization constraint. Therefore, the setting of the minimum image size needs to balance matching efficiency and matching accuracy.

[0034] Furthermore, in order to make adjacent frames better reflect more distinctive and detailed spectral motion features, the number of single-spectral value images of n bands is obtained by continuously extracting frames from the hyperspectral image, and then the spectral dimensions are expanded.

[0035] Furthermore, after obtaining the spectral fusion features for classification, classification is performed using a support vector machine. Attached Figure Description

[0036] Figure 1 This is a schematic diagram of the hyperspectral image classification method process in an embodiment of the hyperspectral image classification method of the present invention;

[0037] Figure 2 This is a schematic diagram illustrating the sub-image block division of adjacent frames of single-spectral value images by the Deep Matching algorithm in an embodiment of the hyperspectral image classification method of the present invention;

[0038] Figure 3a This is a three-dimensional bar chart showing the classification performance of IP scenarios under different α and M settings in the comparative examples of this invention; Figure 3b This is a three-dimensional bar chart showing the classification performance of the SA scene in the comparative examples of the present invention under different α and M settings; Figure 3c This is a three-dimensional bar chart showing the classification performance of the PU scene under different α and M settings in the comparative examples of the present invention; Figure 3d This is a three-dimensional bar chart showing the classification performance of the HO scene under different α and M settings in the comparative examples of the present invention;

[0039] Figure 4a Box plots showing the classification performance of IP scenarios with different frame skipping intervals D in the comparative examples of this invention; Figure 4b Box plots showing the classification performance of the SA scene in the comparative examples of this invention under different frame skipping intervals D; Figure 4c Box plots showing the classification performance of the PU scene in the comparative examples of this invention under different frame skipping intervals D; Figure 4d Box plots showing the classification performance of the HO scene in the comparative examples of this invention under different frame skipping intervals D;

[0040] Figure 5a Real hyperspectral images of IP scenarios; Figure 5b For the true marking of IP scenarios; Figure 5cThis is a global classification thematic map of the Gabor algorithm in the IP scenario in the comparative example of this invention; Figure 5d This is a global classification thematic map of the EMPs algorithm in the IP scenario in the comparative example of this invention; Figure 5e This is a global classification thematic map of the LBP algorithm in the IP scenario in the comparative example of the present invention; Figure 5f This is a global classification thematic map of the DFSL algorithm in the IP scenario in the comparative example of the present invention; Figure 5g This is a global classification thematic map of the 3D-CAE algorithm in the IP scenario in the comparative example of the present invention; Figure 5h This is a global classification thematic map of the SSTN algorithm in the IP scenario in the comparative example of this invention; Figure 5i This is a global classification thematic map of the FullyContNet algorithm in the IP scenario in the comparative example of this invention; Figure 5j This is a global classification thematic map of the SpeMotion algorithm in the IP scenario in the comparative example of the present invention;

[0041] Figure 6a This is a true hyperspectral image of the SA scene; Figure 6b For realistic marking of SA scenes; Figure 6c This is a global classification thematic map of the Gabor algorithm in the SA scene in the comparative example of this invention; Figure 6d This is a global classification thematic map of the EMPs algorithm in the SA scene in the comparative example of the present invention; Figure 6e This is a global classification thematic map of the LBP algorithm in the SA scene in the comparative example of the present invention; Figure 6f This is a global classification thematic map of the DFSL algorithm in the SA scene in the comparative example of the present invention; Figure 6g This is a global classification thematic map of the 3D-CAE algorithm in the SA scene in the comparative example of the present invention; Figure 6h This is a global classification thematic map of the SSTN algorithm in the SA scene in the comparative example of the present invention; Figure 6i This is a global classification thematic map of the FullyContNet algorithm in the SA scene in the comparative example of this invention; Figure 6j This is a global classification thematic map of the SpeMotion algorithm in the SA scene in the comparative example of the present invention;

[0042] Figure 7a A true hyperspectral image of the PU scene; Figure 7b For realistic marking of PU scenarios; Figure 7c This is a global classification thematic map of the Gabor algorithm in the PU scene in the comparative example of this invention; Figure 7d This is a global classification thematic map of the EMPs algorithm in the PU scene in the comparative example of the present invention; Figure 7e This is a global classification thematic map of the LBP algorithm in the PU scene in the comparative example of the present invention; Figure 7fThis is a global classification thematic map of the DFSL algorithm in the PU scene in the comparative example of the present invention; Figure 7g This is a global classification thematic map of the 3D-CAE algorithm in the PU scene in the comparative example of the present invention; Figure 7h This is a global classification thematic map of the SSTN algorithm in the PU scene in the comparative example of the present invention; Figure 7i This is a global classification thematic map of the FullyContNet algorithm in the PU scene in the comparative example of this invention; Figure 7j This is a global classification thematic map of the SpeMotion algorithm in the PU scene in the comparative example of the present invention;

[0043] Figure 8a This is a true hyperspectral image of the HO scene; Figure 8b For realistic labeling of HO scenes; Figure 8c This is a global classification thematic map of the Gabor algorithm in the HO scene in the comparative example of the present invention; Figure 8d This is a global classification thematic map of the EMPs algorithm in the HO scene in the comparative example of the present invention; Figure 8e This is a global classification thematic map of the LBP algorithm in the HO scene in the comparative example of the present invention; Figure 8f This is a global classification thematic map of the DFSL algorithm in the HO scene in the comparative example of the present invention; Figure 8g This is a global classification thematic map of the 3D-CAE algorithm on the HO scene in the comparative example of the present invention; Figure 8h This is a global classification thematic map of the SSTN algorithm in the HO scene in the comparative example of the present invention; Figure 8i This is a global classification thematic map of the FullyContNet algorithm in the HO scene in the comparative example of the present invention; Figure 8j This is a global classification thematic map of the SpeMotion algorithm in the HO scene in the comparative example of the present invention;

[0044] Figure 9a This is a comparative example of the present invention showing the changes in classification performance of different methods in the IP scenario as the number of training samples changes; Figure 9b This is a comparative example of the present invention showing the changes in classification performance of different methods in the SA scenario as the number of training samples changes; Figure 9c This is a comparative example of the present invention showing the changes in classification performance of different methods in the PU scenario as the number of training samples changes; Figure 9d This is a comparative example of the present invention showing the changes in classification performance of different methods in the HO scenario as the number of training samples changes;

[0045] Figure 10a This is a real hyperspectral image of the IP scene in the comparative example of the present invention; Figure 10b This is a global classification thematic map of the IP scene using the SSTN algorithm in the comparative example of this invention with 200 training samples; Figure 10cThis is a global classification thematic map of the IP scene using the FullyContNet algorithm in the comparative example of this invention with 200 training samples; Figure 10d This is a global classification thematic map of the IP scene using the SpeMotion algorithm in the comparative example of this invention, with 200 training samples.

[0046] Figure 11a This is a real hyperspectral image of the SA scene in the comparative example of the present invention; Figure 11b This is a global classification thematic map of the SA scene using the SSTN algorithm in the comparative example of this invention with 200 training samples; Figure 11c This is a global classification thematic map of the SA scene using the FullyContNet algorithm in the comparative example of this invention with 200 training samples; Figure 11d This is a global classification thematic map of the SA scene using the SpeMotion algorithm in the comparative example of this invention with 200 training samples;

[0047] Figure 12a This is a real hyperspectral image of the PU scene in the comparative example of the present invention; Figure 12b This is a global classification thematic map of the SSTN algorithm on the PU scene under the condition of 200 training samples in the comparative example of the present invention; Figure 12c This is a global classification thematic map of the PU scene using the FullyContNet algorithm in the comparative example of this invention with 200 training samples; Figure 12d This is a global classification thematic map of the PU scene using the SpeMotion algorithm in the comparative example of this invention, with 200 training samples.

[0048] Figure 13a This is a real hyperspectral image of the HO scene in the comparative example of the present invention; Figure 13b This is a global classification thematic map of the HO scene using the SSTN algorithm in the comparative example of this invention with 200 training samples; Figure 13c This is a global classification thematic map of the HO scene using the FullyContNet algorithm in the comparative example of this invention with 200 training samples; Figure 13d This is a global classification thematic map of the HO scene using the SpeMotion algorithm in the comparative example of this invention, with 200 training samples.

[0049] Figure 14a This is a single-spectral value image corresponding to 26 bands on the hyperspectral image of the IP scene in the comparative example of the present invention; Figure 14b This is a single-spectral value image corresponding to band 53 on the hyperspectral image of the IP scene in the comparative example of the present invention; Figure 14c This is a single-spectral value image corresponding to band 121 on the hyperspectral image of the IP scene in the comparative example of the present invention; Figure 14dThis is a single-spectral value image corresponding to band 177 in the hyperspectral image of the IP scene in the comparative example of the present invention; Figure 14e This is a dense optical flow map of the 26-band monospectral value image on the hyperspectral image of the IP scene in the comparative example of the present invention; Figure 14f This is a dense optical flow map of the 53-band monospectral value image on the hyperspectral image of the IP scene in the comparative example of the present invention; Figure 14g This is a dense optical flow map of the 121-band corresponding monospectral value image on the hyperspectral image of the IP scene in the comparative example of the present invention; Figure 14h This is a dense optical flow map of the 177-band corresponding monospectral value image on the hyperspectral image of the IP scene in the comparative example of the present invention;

[0050] Figure 15a This is a single-spectral value image corresponding to band 26 on the hyperspectral image of the SA scene in the comparative example of the present invention; Figure 15b This is a single-spectral value image corresponding to band 53 on the hyperspectral image of the SA scene in the comparative example of the present invention; Figure 15c This is a single-spectral value image corresponding to band 121 on the hyperspectral image of the SA scene in the comparative example of the present invention; Figure 15d This is a single-spectral value image corresponding to band 177 on the hyperspectral image of the SA scene in the comparative example of the present invention; Figure 15e This is a dense optical flow map of the 26-band corresponding monospectral values ​​on the hyperspectral image of the SA scene in the comparative example of the present invention; Figure 15f This is a dense optical flow map of the 53-band monospectral value image on the hyperspectral image of the SA scene in the comparative example of the present invention. Figure 15g This is a dense optical flow map of the monospectral value image corresponding to band 121 on the hyperspectral image of the SA scene in the comparative example of the present invention; Figure 15h This is a dense optical flow map of the 177-band monospectral value image on the hyperspectral image of the SA scene in the comparative example of the present invention.

[0051] Figure 16a This is a single-spectral value image corresponding to band 26 on the hyperspectral image of the PU scene in the comparative example of the present invention; Figure 16b This is a single-spectral value image corresponding to band 53 on the hyperspectral image of the PU scene in the comparative example of the present invention; Figure 16c This is a single-spectral value image corresponding to band 121 on the hyperspectral image of the PU scene in the comparative example of the present invention; Figure 16d This is a single-spectral value image corresponding to band 177 on the hyperspectral image of the PU scene in the comparative example of the present invention; Figure 16e This is a dense optical flow map of the 26-band corresponding monospectral values ​​on the hyperspectral image of the PU scene in the comparative example of the present invention; Figure 16f This is a dense optical flow map of the 53-band monospectral value image on the hyperspectral image of the PU scene in the comparative example of the present invention. Figure 16gThis is a dense optical flow map of the 121-band corresponding monospectral value image on the hyperspectral image of the PU scene in the comparative example of the present invention; Figure 16h This is a dense optical flow map of the 177 band corresponding to the monospectral value image on the hyperspectral image of the PU scene in the comparative example of the present invention;

[0052] Figure 17a This is a single-spectral value image corresponding to band 26 on the hyperspectral image of the HO scene in the comparative example of the present invention; Figure 17b This is a single-spectral value image corresponding to band 53 on the hyperspectral image of the HO scene in the comparative example of the present invention; Figure 17c This is a single-spectral value image corresponding to band 121 on the hyperspectral image of the HO scene in the comparative example of the present invention; Figure 17d This is a single-spectral value image corresponding to band 177 on the hyperspectral image of the HO scene in the comparative example of the present invention; Figure 17e This is a dense optical flow map of the 26 bands corresponding to the monospectral values ​​on the hyperspectral image of the HO scene in the comparative example of the present invention. Figure 17f This is a dense optical flow map of the 53-band monospectral value image on the hyperspectral image of the HO scene in the comparative example of the present invention. Figure 17g This is a dense optical flow map of the 121 band corresponding to the monospectral value image on the hyperspectral image of the HO scene in the comparative example of the present invention. Figure 17h This is a dense optical flow map of the 177 band corresponding to the monospectral value image on the hyperspectral image of the HO scene in the comparative example of the present invention.

[0053] Figure 18a This is the spectral curve of the PU data before spectral motion feature extraction in the comparative example of this invention; Figure 18b The image shows the spectral curves after spectral motion features were extracted from the PU data in the comparative example of this invention.

[0054] Figure 19a This is the T-SNE visualization result before spectral motion feature extraction in the IP scene in the comparative example of the present invention; Figure 19b This is the T-SNE visualization result before spectral motion feature extraction in the SA scene in the comparative example of the present invention; Figure 19c This is the T-SNE visualization result before spectral motion feature extraction in the PU scene in the comparative example of the present invention; Figure 19d This is the T-SNE visualization result before spectral motion feature extraction in the HO scene in the comparative example of the present invention; Figure 19e This is the T-SNE visualization result after spectral motion feature extraction in the IP scene in the comparative example of the present invention; Figure 19f This is the T-SNE visualization result after spectral motion feature extraction in the SA scene in the comparative example of the present invention; Figure 19g This is the T-SNE visualization result after spectral motion feature extraction in the PU scene in the comparative example of the present invention; Figure 19hThis is a T-SNE visualization of the spectral motion features extracted in the HO scene in the comparative example of the present invention. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0056] Examples of Hyperspectral Image Classification Methods

[0057] This embodiment provides a hyperspectral image classification method, referring to... Figure 1 The specific steps are as follows:

[0058] 1) Unfold the hyperspectral image to be classified along the spectral dimension to obtain several frames of single-spectral value images.

[0059] The hyperspectral image to be classified can be a full-frame hyperspectral image of any size. Here, H, W, and C represent the image's height, width, and number of bands, respectively. The spectral vector of any pixel in a hyperspectral image possesses inherent motion properties, and the spectral vectors between adjacent pixels also imply motion consistency. Therefore, the spectral motion features corresponding to the relative motion between adjacent image frames, extracted from spectral values, possess both spatial and spectral attributes, serving as a valuable supplement to discriminative features for distinguishing land cover categories. Therefore, hyperspectral image I is unfolded along the spectral dimension to obtain C frames of single-spectral value images.

[0060] Since the trend of spectral motion changes is related to the changes in single-band images of consecutive frames, and the changes in the spectral value sequence are continuous, the changes between adjacent single-band images are relatively small, while the differences between single-band images with a certain interval are larger. Therefore, different frame intervals will affect the final presentation of spectral motion features. In this embodiment, in order to make adjacent frames better reflect more distinguishable and refined spectral motion features, a number of single-spectral value images of n bands are obtained by continuously extracting frames. Spectral dimensions are then expanded, and the spectral motion features of these n single-spectral value images are calculated pairwise between adjacent frames to finally obtain spectral motion features with a dimension of n-2.

[0061] 2) Obtain spectral flow motion information of each frame of single-spectral value image by using the Deep Maching method.

[0062] The Deep Matching method can retrieve dense correspondences from the feature correspondences of deformable non-rigid descriptors, thereby establishing quasi-dense matching relationships between adjacent frames. The process by which the Deep Matching method obtains the spectral flow motion information of each frame of single-spectral image is as follows:

[0063] ① Divide one frame in each adjacent frame monospectral image pair into sub-image blocks according to the minimum image size without overlap;

[0064] For situations where the same matching region contains multiple objects moving in different directions, such as Figure 2 As shown, in this embodiment, for adjacent frame single spectral value image pairs I 1 ,I 2 , will I 1 The images are divided into sub-image blocks without overlap according to the minimum image size M. The minimum image size is usually related to the size of the hyperspectral image to be classified and the required classification accuracy. In this embodiment, the minimum image size is 4×4 pixels. These sub-image blocks are called meta-sub-image blocks, that is, the lowest level of sub-image block size.

[0065] In practice, a large minimum image size may lead to excessive interference from weakly correlated matching information, resulting in low matching efficiency. Conversely, a small minimum image size may cause the loss of crucial correlated matching information, reducing matching accuracy. The fineness of the matching affects the actual matching effect, which in turn affects the contribution of the matching term in the subsequent energy minimization constraint. Therefore, the setting of the minimum image size needs to balance matching efficiency and matching accuracy.

[0066] ② Obtain the response of another frame image in the monospectral image pair to each sub-image block of the current frame image; then stitch together any four adjacent sub-image blocks to obtain different parent image blocks, and obtain the response of the corresponding parent image block through the response of the sub-image blocks. The response of a sub-image block is the similarity between each sub-image block in the other frame image and the sub-image blocks in the current frame image; the higher the similarity, the larger the response value. A sub-image block in the other frame image refers to any sub-image block in the other frame image with the same size as a sub-image block in the current frame image. The specific implementation of the above process is as follows: I 1 ,I 2 For adjacent frame single-spectral value image pairs, let I 1 N,p Representing I 1 The sub-image patch has a size of N×N and a center p. In this embodiment, N=4, so the set of centers p is φ4={2,6,10,...,W-2}×{2,6,10,...,H-2}, where φ4 is a grid with a spacing of 4 pixels, and W and H represent I. 1 ,I 2 Width and height;

[0067] First, a convolution operation is used to compute I. 2 About I 1 4,p Response diagram:

[0068] C 4,p=I 1 4,p *I 2

[0069] Then for I 2 Any sub-image patch I centered at p' 2 4,p' Its relationship with I 1 4,p The similarity is C. 4,p (p'), where p' is I 2 The center of the sub-image block with the same size as the sub-image block of the current layer in the image frame is determined. After traversing the set φ4, the responses of all the bottom-level sub-image blocks are obtained. However, such low-level responses are not enough to obtain good discriminability. Therefore, step ③ is continued to iteratively stitch together sub-image blocks with a larger coverage area to obtain higher-level responses.

[0070] ③ The obtained parent image patch is used as a higher-level sub-image patch, and the process in ② is repeated iteratively until the obtained parent image patch is the original single-spectral image before partitioning. The maximum local response of sub-image patches at different levels is found, and then a fine pixel-level correspondence is obtained step by step from the top to the bottom as the matching result: The sub-image patch obtained in ① is used as a meta-sub-image patch. Each meta-sub-image patch is matched with the sub-image patches at each level that contain the meta-sub-image. By finding the correspondence between the maximum local response of each meta-sub-image patch and the sub-image patches at each level that contain the meta-sub-image, the matching vector field is obtained as the matching result; the maximum local response is the maximum value in the response of each level of sub-image patch.

[0071] Any four adjacent sub-image blocks can be stitched together to form a unique parent image block. Where s i This represents the center position shift of a sub-image patch relative to its parent image patch. The response corresponding to a higher-level parent image patch can then be obtained by the maximum similarity weighted average of the responses corresponding to each sub-image patch. This process is essentially an aggregation transformation of lower-level responses to obtain higher-level responses.

[0072]

[0073] Where θ is the sub The set of centers m', N = M × 2 t t is the number of iterations; M is the minimum image size, which in this embodiment is M = 4; here p' is correspondingly I 2 The center of the parent image block with the same size as the parent image block in the current layer of the image; p+S N，i S represents the center location of the low-level response map. N，iThis represents the shift in the center position of the lower-level response relative to the higher-level response. In each iteration step, to maintain good transitivity between responses at different levels, a nonlinear exponential transformation C(·)→C(·) is applied to the above equation. λ The correction is performed to obtain the response corresponding to the corrected parent image block.

[0074] Finally, the maximum values ​​of local responses at different levels are found, and then a fine pixel-level correspondence is obtained step by step from the top to the bottom, thereby achieving deep multi-level matching. Specifically, the sub-image blocks obtained in ① are taken as meta-sub-image blocks, and each meta-sub-image block is matched with the sub-image blocks containing that meta-sub-image at each level. By finding the correspondence between the maximum values ​​of local responses of each meta-sub-image block and the sub-image blocks containing that meta-sub-image at each level from the top to the bottom, the matching vector field is obtained. As a matching result, the maximum local response value here is the maximum value among the responses of sub-image blocks at each level. The response of a sub-image block is the similarity between each sub-image block in another frame and the sub-image blocks in this frame.

[0075] ④ The matching results are incorporated into the optimization framework of minimizing optical flow energy to obtain the spectral flow motion information of each frame of single-spectral value image.

[0076] Based on the LDOF method, the optimization objective of minimizing energy is defined as follows, and the optical flow is obtained through optimization. As spectral flow motion information.

[0077]

[0078] Where Ω represents the image pair I of adjacent frames with single spectral values. 1 ,I 2 The spatial location, i.e., the full-frame hyperspectral image; E D E S and E M There are three different constraint terms, where E D For data items; E S E is a smoothing term. M For the matching term, a robust penalty term is used for the above expression. In a preferred embodiment, setting ε = 0.001 yields better results. α and β are the weighting parameters for the smoothing and matching terms, respectively. Controlling the contribution of these two constraint terms during optimization will have a significant impact on the optical flow estimation results.

[0079] After normalizing the color and gradient consistency assumptions, the two are summed together using a penalty term ψ(·) to form a data term. It can be converted into A is the parameter to be substituted; first, a brightness uniformity assumption is made: If it is a space-time gradient, then its corresponding penalty term is ψ(w). T M0 i w), where i is a constant term, and in this embodiment, i takes the value of 1; where the tensor In addition, a coefficient θ is added to ensure the numerical stability of the normalization process, i.e. Based on the consistency of the penalty gradient, assuming I x ,I y Let these represent changes along the horizontal and vertical directions, respectively. This represents the normalized tensor. Therefore, the data item includes penalty terms for both of the above, and is constrained by weight coefficients δ and γ respectively:

[0080]

[0081] The smoothing term uses a conventional, robust gradient paradigm penalty term:

[0082]

[0083] The matching term needs to penalize the difference between the pre-calculated vector field w' and the optical flow w, and also add a binomial term b(x), where the vector field w' is the matching vector field obtained in step ③; in addition, a smaller penalty term φ(x) is added to constrain the region of ambiguous matching, and the matching term is then:

[0084] E M =bφψ(||w-w'|| 2 )

[0085] During minimization, the optimization framework in the Brox optical flow method is used to minimize the non-convex nonlinear energy expression.

[0086] The obtained optical flow This is the corresponding spectral flow motion information.

[0087] 3) By connecting them along the channel dimension, the spectral flow motion information of all single-spectral images is fused to obtain the spectral motion features of the hyperspectral image to be classified. These spectral motion features are then fused with the original spectral information of the hyperspectral image to be classified to obtain the spectral fusion features used for classification. In this embodiment, the channel dimension connection involves concatenating the extracted features (i.e., the spectral flow motion information of the single-spectral images) along the channel dimension; this is essentially a matrix concatenation operation.

[0088] Spectral motion features serve as a valuable supplement to the original spectral information. The spectral fusion features obtained by fusing the two are used as input to the classifier. In a preferred embodiment, the spectral motion features are directly connected to the original features for fusion, which allows the information of the two to be fully complementary. In this embodiment, the classifier is a support vector machine (SVM), thereby obtaining the classification result. In other embodiments, sparse representation classifiers, random forests, or extreme learning machines can also be selected as classifiers.

[0089] Comparative example:

[0090] To comprehensively compare and analyze the hyperspectral image classification method of this invention, this comparative example selects four mainstream hyperspectral image benchmark datasets, including two types of field scenes: Indian Pines (IP) and Salinas (SA), and two types of urban scenes: Pavia University (PU) and Houston 2013 (HO), to experimentally verify the classification performance of the method of this invention. Detailed data for the four datasets are shown in Table 1.

[0091] Table 1. Detailed data on four types of hyperspectral images.

[0092]

[0093] The experimental hardware environment consisted of an Intel® Xeon® Gold 6152 CPU, an Nvidia A100 PCIe GPU, and 128GB of RAM. The experimental algorithm was developed and implemented using Python and the OpenCV library. All experiments were conducted iteratively 10 times on four datasets, and the results were averaged. The selection of training samples varied across these ten experiments to ensure the method's robustness under different sample distributions. However, in comparing different methods, the selection of training samples remained the same in each iteration, ensuring fairness in the comparison.

[0094] This comparative example experimentally analyzes key parameters that may affect the extraction performance of spectral motion features. In the Deepmatching method, the size M of the smallest image in the multi-scale response map determines the fineness of the matching, but this determination is often relative, as the size of the original image also influences the magnitude of this parameter to some extent. Theoretically, if M is large, it may lead to excessive interference from weakly correlated matching information; if M is small, it may lose key correlated matching information. The fineness of the matching affects the actual matching effect, and thus affects the contribution of the matching term in the subsequent energy minimization constraint. Furthermore, in the energy minimization optimization process, the smoothing term weight parameter α directly affects the degree of influence of smoothing consistency on optical flow estimation. To analyze the influence of the above two factors on spectral motion feature extraction and determine the optimal parameters, a cross-validation strategy was used on four sets of hyperspectral images to determine the classification performance under different parameter combinations, where M∈{15,20,25,30,35}, α∈{1,2,3,4,5,6,7,8,9,10}. The results are as follows. Figures 3a-3d As shown; Figures 3a-3d The charts present 3D bar graphs showing the classification performance of the IP, SA, PU, ​​and HO scenarios under different α and M settings. The IP and SA scenarios are outdoor scenes with predominantly areal features, while the PU and HO scenarios are urban scenes with a more complex distribution of linear and areal features. In the IP and SA scenarios, the classification performance initially decreases and then increases with increasing α, while in the PU and HO scenarios, the classification performance increases slowly with increasing α until saturation. Furthermore, different M values ​​significantly affect the classification performance in the IP and SA scenarios, but have little impact on the PU and HO scenarios. Therefore, for the four datasets, the final α values ​​for the IP, SA, PU, ​​and HO scenarios were set to 0.5, 7, 8, and 8, respectively, and the M value was set to 15 for all of them.

[0095] Since optical flow is generated by the relative changes between consecutive frames, the trend of spectral motion is also related to the changes in single-band images between consecutive frames. Because the spectral value sequence changes continuously, the changes between adjacent single-band images are relatively small, while the differences between single-band images with a certain interval are larger. Therefore, different frame intervals D affect the final presentation of spectral motion features. On the hyperspectral images corresponding to four scenes, spectral motion feature extraction was performed under different D values, and the corresponding classification results are as follows: Figures 4a to 4d As shown above, Figures 4a to 4d In the diagram, the horizontal axis represents the interval between adjacent frames during frame extraction, and the vertical axis represents the overall classification accuracy (OA). This is achieved through observation... Figures 4a to 4dBox plots revealed that when D=0, the average classification accuracy and relatively concentrated data distribution were often higher across the four sets of scene data. This indicates that continuous frame sampling allows adjacent frames to better reflect more discriminative and detailed spectral motion features, thus improving classification accuracy.

[0096] To analyze the impact of the method of the present invention (referred to as SpeMotion in the following comparative experiments to represent the hyperspectral image classification method of the present invention) on the acquisition of spectral motion features and the method of fusion on classification, this comparative example designs an ablation experiment for verification. First, the original features of the hyperspectral image without any processing are used as input, which is regarded as the Baseline. Then, the spectral motion features extracted according to the method of the present invention are used as input alone, denoted as SpeMotion_single. Next, this comparative example designs four fusion methods. The first method is to multiply the spectral motion feature map with the corresponding original single-band image to assign weights. This can be regarded as a simple spectral attention focusing on the spectral dimension of the global image, making the spectral features more discriminative, denoted as SpeMotion_mul. The second method is to add the spectral motion feature map with the corresponding original single-band image. This can be regarded as a kind of information supplementation, denoted as SpeMotion_sum. The third method is to directly connect the original features of the hyperspectral image and its spectral motion features, denoted as SpeMotion_con. The fourth method combines the features extracted at different frame extraction distances D to form a multi-scale spectral motion feature, which is then concatenated with the original features, denoted as SpeMotion_multi-scale. The classification results for different feature input methods in the four scenarios are shown in Table 2.

[0097] Table 2. Classification performance of SpeMotion under different feature fusion modes

[0098]

[0099]

[0100] Observations show that the extracted spectral motion features are more discriminative than the original spectral features. Inappropriate fusion methods, such as `SpeMotion_multiply` and `SpeMotion_sum`, may result in varying degrees of reduced discriminative power. In contrast, directly concatenating the spectral motion features with the original features allows for sufficient complementarity of their information, leading to better discriminative power. Furthermore, combining spectral motion features at different scales does not further improve the discriminative power of the features. Therefore, in subsequent experiments, this comparative example uses `SpeMotion_concat` as the fusion method for spectral motion features.

[0101] To comprehensively verify the effectiveness of the method of this invention, traditional feature extraction-based classification methods and advanced deep learning-based feature extraction and classification methods were compared with the classification method of this invention. First, classic spatial feature extraction methods that perform better among traditional methods were compared, including Gabor features, Extended Morphological Profile (EMP), and Local Binary Pattern (LBP), a representative texture feature extraction method. Among deep learning-based feature extraction methods, two representative methods were selected: one is the Deep Small Sample Feature Extraction (DFSL) method, which addresses the problem of insufficient samples by learning hyperspectral image features from a constructed source dataset; the other is the unsupervised 3D Convolutional Autoencoder (3D-CAE) feature extraction framework, which can learn data features unsupervised from large amounts of unlabeled data. All of the above-mentioned traditional and deep learning-based feature extraction methods, including the method of this invention, input the features into an SVM for training and classification after feature extraction; the level of feature extraction directly determines the classification effect. Finally, two state-of-the-art (SOTA) deep learning-based classification methods were compared: one is the Spatial Spectral Network (SSTN) built based on Transformer, which better mines the spatial spectral information of images by introducing Transformer. Another approach is the FullyContNet, a global context network with embedded multi-scale attention modules. This method uses global images as input and can significantly improve classification performance by mining contextual information. The parameters of all compared methods were configured and optimized according to existing techniques. Ten training samples were used, and all other samples with ground truth labels were used as test samples. To ensure fairness and fully examine the true performance of each method, the samples drawn in the 10 repeated experiments were completely identical across different methods, while the samples drawn in the 10 repeated experiments for the same method were different for each method. The experimental results are presented as the mean and variance of the 10 repeated experiments; the quantitative comparison results are shown in Tables 3–6.

[0102] Table 3. Classification results (%) of different algorithms on the Indian Pines dataset.

[0103]

[0104]

[0105] Table 4. Classification results (%) of different algorithms on the Salinas dataset

[0106]

[0107] Table 5. Classification results (%) of different algorithms on the Pavia University dataset

[0108]

[0109]

[0110] Table 6. Classification results (%) of different algorithms on the Houston dataset.

[0111]

[0112] Based on Tables 3–6, it can be seen that among traditional classical feature extraction methods, Gabor features perform the worst, while LBP performs better than EMPs in scenarios with obvious texture features, such as IP and PU scenarios. DFSL constructs a deep 3D convolutional neural network model and performs Few-Shot Learning on the source dataset, which is a supervised learning process that enables the model to initially extract deep features. 3D-CAE constructs a 3D convolutional autoencoder, extracting features during the encoding process and recovering features during the decoding process, enabling the model to learn its feature extraction capabilities unsupervised. The performance of DFSL and 3D-CAE demonstrates that deep learning-based feature extraction methods can significantly improve the model's feature extraction capabilities through a data-driven approach, thereby obtaining more robust discriminative features than traditional classical feature extraction methods. All the above methods were trained and tested using SVM, and the classification results support this conclusion. SSTN and FullyContNet, as state-of-the-art deep learning-based classification methods, achieve excellent classification performance in all four classification scenarios. The SpeMotion method of this invention achieves classification by extracting motion features from the spectrum. Experimental results show that the discriminative power and robustness of spectral motion features are far superior to traditional and deep learning-based feature extraction methods. Furthermore, training spectral motion features using only a simple SVM achieves classification accuracy comparable to state-of-the-art deep learning classification methods, obtaining the highest overall accuracy (OA) in IP, SA, and PU scenarios, further demonstrating the powerful discriminative ability of spectral motion features.

[0113] To further facilitate a more intuitive comparison, we created global classification thematic maps for different methods across four scenarios, such as... Figure 5a – Figure 8j As shown, where, Figure 5a Real hyperspectral images of IP scenarios; Figure 5b For the true marking of IP scenarios; Figures 5c-5j These are global classification thematic maps of the Gabor algorithm, EMPs algorithm, LBP algorithm, DFSL algorithm, 3D-CAE algorithm, SSTN algorithm, FullyContNet algorithm, and SpeMotion algorithm in the IP scenario. Figure 6a This is a true hyperspectral image of the SA scene; Figure 6b For realistic marking of SA scenes; Figures 6c-6j These are global classification thematic maps of the Gabor algorithm, EMPs algorithm, LBP algorithm, DFSL algorithm, 3D-CAE algorithm, SSTN algorithm, FullyContNet algorithm, and SpeMotion algorithm on SA scenes, respectively. Figure 7a A true hyperspectral image of the PU scene; Figure 7b For realistic marking of PU scenarios; Figures 7c-7j These are global classification thematic maps of the Gabor algorithm, EMPs algorithm, LBP algorithm, DFSL algorithm, 3D-CAE algorithm, SSTN algorithm, FullyContNet algorithm, and SpeMotion algorithm in the PU scene. Figure 8a This is a true hyperspectral image of the HO scene; Figure 8b For realistic labeling of HO scenes; Figures 8c-8j These are global classification thematic maps of the Gabor, EMPs, LBP, DFSL, 3D-CAE, SSTN, FullyContNet, and SpeMotion algorithms in HO scenes.

[0114] As can be seen, the classification map performance is generally consistent with the classification accuracy. Low classification accuracy introduces a large amount of scattered noise, such as Gabor and EMPs. While LBP has slightly higher classification accuracy, its extracted texture features result in a distorted classification map, with excessive texturing leading to poor visual quality. This illustrates that high-accuracy methods do not necessarily produce classification maps that meet application requirements. DFSL and 3D-CAE classification maps significantly reduce noise, but their overall visual quality remains poor under small sample conditions. SSTN and FullyContNet classification maps effectively smooth out noise, achieving better visual quality, but they also suffer from edge bulging and detail loss. Our proposed method, SpeMotion, achieves classification map results closest to the original image and ground truth labels. The strong discriminative power of spectral motion features ensures that SVM classification alone can significantly reduce noise. Furthermore, relying solely on the motion features of the spectrum itself for classification ensures that the details of the classification map are preserved as much as possible, resulting in better classification map performance.

[0115] To more comprehensively compare the adaptability of different methods under various training sample sizes, the training sample size was expanded to 20, 50, 80, 100, 150, and 200, and the experiments were repeated for different methods with other experimental settings unchanged. The experimental results are as follows. Figures 9a-9d As shown; where Figures 9a-9dThe figures show the classification performance changes of different methods with varying training sample sizes in the IP, SA, PU, ​​and HO scenarios of the comparative examples of this invention. It is evident that the method of this invention achieves leading classification levels under various sample conditions with different data, and even with sufficient samples, it achieves classification results comparable to or even better than advanced deep learning methods. The excellent performance in various scenarios fully validates the powerful discriminative ability of the spectral motion features extracted by the method presented in this paper.

[0116] Since the method of this invention still achieves the highest classification accuracy when the training samples reach 200 per class, along with the other two best-performing deep learning-based methods, SSTN and FullyContNet, it can achieve almost 100% classification accuracy. At this point, simply comparing accuracy is insufficient to demonstrate the full effect. To further intuitively showcase the advantages of the proposed method, global classification maps for four scenarios corresponding to the three methods under the condition of 200 training samples per class are created, such as... Figures 10a-13d As shown, Figure 10a Real hyperspectral images in IP scenarios; Figure 10b This is a global classification thematic map of the IP scene using the SSTN algorithm with 200 training samples. Figure 10c This is a global classification thematic map of the IP scene using the FullyContNet algorithm with 200 training samples. Figure 10d This is a global classification thematic map of the IP scene using the SpeMotion algorithm with 200 training samples. Figure 11a This is a true hyperspectral image of an SA scene; Figure 11b This is a global classification thematic map of the SA scene using the SSTN algorithm with 200 training samples. Figure 11c This is a global classification thematic map of the SA scene using the FullyContNet algorithm with 200 training samples. Figure 11d This is a global classification thematic map of the SA scene using the SpeMotion algorithm with 200 training samples. Figure 12a Real hyperspectral images of the PU scene; Figure 12b This is a global classification thematic map of the PU scene using the SSTN algorithm with 200 training samples. Figure 12c This is a global classification thematic map of the PU scene using the FullyContNet algorithm with 200 training samples. Figure 12d This is a global classification thematic map of the PU scene using the SpeMotion algorithm with 200 training samples. Figure 13a This is a true hyperspectral image of the HO scene; Figure 13b This is a global classification thematic map of the HO scene using the SSTN algorithm with 200 training samples. Figure 13cThis is a global classification thematic map of the HO scene using the FullyContNet algorithm with 200 training samples. Figure 13d This is a global classification thematic map of the HO scene using the SpeMotion algorithm with 200 training samples.

[0117] It should be noted that in comparing experimental results of hyperspectral image classification, classification maps containing only the true marked areas of ground features are typically used for comparison. However, non-global classification maps cannot fully verify the effectiveness of the classification method. A method with high classification accuracy may not necessarily produce a high-quality global classification map, and therefore lacks practical application value. Therefore, this comparative example uses global classification maps for a clearer comparison to fully verify the effectiveness and practicality of the method of this invention. Since the most discriminative feature of hyperspectral images is the pixel-by-pixel spectral feature, classification methods that only utilize spectral features can often depict ground feature classification maps that better match the details of the original image. However, the similarity between spectral classes and the differences within classes often lead to low classification accuracy and high noise in the classification map. While utilizing spatial attributes can significantly improve classification accuracy, it often leads to problems such as blurred feature boundaries and unclear identification of detailed features in the classification map. Therefore, this invention extracts spectral motion features, allowing the significant discriminative potential inherent in spectral attributes to be truly realized. The extracted spectral motion features can then be finely classified pixel-by-pixel using a simple classifier (SVM), achieving high classification accuracy while perfectly solving the problem of massive noise in the classification map of spectral feature classification methods. Furthermore, the small features and feature boundaries of the original image in the classification map can be accurately reproduced. For example, trees in soybean fields and the slender road and trees above the area in the IP scene; the rough fallow land boundary in the lower left of the area in the SA scene; the metal roof in the upper right of the area in the PU scene; and the circular commercial center in the lower left of the area and the road above the area in the HO scene. These details appear to varying degrees of dilation, distortion, or loss of detail in the classification maps of the other two state-of-the-art (SOTA) methods. This demonstrates that the method presented in this paper achieves high classification accuracy while producing classification maps that maximally reproduce the detailed information of the real image, possessing good practicality.

[0118] To analyze the computational efficiency of the proposed method, three methods with superior performance—SSTN, FullyContNet, and the SpeMotion method of this invention—were selected for comparison in training and testing efficiency. The results are shown in Table 7.

[0119] Table 7. Computation time of SSTN, FullyContNet, and SpeMotion methods on different datasets.

[0120]

[0121] Deep learning-based methods only include training and testing phases. As shown in Table 7, SSTN has a relatively long training and testing time. In contrast, FullyContNet, which uses full-frame images as input, has higher training and testing efficiency. SpeMotion, in addition to training and testing, also includes feature extraction. In motion feature extraction, SpeMotion's time typically increases with the image size, potentially leading to significant computational time for large images. However, in classification, the classifier is highly efficient because it directly trains and tests the extracted motion features, thus compensating for the long feature extraction time. Furthermore, SSTN, which uses local data cubes as input, experiences a significant increase in training time with the increase in training samples, while FullyContNet, as an image-level classification method, does not have this problem. Similarly, SpeMotion, which uses SVM for training and classification, does not experience significant computational pressure even with a large increase in training samples, and training and testing can be repeated after motion feature extraction. Therefore, in practical applications, the hyperspectral image classification method of this invention is a fast and flexible approach.

[0122] Furthermore, to verify the effectiveness of the Deepmatching-based optical flow method used in extracting spectral motion features, several classic optical flow calculation methods were selected and replaced, including the pyramid LK optical flow method, the Brox optical flow method, LDOF, and SIFTflow. All other settings remained unchanged. The differences in classification performance brought about by the different methods after optical flow calculation method replacement were compared on four sets of hyperspectral images. The results are shown in Table 8.

[0123] Table 8 Performance of Spectral Motion Feature Extraction by Different Optical Flow Calculation Methods

[0124] IP SA PU HU SpeMotion_PyrL-K 69.85 87.12 66.99 75.41 SpeMotion_Brox 73.02 91.82 73.63 77.89 SpeMotion_LDOF 76.23 94.47 83.97 80.67 SpeMotion_SIFTFlow 72.93 92.27 79.84 78.42 SpeMotion_Deepmatch 85.46 96.09 90.49 82.07

[0125] Table 8 shows that the performance of the optical flow calculation method directly affects the representation of spectral motion features, and thus the classification effect. Compared with the classic LK optical flow method, the Brox optical flow method achieves better results by adding gradient consistency constraints and adopting a coarse-to-fine optimization strategy. LDOF and SIFTflow integrate the matching relationship between adjacent frames into the variational method, thus exhibiting better adaptability in optical flow estimation. For variational optical flow methods that incorporate matching terms, including the method of this invention, the establishment of matching descriptors and matching relationships becomes crucial to algorithm performance. Both LDOF and SIFTflow use local rigid matching descriptors, and the rigidity dependency assumption often leads to poor performance in complex non-rigid transformation scenarios. The Deep Matching method used in this paper can retrieve dense correspondences from the feature correspondences of deformable non-rigid descriptors, thus exhibiting stronger adaptability to complex changes. Experimental results show that the variational optical flow method based on Deep Matching adopted in this invention can better adapt to the task of extracting spectral motion features from hyperspectral images and achieves optimal classification results.

[0126] Hyperspectral imagery is a high-dimensional image data, and optical flow is an important method for estimating motion information in sequential frames of video data. This paper treats hyperspectral imagery as sequential data and attempts to extract motion information in the spectral dimension of hyperspectral imagery to obtain more discriminative features. This motion information actually reflects the grayscale value changes in the spectral dimension. Therefore, this invention proposes to introduce optical flow technology to extract such motion information, thereby achieving motion feature extraction in the spectral dimension. To visually demonstrate this process, this comparative example randomly selects single-spectral value images of five bands from four sets of hyperspectral imagery and their corresponding dense optical flow maps as spectral motion information, as shown in Figures 14-17. Figures 14a-14d The image shows the monospectral values ​​corresponding to bands 26, 53, 121 and 177 in the hyperspectral imagery of the IP scene. Figure 14e-14h Dense optical flow maps of the monospectral values ​​corresponding to bands 26, 53, 121 and 177 in the hyperspectral imagery of the IP scene; Figures 15a-15d The image shows the monospectral values ​​corresponding to bands 26, 53, 121 and 177 in the hyperspectral image of the SA scene. Figure 15e-15h Dense optical flow maps of the monospectral values ​​corresponding to bands 26, 53, 121, and 177 in the hyperspectral imagery of the SA scene; Figures 16a-16d The image shows the monospectral values ​​of bands 26, 53, 121 and 177 on the hyperspectral image of the PU scene. Figure 16e-16hDense optical flow maps of the monospectral values ​​corresponding to bands 26, 53, 121 and 177 on the hyperspectral image of the PU scene; Figures 17a-17d The image shows the monospectral values ​​corresponding to bands 26, 53, 121 and 177 in the hyperspectral image of the HO scene. Figure 17e-17h Dense optical flow maps of the monospectral values ​​corresponding to bands 26, 53, 121, and 177 in the hyperspectral imagery of the HO scene.

[0127] It should be noted that this comparative example uses the HSV color mode to render the calculated optical flow results. Differences in color represent different directions of optical flow, and variations in color brightness represent the magnitude of optical flow. As can be seen, the changes and differences in grayscale values ​​across different spectral bands are reflected as motion information in the optical flow results. The optical flow results presented in dense optical flow maps indicate that homogeneous regions tend to exhibit consistency in variation, thus presenting regions of similar color variation with a certain spatial range and outline, while heterogeneous regions show the opposite. This consistency in variation is actually extended from the motion changes of the spectrum to the spatial dimension; therefore, the extracted spectral motion features also possess spatial-spectral properties.

[0128] After the optical flow calculation was completed, to further demonstrate the extracted motion spectral features, this comparative example plotted spectral curves before and after feature extraction. Taking a PU scene containing nine types of land features as an example, an arbitrary entire spectral sequence was taken for each type of land feature and represented by different colors. The results are as follows: Figure 18a , 18b As shown. Figure 18a These are the spectral curves of the original data after normalization. It can be seen that categories with large spectral differences are easily distinguished, such as Metal Sheets and Trees. However, for categories with small inter-class differences, their spectral curves almost overlap, making them more difficult to distinguish, such as Graval and Bricks. Figure 18b This is the spectral motion curve obtained by multiplying u and v for each band after the spectral motion features are extracted. It can be seen that the differences between the spectral motion curves of different categories are amplified, thus making them more distinguishable, especially in the band range of approximately 65-75. This indicates that the spectral motion features actually reshape their original characteristics, significantly improving their distinguishability and discriminative power, thereby making them more beneficial for land cover classification.

[0129] To visually verify this conclusion, this comparative example uses T-SNE nonlinear dimensionality reduction visualization technology to visualize the spectral features before and after feature extraction on four sets of data. Specifically, 100 feature samples are taken for each land cover class, and for classes with insufficient samples, the total number of samples for that class is used. The visualization results are as follows: Figures 19a-19hAs shown, the left side of each set of images represents the original feature results, and the right side represents the extracted spectral motion feature results. Figures 19a-19d These are the T-SNE visualization results before spectral motion feature extraction in the IP, SA, PU, ​​and HO scenarios, respectively, in the comparative examples of this invention. Figure 19e-19h This is a T-SNE visualization result after spectral motion feature extraction in the IP, SA, PU, ​​and HO scenarios in the comparative examples of this invention. Different categories of features are rendered with different colors, and the spatial distance between feature points represents their similarity. It can be seen that before feature extraction, different categories in the original features overlap due to similarity, while the same category exhibits overly dispersed distribution. After feature extraction, the spectral motion features of different categories show better separability, while features of the same category are more clustered in distribution. This distribution characteristic effectively verifies the above conclusions.

[0130] This invention proposes a hyperspectral image classification method that utilizes Deep Maching optical flow to extract motion features in the spectral dimension as classification features for hyperspectral images. This method can extract the relative motion between adjacent frames in the spectral dimension. It leverages the inherent motion attributes of the spectral vectors of any pixel in a hyperspectral image and the motion consistency implied by the spectral vectors of adjacent pixels to obtain spectral motion features with both spatial and spectral attributes. This serves as a valuable supplement to discriminative features for distinguishing land cover categories, thereby improving classification accuracy. Extensive experiments demonstrate that: 1) The spectral motion features extracted in this invention have excellent discriminative power, outperforming traditional classical feature extraction methods and deep learning-based feature extraction methods. 2) The classification method proposed in this invention achieves good classification performance even with a small sample size of 10 land cover categories, and under various sample conditions, it can obtain classification results that compete with state-of-the-art deep learning-based classification methods. 3) The classification method proposed in this invention achieves high classification accuracy while also producing more detailed thematic maps, demonstrating good practicality.

[0131] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A hyperspectral image classification method, characterized in that, The steps are as follows: 1) unfolding the hyperspectral image to be classified along the spectral dimension to obtain a plurality of frames of single-spectral-value images; 2) using a Deep Maching method to obtain spectral flow motion information of each frame of single-spectral-value image: one frame in each pair of adjacent frame monospectral value image is divided into sub-image blocks without overlap according to the minimum image size; obtaining the response of the other frame image in the pair of adjacent frame monochromatic value images to each sub-image block of the frame image; then splicing any four adjacent sub-image blocks to obtain different parent image blocks, and obtaining the response of the corresponding parent image block through the response of the sub-image; the response of the sub-image block refers to the similarity between each sub-image block in the other frame image and the sub-image block of the frame image, and the higher the similarity, the greater the response value; the sub-image block in the other frame image refers to any sub-image block with the same size as the sub-image block of the frame image in the other frame image; The obtained parent image block is taken as a higher level sub-image block, and the process in ② is repeated for iteration until the obtained parent image block is the original single spectral value image before division. The sub-image block obtained in ① is taken as a meta sub-image block. Each meta sub-image block is matched with the sub-image blocks containing the meta sub-image at each level. By searching from the top level to the bottom level for the correspondence of the local response maximum value of each meta sub-image block and the sub-image blocks containing the meta sub-image at each level, a matching vector field is obtained as a matching result. The local response maximum value is the maximum value in the response of each level sub-image block. The matching result is incorporated into an optimization framework of optical flow energy minimization to obtain spectral flow motion information of each frame monochromatic value image. 3) fusing the spectral flow motion information of all single-spectral-value images to obtain spectral motion features of the hyperspectral image to be classified, and then fusing the spectral motion features with original spectral information of the hyperspectral image to be classified to obtain spectral fusion features for classification.

2. The hyperspectral image classification method of claim 1, wherein, The response of the parent image block of one frame of single-spectral-value image in the adjacent frame single-spectral-value image pair is obtained by maximum similarity weighted average of the corresponding response of the child image block, specifically: ； wherein is a parent image block of the frame image is a response of the parent image block of the frame image is a response of each sub-image block of the frame image is a parent image block of the frame image having a size identical to a center of a parent image block of the frame image is another frame image of the pair of adjacent frame monochromatic value images; p is a center of a sub-image block of the frame image is a center of each sub-image block of the frame image is a set of , t is a number of iterations, M is a minimum image size; and p + S N，i represents a center position of a low layer response map, and S N，i represents a center position drift of a low layer response relative to a high layer response.

3. The method of claim 1, wherein, The optimization objective of the optimization framework of the light flow energy minimization is: ； wherein, is a spectral flow motion information; is a neighboring frame single-spectrum value image pair is a space, is a data item, is a smoothing term, is a matching term; Wherein, the data term is: ； The smoothing term is: ； The matching term is: ； wherein, is a matching vector field; b is a binomial term ; is a penalty term; is a weight coefficient; wherein is a tensor; is a normalized tensor; c is the total number of frames of the single spectral value image; and the penalty term wherein is a parameter substituted; and i are both constant terms; When minimizing, the optimization objective is minimized by using the optimization framework in the Brox light flow method.

4. The method of claim 3, wherein, 。 5. The method of claim 1, wherein, In each iteration process, after obtaining the response of the parent image block, the response is also subjected to nonlinear exponential transformation correction.

6. The method of claim 1, wherein, The value of the minimum image size is related to the size of the hyperspectral image to be classified and the required classification accuracy.

7. The method of claim 1, wherein, The hyperspectral image with a band number of n is continuously framed to obtain a plurality of single-spectral-value images with a number of n, and the spectral dimension is unfolded.

8. The method of claim 1, wherein, After obtaining the spectral fusion features for classification, classification is performed by using a support vector machine.

Images