Hyperspectral image classification method based on adaptive filter and feature complementary network

Abstract

The present invention relates to a hyperspectral image classification method based on an adaptive filter and a feature complementary network. The purpose of the present invention is to address the problem that existing methods fail to establish a good interdependence between short-range spectral information and long-range spectral information, and that the interference of redundant information on the network is not well resolved. The specific process of the hyperspectral image classification method based on an adaptive filter and a feature complementary network is as follows: 1. Establishing a feature complementary network model FCAN_AKF based on adaptive knowledge filtering; obtaining a trained feature complementary network model FCAN_AKF based on adaptive knowledge filtering; 2. Inputting the hyperspectral image to be tested into the trained feature complementary network model FCAN_AKF based on adaptive knowledge filtering to complete the classification of the hyperspectral image to be tested. The present invention is used in the field of hyperspectral image classification.

Description

Technical Field

[0001] The invention relates to a hyperspectral image classification method. Background Art

[0002] With the rapid development of hyperspectral remote sensing technology, hyperspectral images (HSIs) have been applied to more and more fields[1]-[2]. For example, coastal environmental monitoring[3]-[4], ecosystem protection[5], crop monitoring[6]-[8], medical diagnosis[9]-

[10] , etc. The premise of these applications is to accurately identify and classify the objects in hyperspectral images.

[0003] In the early days, the methods used for hyperspectral image classification were mainly classic machine learning methods. For example, some methods that use spectral information for classification include distance classifiers

[11] , maximum likelihood classifiers

[12] , and sparse representation classification (SRC)

[13] . Although these methods are simple in principle and easy to implement, they rely too much on manual features and prior information, resulting in poor generalization. In addition, using only spectral information for classification ignores the importance of spatial information in hyperspectral images, which also limits their classification effectiveness.

[0004] In the past decade, with the development of deep learning, convolutional neural networks (CNNs) have gradually become a research hotspot in the field of hyperspectral image classification

[14] -

[16] . CNNs can autonomously fit the features of hyperspectral images through training, avoiding the manual features of traditional methods and thus improving the generalization of the network. In

[17] , a deep convolutional network was used for hyperspectral image classification. Since each pixel of a hyperspectral image contains hundreds of spectral bands and the spectral features of pixels corresponding to different objects are quite different. Therefore, this method regards each pixel of the hyperspectral image as a one-dimensional speech signal input and uses a one-dimensional convolutional neural network (1D-CNN) to extract the spectral features of the hyperspectral image. Finally, this method achieves more competitive classification performance than the classical classifier. However, hyperspectral images are easily affected by external interference during imaging, and the spectral information between different objects is not completely separated in an ideal state. Therefore, hyperspectral image classification requires not only the extraction of spectral features, but also the extraction of spatial features. In

[18] , a three-dimensional convolutional neural network (3D-CNN) was proposed. This method uses 3D convolution to simultaneously extract the spatial and spectral features of HSI, which significantly improves the classification performance compared to 1D-CNN. However, early CNNs often improve the classification effect of HSI by directly stacking convolutional layers, which easily leads to overfitting of the network. To this end, a spectral-spatial residual network (SSRN) was proposed

[19] . SSRN constructs a network through a residual structure, which can better extract the deep features of HSI. In addition, in

[20] , a fast densely connected network was proposed, and the same features of HSI were reused through dense connections to avoid overfitting of the network.

[0005] Recently, in order to extract the important features of HSI, a series of attention mechanisms

[21] -

[23] have been applied to hyperspectral image classification (HSIC). In

[24] , two different branches are used to capture the spatial attention and spectral attention of HSI respectively, avoiding the mutual interference of spatial and spectral information. In

[25] , a double-branch dual-attention network (DBDA) was proposed. Similar to

[24] , DBDA also captures spatial and spectral attention through two different branches. The difference is that DBDA captures spatial and spectral features through adaptive self-attention and achieves better classification performance. Subsequently, a self-attention network using a network search strategy was proposed

[26] . This method uses a network framework search strategy to determine the optimal network structure to achieve the best classification results. In order to fully utilize the multi-scale features of hyperspectral images, a hierarchical residual attention network was proposed

[27] . However, due to the sensitivity of the convolution kernel to spatial rotation, the classification performance of conventional attention networks in rotated HSIs is poor. To this end, a rotation-invariant attention network (RIAN) was proposed

[28] . RIAN uses a rectified attention module to extract rotation-invariant spatial spectral attention, which effectively alleviates the problem of spatial rotation. In addition, to capture the bidirectional correlation of the internal spectra of hyperspectral images, an attention network based on bidirectional long short-term memory was proposed

[29] .

[0006] Although these methods have solved some problems, there are still some challenges in the application of CNNs in HSIC:

[0007] 1) The receptive field of CNNs is limited by the size of the convolution kernel. The limited receptive field makes it difficult to capture the interdependence between short-range and long-range spectral information.

[0008] 2) Although the attention mechanism can suppress some redundant information, there is still some redundant information in the features extracted by CNNs. This information will interfere with the network's judgment and limit the network's classification performance. Summary of the Invention

[0009] The purpose of the present invention is to solve the problem that the existing methods fail to establish the interdependence between short-range spectral information and long-range spectral information, and the interference of redundant information on the network is not well solved, and a hyperspectral image classification method based on adaptive filters and feature complementary networks is proposed.

[0010] The specific process of the hyperspectral image classification method based on adaptive filter and feature complementary network is as follows:

[0011] First, establish a feature complementary network model FCAN_AKF based on adaptive knowledge filtering;

[0012] Obtain the trained feature complementary network model FCAN_AKF based on adaptive knowledge filtering;

[0013] Second, the hyperspectral image to be tested is input into the trained feature complementary network model FCAN_AKF based on adaptive knowledge filtering to complete the classification of the hyperspectral image to be tested.

[0014] The beneficial effects of the present invention are:

[0015] In order to alleviate these two problems at the same time, in this invention, a feature complementary attention network based on adaptive knowledge filtering (FCAN_AKF) is proposed. First, to address the problem of limited receptive field of CNNs, a non-local band regrouping (NBR) strategy is designed. NBR realizes the interaction between close-range spectral information and long-range spectral information by reorganizing spectral information. In addition, two different branches are used to extract non-local features after NBR and local features of the original image, respectively, to achieve feature complementarity. Then, in order to suppress the interference of redundant information on the network, a dual pyramid spectral spatial attention (DPSSA) module is proposed to capture spectral spatial attention. Then, an adaptive knowledge filter (AKF) is designed to further remove redundant information and gain important feature information.

[0016] The main contributions of this invention include the following three parts:

[0017] To establish the interdependence between the near- and long-range spectral information of hyperspectral images within a limited receptive field, a non-local band regrouping (NBR) strategy is proposed. Furthermore, two branches are used to extract non-local features and local features, respectively, to achieve feature complementarity in hyperspectral images.

[0018] A dual pyramid spectral spatial attention (DPSSA) module is designed. DPSSA models spectral spatial attention in an autocorrelation manner and embeds multi-scale pooling pyramids in the attention to reduce information loss during feature extraction.

[0019] An adaptive knowledge filter (AKF) is designed. AKF can adaptively remove redundant information and enhance feature information that is beneficial to classification through network training iterations. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is the overall framework diagram of FCAN_AKF of the present invention;

[0021] Figure 2 Schematic diagram of spectral curves: (a) spectrum curve of IN dataset; (b) spectrum curve of SV dataset; (c) spectrum curve of UP dataset; (different curves correspond to spectra of different objects);

[0022] Figure 3 Spectral correlation analysis diagram, (a) IN dataset; (b) SV dataset; (c) UP dataset; (the spectral correlation degree gradually increases from dark to bright);

[0023] Figure 4 is a dual-pyramid spectral attention map; Figure 5 It is a dual-pyramid spatial attention map; Figure 6 This is a schematic diagram of AKF;

[0024] Figure 7 The effect diagram of the adaptive knowledge filter; (a) The overall classification accuracy corresponding to the use of different T and N in the knowledge filter on the IN dataset; (b) The adaptive process diagram of AKF on the IN dataset;

[0025] Figure 8 It is the detail diagram of IN dataset, NO is the number, LEGEND is the legend, NAME is the name, SAMPLE is the sample; Figure 9 Detailed diagram of the SV dataset; Figure 10 This is a detailed diagram of the UP dataset;

[0026] Figure 11 Graph showing the impact of using different training batches on the classification performance of the network at different learning rates; (a) Experimental results on the IN dataset; (b) Experimental results on the SV dataset; (c) Experimental results on the UP dataset.

[0027] Figure 12Graph showing the effect of the size of the network input patch on classification performance on different datasets (batch size is P×P);

[0028] Figure 13 Visualization of the classification results of different strategies on the IN dataset; (a) FCAN_AKF (basic network, i.e., without using NBR, AKF and DPSSA); (b) FCAN_AKF (using only the DPSSA module; (c) FCAN_AKF (using DPSSA and AKF); (d) complete FCAN_AKF.

[0029] Figure 14 Visualization of the classification results of different strategies on the SV dataset; (a) FCAN_AKF (basic network, i.e., without using NBR, AKF and DPSSA); (b) FCAN_AKF (using only the DPSSA module; (c) FCAN_AKF (using DPSSA and AKF); (d) complete FCAN_AKF.

[0030] Figure 15 Visualization of the classification results of different strategies on the UP dataset; (a) FCAN_AKF (basic network, i.e., without using NBR, AKF and DPSSA); (b) FCAN_AKF (using only the DPSSA module; (c) FCAN_AKF (using DPSSA and AKF); (d) complete FCAN_AKF.

[0031] Figure 16 Classification maps of different methods on the IN dataset; (a) real map; (b) DBMA; (c) DBDA; (d) HPDM-SPRN; (e) SSTN; (f) A2S2KResNet; (g) HResNetAM; (h) FECNet; (i) FCAN_AKF.

[0032] Figure 17 Classification maps of different methods on the SV dataset; (a) real map; (b) DBMA; (c) DBDA; (d) HPDM-SPRN; (e) SSTN; (f) A2S2KResNet; (g) HResNetAM; (h) FECNet; (i) FCAN_AKF.

[0033] Figure 18 Classification maps of different methods on the UP dataset; (a) real map; (b) DBMA; (c) DBDA; (d) HPDM-SPRN; (e) SSTN; (f) A2S2KResNet; (g) HResNetAM; (h) FECNet; (i) FCAN_AKF.

[0034] Figure 19 Classification accuracy diagrams of all methods under different training sample ratios; (a) Experimental results on the IN dataset; (b) Experimental results on the SV dataset; (c) Experimental results on the UP dataset. DETAILED DESCRIPTION

[0035] Specific embodiment 1: The specific process of the hyperspectral image classification method based on the adaptive filter and feature complementary network in this embodiment is as follows:

[0036] In recent years, convolutional neural networks (CNNs) have been widely used in hyperspectral image classification (HSIC). However, the scale of the convolution kernel in CNNs is fixed, which limits the network's receptive field and makes it difficult to capture the dependency of long-range feature information. In addition, during the HSIC process, the extracted features contain a large amount of redundant information, which will affect the final classification effect. In order to alleviate these problems, in the present invention, a feature complementary attention network based on adaptive knowledge filtering (FCAN_AKF) is proposed. Specifically, first, in order to alleviate the problem that CNN has difficulty in capturing the dependency between short-range spectral features and long-range spectral features due to its limited receptive field, a non-local band regrouping (NBR) strategy is designed. NBR enables CNN to capture non-local spectral features under a limited receptive field to establish the mutual dependency between short-range spectral features and long-range spectral features. In addition, in order to enable the proposed network to better perform classification, the present invention integrates the non-local features extracted after using NBR with the intrinsic local features of the original hyperspectral image to achieve the complementarity of non-local features and local features. Then, in order to eliminate the interference of redundant information on the network, a dual pyramid spectral-spatial attention (DPSSA) module is proposed and used to capture spectral-spatial attention. Then, an adaptive knowledge filter (AKF) is designed, which can adaptively further filter out redundant information that interferes with classification and gain feature information that is beneficial to classification. Finally, the present invention conducts extensive experiments on three challenging datasets, demonstrating that the proposed method is more competitive than some state-of-the-art hyperspectral image classification methods.

[0037] The overall framework of FCAN_AKF: The proposed FCAN_AKF network is as follows Figure 1 As shown in Figure 3 , FCAN_AKF takes patches as input. Therefore, the original HSIs are first divided into different patches, and the patches are input in batches of 64. Then, to enable the network to establish long-range relationships between spectral features within a limited receptive field, a non-localized bounding box (NBR) strategy is proposed. NBR first analyzes the correlation between spectral bands and groups them according to their correlation. Spectra within each group are then reorganized to enable the network to capture the interdependence between near- and long-range spectral information. Furthermore, to achieve complementarity between non-local and local features, two branches are designed to extract the non-local features after band reorganization and the local features of the original hyperspectral image, respectively. To simplify the network, the network architecture for extracting non-local features is identical to that for extracting local features. Specifically, this paper uses 3D convolution kernels of multiple scales to simultaneously extract spectral-spatial features, combined with a densely connected structure to reuse the same features and avoid overfitting. Furthermore, depthwise separable convolutions are embedded to reduce the number of network parameters. A pointwise convolution layer is then used to fuse the local and non-local features. The point-by-point convolution layer includes three consecutive operations: 2D point-by-point convolution, BatchNorm, and ReLU. Next, a DPSSA is used to model spectral-spatial attention. DPSSA includes dual pyramid spectral attention (DPSpe_A) and dual pyramid spatial attention (DPSpa_A). In general, DPSSA first captures spectral attention through DPSpe_A and then uses DPSpa_A to model spatial attention. To better integrate spectral-spatial attention, a residual structure is introduced into DPSSA. Subsequently, to further eliminate the interference of redundant information, an AKF is designed. AKF can iteratively cross-update parameters N and T through network training to adaptively filter out redundant information that interferes with network classification and gain information that is beneficial to classification. In particular, N and T here are the gain value and threshold of AKF, respectively. Finally, classification is performed through a fully connected layer.

[0038] 1. Establish a feature complementary attention network model based on adaptive knowledge filtering (FCAN_AKF);

[0039] Obtain the trained feature complementary network model FCAN_AKF based on adaptive knowledge filtering;

[0040] Second, the hyperspectral image to be tested is input into the trained feature complementary network model FCAN_AKF based on adaptive knowledge filtering to complete the classification of the hyperspectral image to be tested.

[0041] Specific embodiment 2: This embodiment differs from specific embodiment 1 in that: in the first embodiment, a feature complementary attention network model FCAN_AKF (feature complementary attention network based on adaptive knowledge filtering, FCAN_AKF) is established;

[0042] Obtain the trained feature complementary network model FCAN_AKF based on adaptive knowledge filtering;

[0043] The specific process is:

[0044] Step 1: Randomly sample hyperspectral images;

[0045] The hyperspectral image training set is represented as X∈R h×w×b , label Y∈R h×w ;

[0046] Where h, w, and b represent the height, width, and number of spectral bands of the hyperspectral image HSI, respectively, and R represents a real number;

[0047] The training set is randomly selected from each category according to a preset ratio;

[0048] Step 2: Establish a feature complementary network model FCAN_AKF based on adaptive knowledge filtering;

[0049] Obtain the trained feature complementary network model FCAN_AKF based on adaptive knowledge filtering.

[0050] Other steps and parameters are the same as those in the first embodiment.

[0051] Specific embodiment three: This embodiment differs from specific embodiment one or two in that: in step two, a feature complementary network model FCAN_AKF based on adaptive knowledge filtering is established;

[0052] Obtain the trained feature complementary network model FCAN_AKF based on adaptive knowledge filtering;

[0053] The specific process is:

[0054] The feature complementation network model FCAN_AKF based on adaptive knowledge filtering includes: non-local band regrouping (NBR) strategy module, non-local feature extraction module, local feature extraction module, feature complementation fusion module, dual pyramid spectral spatial attention (DPSSA) module, adaptive knowledge filter (AKF) and classification layer;

[0055] Step 2.1 Divide the original hyperspectral image HSIs into different patches x∈R h×w×b ;

[0056] Step 2. Patch x∈R h×w×b The non-local band regrouping (NBR) strategy module NBR of the feature complementation network model FCAN_AKF based on adaptive knowledge filtering is input with a batch size of 64 (64 patches as a batch, so the input is batch by batch). The non-local band regrouping strategy module NBR first analyzes the correlation between spectral bands and groups them according to the correlation. Then, the spectra in each group are reorganized so that the network can capture the interdependence between the short-range spectral information and the long-range spectral information.

[0057] Step 2 and 3: Input the NBR output x′ into the non-local feature extraction module and output the non-local features;

[0058] Step 24: Patch x∈R h×w×b The local feature extraction module of the feature complementation network model FCAN_AKF based on adaptive knowledge filtering is input with a batch size of 64, and the local features are output;

[0059] Step 25: Concatenate the non-local features obtained in step 23 and the local features obtained in step 24, input the concatenated features into a feature complementary fusion module, and output the fused features;

[0060] Step 26: Input the fused features into the dual pyramid spectral spatial attention (DPSSA) module, and the dual pyramid spectral spatial attention module outputs the spectral spatial attention;

[0061] Step 27: Input the spectral spatial attention output of the dual pyramid spectral spatial attention module into the adaptive knowledge filter (AKF), and the adaptive knowledge filter outputs features;

[0062] Step 28: Input the output features of the adaptive knowledge filter into the classification layer for classification;

[0063] Step 29: Obtain the trained feature complementary network model FCAN_AKF based on adaptive knowledge filtering.

[0064] Other steps and parameters are the same as those in the first or second embodiment.

[0065] Specific embodiment 4: This embodiment differs from any one of specific embodiments 1 to 3 in that: in step 22, the patch x∈R h×w×b The non-local band regrouping (NBR) strategy module NBR of the feature complementation network model FCAN_AKF based on adaptive knowledge filtering is input with a batch size of 64. The non-local band regrouping (NBR) strategy module NBR first analyzes the correlation between spectral bands and groups them according to the correlation. Then, the spectra in each group are reorganized so that the network can capture the interdependence between the short-range spectral information and the long-range spectral information. The specific process is as follows:

[0066] B. Non-local band recombination strategy

[0067] In the research field of hyperspectral image classification, the extraction of spectral information is crucial. Figure 2 As shown in the figure, hyperspectral images are interfered by some external factors during imaging, so the spectral curves corresponding to different objects in the hyperspectral images are not completely separated. These overlapping spectral information will seriously interfere with the classification of the network. To this end, this paper alleviates the interference of spectral redundant information by establishing the dependency relationship between short-range and long-range spectral information. However, the receptive field of CNNs is limited by the size of the convolution kernel, making it difficult to establish long-range dependency relationships of spectral information. Figure 2 As shown in Figure 2, assuming that the bands in the boxes connected by dotted lines need to establish dependencies, it is difficult to achieve using existing methods. Therefore, in this paper, a non-local band regrouping (NBR) strategy is proposed.

[0068] Specifically, the original bands are first grouped according to the correlation between the bands. In this paper, a simple and effective spectral grouping method

[30] -

[32] is used. Figure 3As shown in the figure, each yellow box in different datasets is a group. Specifically, the spectral bands of the Indian Pines (IN) dataset are divided into three groups (1-35 in one group, 36-104 in one group, and 105-200 in one group); the spectral bands of the Salinas Valley (SV) dataset are also divided into three groups (1-40 in one group, 41-104 in one group, and 105-204 in one group); and the spectral bands of the Pavia University (UP) dataset are divided into two groups (1-40 in one group and 41-104 in one group).

[0069] Step 221: Group the spectral bands of the patch according to the correlation between the bands; the specific process is:

[0070] The process of spectral correlation analysis can be expressed as

[0071]

[0072] Where C(i,j) represents the correlation between the i-th band and the j-th band, Cov(·) represents the covariance operation, Cov(i,j) represents the covariance operation between the i-th band and the j-th band, Cov(i,i) represents the covariance operation between the i-th band and the i-th band, Cov(j,j) represents the covariance operation between the j-th band and the j-th band, and · represents the dot product;

[0073] Step 222: Reorganize the spectra within each group. The specific process is as follows:

[0074] Each group is evenly divided into B1, B2, and B3 again. Then, non-adjacent band groups can be inserted between adjacent band groups, that is, B1 is inserted between the adjacent B2 and B3 groups. In this way, CNNs can establish non-local spectral band dependencies even with a limited receptive field.

[0075] The process is described as

[0076]

[0077] x′=NBR(B1,B2,B3) (3)

[0078] Among them, Slice -1 (·) represents the spectrum slicing operation; x C is the output of correlation grouping; is a rounding function, and b represents the number of spectral bands; n represents the number of spectral recombined groups; B nis the result of grouping; NBR(·) represents the non-local band splicing operation, which is to insert B1 between the two adjacent groups B2 and B3, and x′ represents the output of NBR (the output after spectral recombination).

[0079] The other steps and parameters are the same as those in the first to third embodiments.

[0080] Specific embodiment 5: This embodiment differs from any one of specific embodiments 1 to 4 in that: in steps 2 and 3, the output x′ of NBR is input into the non-local feature extraction module to output the non-local features; the specific process is as follows:

[0081] The non-local feature extraction module includes submodule 1, submodule 2, submodule 3, depthwise separable convolution submodule, submodule 5, and submodule 6;

[0082] Submodule 1 includes 3D convolution (1, 1, 9), BN (3d), and ReLU in sequence;

[0083] Submodule 2 includes 3D convolution (1, 1, 7), BN (3d), and ReLU in sequence;

[0084] Submodule 3 includes 3D convolution (1, 1, 5), BN (3d), and ReLU in sequence;

[0085] The depthwise separable convolution submodule includes three-dimensional depth convolution (3D Depthwise convolution) (1,1,3), BN (3d), Relu, three-dimensional pointwise convolution (3D Pointwise convolution) (1,1,1), BN (3d), Relu in sequence;

[0086] Submodule 5 includes 3D pointwise convolution (1,1,1), BN (3d), and ReLU in sequence;

[0087] Submodule 6 includes 3D pointwise convolution (1,1,1), BN (3d), ReLU, deformation and compression operations (reshape & squeeze) in sequence;

[0088] x′ passes through submodule 1 to obtain feature 1;

[0089] Feature 1 passes through submodule 2 to obtain feature 2;

[0090] Feature 1 and feature 2 are combined into feature 3 through submodule 3;

[0091] Feature 1, feature 2 and feature 3 are processed through the depth-wise separable convolution submodule to obtain feature 4;

[0092] Feature 1, feature 2, feature 3 and feature 4 are processed through submodule 5 to obtain feature 5;

[0093] Feature 5 passes through submodule 6 to obtain feature 6.

[0094] Other steps and parameters are the same as those in Specific Embodiments 1 to 4-1.

[0095] Specific embodiment 6: This embodiment differs from any one of the specific embodiments 1 to 5 in that: in step 24, the patch x∈R h×w×b The local feature extraction module of the feature complementation network model FCAN_AKF based on adaptive knowledge filtering is input with a batch size of 64, and the local features are output; the specific process is as follows:

[0096] The local feature extraction module includes submodule 1, submodule 2, submodule 3, depthwise separable convolution submodule, submodule 5, and submodule 6;

[0097] Submodule 1 includes 3D convolution (1, 1, 9), BN (3d), and ReLU in sequence;

[0098] Submodule 2 includes 3D convolution (1, 1, 7), BN (3d), and ReLU in sequence;

[0099] Submodule 3 includes 3D convolution (1, 1, 5), BN (3d), and ReLU in sequence;

[0100] The depthwise separable convolution submodule includes three-dimensional depth convolution (3D Depthwise convolution) (1,1,3), BN (3d), Relu, three-dimensional pointwise convolution (3D Pointwise convolution) (1,1,1), BN (3d), Relu in sequence;

[0101] Submodule 5 includes 3D pointwise convolution (1,1,1), BN (3d), and ReLU in sequence;

[0102] Submodule 6 includes 3D pointwise convolution (1,1,1), BN (3d), ReLU, deformation and compression operations (reshape & squeeze) in sequence;

[0103] x passes through submodule 1 to obtain feature A;

[0104] Feature A is converted into feature B through submodule 2;

[0105] Feature A and feature B are converted into feature C through submodule 3;

[0106] Feature A, feature B and feature C are processed through the depth-wise separable convolution submodule to obtain feature D;

[0107] Feature A, feature B, feature C and feature D are processed through submodule 5 to obtain feature E;

[0108] Feature E is converted into feature F through submodule 6.

[0109] Other steps and parameters are the same as those in Specific Implementations 1 to 5-1.

[0110] Specific embodiment seven: This embodiment differs from any one of specific embodiments one to six in that: in step 25, the non-local features obtained in step 23 and the local features obtained in step 24 are spliced, and the spliced ​​features are input into a feature complementary fusion module, which outputs the fused features; the specific process is as follows:

[0111] The feature 6 obtained in step 2 and 3 and the feature F obtained in step 2 and 4 are concatenated and input into the fully connected layer. The output of the fully connected layer is sequentially input into the two-dimensional pointwise convolution (2D Pointwise convolution) (1,1), BN (2d), and Relu, and the fused feature is output.

[0112] The other steps and parameters are the same as those in the first to sixth embodiments.

[0113] Specific embodiment eight: This embodiment differs from any one of specific embodiments one to seven in that: in step twenty-six, the fused features are input into a dual pyramid spectral spatial attention (DPSSA) module, and the dual pyramid spectral spatial attention module outputs spectral spatial attention; the specific process is as follows:

[0114] Dual Pyramid Spectral Spatial Attention: The attention mechanism is a strategy developed based on human vision. The attention mechanism enables the network to focus on important features that are beneficial to classification and suppress irrelevant features that interfere with classification. In other words, the attention mechanism can alleviate the interference of redundant information and effectively improve the classification performance of the network. In our previous work

[34] , an effective multi-scale dual-aggregation attention network has been proposed. Although

[33] achieved effective classification performance, the computational complexity of the network is relatively high. To this end, in this paper, a dual pyramid spectral spatial attention (DPSSA) is designed.

[0115] The DPSSA includes a dual pyramid spectral attention module (DPSpe_A) and a dual pyramid spatial attention module (DPSpa_A);

[0116] The fused features are input into the dual pyramid spectral attention module (DPSpe_A), which outputs spectral attention.

[0117] The dual pyramid spectral attention module outputs spectral attention and inputs it into the dual pyramid spatial attention module (DPSpa_A). The dual pyramid spatial attention module outputs spatial attention.

[0118] The spectral attention and spatial attention are summed to obtain the spectral spatial attention;

[0119] The fused features are input into the dual pyramid spectralattention module (DPSpe_A), which outputs spectral attention. The specific process is as follows:

[0120] like Figure 4 and Figure 5 As shown in Figure 3, DPSSA consists of dual-pyramid spectral attention and dual-pyramid spatial attention. Overall, DPSSA first models the spectral attention using DPSpe_A and then captures the spatial attention using DPSpa_A. Specifically, skip connections are used in DPSpe_A to reduce information loss, facilitating the subsequent capture of spatial attention. Furthermore, to better integrate spectral and spatial attention, a residual structure is employed between DPSpe_A and DPSpa_A.

[0121] In the dual-pyramid spectral attention module DPSpe_A, the key tensor K, query tensor Q, and value tensor V are obtained by linear transformation of the input; this process is described as

[0122]

[0123]

[0124]

[0125] Where b, h, and w represent the number of input bands, height, and width respectively; W K 、W Q and W V Represents the operation of the linear transformation layer; r(x in ) is to input x in The spatial dimension of is transformed from h×w to N, that is, N=h×w; () Γ represents transpose, · represents dot product;

[0126] Then, different from

[33] , in order to reduce the computational complexity of the network, dual pyramid pooling is used to obtain the attention mask and model the high-level features V′ respectively. In particular, in order to avoid the loss of key information, dual pyramid pooling uses a multi-scale pooling method. This process can be described as

[0127] K′=P1(K)||P2(K)||P4(K)||P8(K) (7)

[0128] V′=P1(V)||P2(V)||P4(V)||P8(V) (8)

[0129] Where P1, P2, P4, and P8 represent adaptive average pooling with scales of 1, 2, 4, and 8, respectively; || represents the concatenation operation; K′ represents the key tensor after adaptive average pooling; V′ represents the value tensor after adaptive average pooling;

[0130] Next, K′ is dot-multiplied with Q to obtain the spectral attention mask. Finally, the spectral attention mask is used as the weighted high-level feature V′. In addition, skip connections are used in the final stage to prevent the network from overfitting. This process can be expressed as

[0131] x spe =relu(r(x in )+sf(K′·Q)·V′) (9)

[0132] Among them, sf(·) represents the softmax function; relu(·) is the activation function; x spe is the spectral attention, · represents the dot product;

[0133] In terms of network computational complexity, DPSpe_A mainly relies on dual pyramid pooling to reduce the computational complexity of the network. The computational complexity of DPSpe_A can be expressed as O=o1(b×N 2 ×15)+o2(N×b×15 2 ) (10)

[0134] The computational complexity of not using double pyramid pooling can be expressed as O′=o1(b 2 ×N 2 )+o2(N 3 ×b)(11)

[0135] Where o1 is the complexity of computing the spectral attention mask. o2 is the complexity of computing the spectral attention. In comparison, the computational complexity of DPSpe_A is significantly reduced. Specifically, the computational complexity of the attention mask is reduced by a factor of b / 15, and the complexity of the attention is reduced by a factor of b. 2 / 15 times.

[0136] The dual-pyramid spectral attention module outputs spectral attention and inputs it into the dual-pyramid spatial attention module, and the dual-pyramid spatial attention module outputs spatial attention. The specific process is:

[0137] In the dual-pyramid spatial attention module DPSpa_A, the key tensor k, query tensor q, and value tensor v are obtained by linear transformation of the input; this process can be described as

[0138]

[0139]

[0140]

[0141] Where b, h, and w represent the number of input bands, height, and width respectively; w k 、w q and w v Represents the operation of the linear transformation layer; r(x spe ) is to input x spe The spatial dimension of is transformed from h×w to N, that is, N=h×w; () Γ represents transpose, · represents dot product;

[0142] Then, different from

[33] , in order to reduce the computational complexity of the network, dual pyramid pooling is used to obtain the attention mask and model the high-level features v′ respectively. In particular, in order to avoid the loss of key information, dual pyramid pooling uses a multi-scale pooling method. This process can be described as

[0143] k′=P1(k)||P2(k)||P4(k)||P8(k)

[0144] v′=P1(v)||P2(v)||P4(v)||P8(v)

[0145] Where P1, P2, P4, and P8 represent adaptive average pooling of scales 1, 2, 4, and 8, respectively; || represents the concatenation operation; k′ represents the key tensor after adaptive average pooling; v′ represents the value tensor after adaptive average pooling;

[0146] Next, k′ is dot-multiplied with Q to obtain the spectral attention mask. Finally, the spectral attention mask is used as the weighted high-level feature v′. In addition, skip connections are used in the final stage to prevent the network from overfitting. This process can be expressed as

[0147] x′ spe =relu(r(x spe )+sf(k′·q)·v′)

[0148] Among them, sf(·) represents the softmax function; relu(·) is the activation function; x′ spe is the final spectral space attention, and represents the dot product.

[0149] In this paper, to simplify the network structure, the settings of DPSpa_A and DPSpe_A are similar. The difference is that the double pyramid pooling layer in DPSpa_A operates on the spatial dimension, while the double pyramid pooling layer in DPSpe_A operates on the spectral dimension.

[0150] Other steps and parameters are the same as those in Specific Embodiments 1 to 7-1.

[0151] Specific embodiment 9: This embodiment differs from any one of specific embodiments 1 to 8 in that: in step 27, the spectral spatial attention output of the double pyramid spectral spatial attention module is input into the adaptive knowledge filter (AKF), and the adaptive knowledge filter outputs features; the specific process is:

[0152] Adaptive knowledge filter: In order to alleviate the interference of redundant information on the network, the attention mechanism has provided an effective solution. However, the attention mechanism relies on the feature extraction capability of the network and cannot directly remove redundant information. Recently, a feature filter

[34] has been used to eliminate redundant information. Although the feature filter proposed in

[34] has achieved some results in the application of hyperspectral image classification, the method has not been adaptive. This requires the feature filter to manually adjust the parameters to adapt to different data sets, and it is also difficult to be integrated into other networks. In addition, the feature filter will change all the eigenvalues ​​greater than the threshold to 1, which will increase the risk of redundant information that is not completely filtered interfering with the network. In the present invention, an adaptive knowledge filter (AKF) is proposed to further alleviate the interference of redundant information on the network. As Figure 6 As shown, the threshold T and gain value N are first set for the proposed knowledge filter.

[0153] Specifically, the feature value of the input filter is represented as x DPSSA ; First, use the sigmiod function to calculate x DPSSA Normalize to obtain the eigenvalue X, and then input the eigenvalue X into the knowledge filter;

[0154] When the eigenvalue X of the input knowledge filter is greater than or equal to T, the input eigenvalue X will be increased, that is, the eigenvalue X becomes N*X; conversely, when the eigenvalue X of the input knowledge filter is less than T, the input eigenvalue X will be set to 0; the process is described as

[0155] X = sigmiod(x DPSSA )

[0156]

[0157] Where N is the gain value, T is the threshold, and * is the multiplication sign;

[0158] The filter output feature value and the double pyramid spectral spatial attention module output spectral spatial attention are added together as the final adaptive knowledge filter output feature.

[0159] The other steps and parameters are the same as those in the specific implementation modes 1 to 8-1.

[0160] Specific embodiment ten: This embodiment differs from any one of specific embodiments one to nine in that: in step twenty-nine, a trained feature complementary network model FCAN_AKF based on adaptive knowledge filtering is obtained; the specific process is as follows:

[0161] Step 291: Initialize the network, i.e. set the flag of adaptive knowledge filtering to 0;

[0162] Update the flag bit NFlag of N to 1, and update the flag bit TFlag of T to 0;

[0163] The N update step size S and T update step size s′ are set to 1 and 0.1 respectively;

[0164] The flag bit of adaptive knowledge filtering indicates the state of adaptive filtering, that is, when the flag bit of adaptive knowledge filtering is 0, adaptive knowledge filtering is stopped; when the flag bit of adaptive knowledge filtering is 1, adaptive knowledge filtering is enabled; the flag bit NFlag of N indicates the update state of N, that is, when NFlag is 0, the update of N is stopped; when NFlag is 1, the update of N is enabled; the flag bit TFlag of T indicates the update state of T, that is, when TFlag is 0, the update of T is stopped; when TFlag is 1, the update of T is enabled;

[0165] Step 2912: the current prediction accuracy ROA and the best prediction accuracy BOA of the feature complementary network model FCAN_AKF based on adaptive knowledge filtering are initialized to 0;

[0166] Step 2913: Start training the feature complementary network model FCAN_AKF based on adaptive knowledge filtering and save the training parameters;

[0167] Step 2914: Perform verification based on the feature complementary network model FCAN_AKF based on adaptive knowledge filtering, calculate the loss value and save the current prediction accuracy ROA;

[0168] Determine whether the loss value of the network model FCAN_AKF decreases 10 times in a row. If the loss value of the network model FCAN_AKF does not decrease 10 times in a row, that is, the adaptive knowledge filtering flag Flag = 0, execute step 2913 (the network model FCAN_AKF will return to the training stage) until the maximum number of training times is reached. If the loss value of the network model FCAN_AKF decreases 10 times in a row, that is, the adaptive knowledge filtering flag Flag = 1, the network model FCAN_AKF will execute the update of N and T, execute step 2913 (the network model FCAN_AKF will return to the training stage) until the maximum number of training times is reached. The specific process is as follows:

[0169] When performing the update of N and T, the network model FCAN_AKF will first determine the status of NFlag;

[0170] When NFlag is 1, the network model FCAN_AKF compares the current prediction accuracy with the best prediction accuracy. If the current prediction accuracy is greater than the best prediction accuracy, the current prediction accuracy is assigned to the best prediction accuracy, and NFlag and TFlag are set to 0 and 1 respectively. N forward updates are performed, that is, N = N + S. If the current prediction accuracy is less than or equal to the best prediction accuracy, TFlag is set to 1 and N reverse updates are performed, that is, N = N - 2S.

[0171] When NFlag is 0, the network model FCAN_AKF will determine the status of TFlag; when TFlag is 1, the network model FCAN_AKF will compare the current prediction accuracy with the best prediction accuracy; if the current prediction accuracy is greater than the best prediction accuracy, the current prediction accuracy will be assigned to the best prediction accuracy, and TFlag and NFlag will be set to 0 and 1 respectively; the forward update of T will be executed, that is, T=T+s′; if the current prediction accuracy is less than or equal to the best prediction accuracy, NFlag will be set to 1 and the reverse update of T will be executed, that is, T=T-2s′.

[0172] Subsequently, in order to improve the generalization of the network and enable the knowledge filter to be widely used in hyperspectral image classification, this paper proposes an adaptive knowledge filter. The working process of the adaptive knowledge filter is shown in Algorithm 1.

[0173]

[0174] In Algorithm 1, Flag, NFlag, and TFlag represent the flags for executing adaptive knowledge filtering, updating N, and updating T, respectively. Specifically, when Flag = 10, the knowledge filter begins adapting. When NFlag = 1, the gain N is updated. When TFlag = 1, the threshold is updated. BOA and ROA represent the model's best prediction accuracy and the network's current prediction accuracy, respectively. S and s represent the update step size for N and T, respectively. Epoch represents the network training epoch. Generally speaking, AKF adaptation is updated through network training iterations. Specifically, the network saves parameters during training and then uses these parameters for validation. During validation, the network's current loss is calculated and its prediction accuracy is saved. The network then determines whether it has pre-converged (validation loss increases 10 times consecutively, i.e., Flag = 10). When the network reaches pre-convergence, the AKF begins updating the N and T values. This process is repeated until network training is complete. In particular, in order to avoid the local optimum of the network's classification performance, a method of cross-updating N and T is designed. In addition, the update step size of N and T values ​​in AKF is adjustable.

[0175] like Figure 7 As shown in (b), the adaptive update process of N and T in AKF is demonstrated on the IN dataset. Among them, the three guiding lines are the adaptive processes of the network under different initial N and T values. Figure 7 As shown in (b), regardless of the initial values ​​of N and T, the network achieves optimal classification performance through adaptive AKF. Taking the red guide line as an example, the classification accuracy corresponding to the initial N and T (i.e., the classification accuracy corresponding to the network when N = 1 and T = 0.1) is shown. AKF then filters out redundant information through repeated network training and cross-updating N and T, ultimately achieving optimal classification performance.

[0176] In summary, the proposed AKF can effectively alleviate the interference of redundant information on the network. In addition, compared with the feature filter of

[35] , the knowledge filter proposed in this invention is adaptive. And for features greater than the threshold, AKF uses a linear gain method to enhance the features, which ensures that there are still differences between the features after gain. Through the adaptive method of AKF, redundant information can be better filtered out.

[0177] The following examples are used to verify the beneficial effects of the present invention:

[0178] Example 1:

[0179] Experiments and analysis: In order to verify the effectiveness of the proposed method, experiments and analysis are carried out on the method proposed in this part. First, three challenging datasets are introduced in detail. Then, the hyperparameter setting details of the network are given. Next, the ablation experiment is used to analyze the effectiveness of the module proposed in this invention. In addition, the classification performance of the proposed network is analyzed in detail. Finally, the present invention proves that the proposed method is more competitive than some state-of-the-art methods. In particular, in order to avoid the contingency of the experiment, the results of all experiments are the average results of 10 repeated experiments. In addition, all experiments are carried out in the same experimental environment. Specifically, the experiment is equipped with NVIDIA GeForceRTX 3070, and the compilation software is pycham 2020. The running environment of the code is pytorch 1.10.0 and python 3.7.12.

[0180] A. HSI Dataset and Classification Performance Evaluation Indicators

[0181] The present invention verifies the classification performance of the proposed method on three challenging data. The three datasets are Indian Pines (IN), Salinas Valley (SV) and Pavia University (UP). IN has 200 continuous spectral bands, 16 categories and 10,249 annotated pixels. SV has 204 continuous bands, also contains 16 ground feature categories, and these ground features cover 54,129 pixels. UP has 104 bands. And contains 9 categories. The number of pixels covered by its ground features is 42,776. The detailed information of the three datasets is in Figures 8-10 In this paper, all experiments use a uniform ratio of training samples and test samples. Specifically, 3% of the training samples are used on the IN dataset. 1% of the training samples are used on both the SV dataset and the UP dataset. Figures 8-10 The number of training samples and test samples in each category is given in . In addition, in order to quantitatively evaluate the classification performance of the network, this paper uses three conventional evaluation indicators, including overall classification accuracy (OA), average accuracy (AA), and KAPPA coefficient.

[0182] B. Hyperparameter Setting: In CNNs, the impact of hyperparameters on model training cannot be ignored. To achieve optimal classification performance, appropriate hyperparameters need to be set for the network. First, the epoch number of FCAN_AKF is set to 400. Figure 11 The effects of learning rate and training batch on the classification performance of the proposed network are given on different datasets. Figure 11 As shown in (a), on the IN dataset, when the learning rate of the proposed network is determined, the classification performance of the network first increases and then decreases with the increase of the batch size of the network input. And when the input batch size is 64, the best classification performance is achieved. Similarly, when the input batch size is determined, the classification performance of the network first increases and then decreases with the increase of the learning rate. And when the learning rate is 5e -4 , to achieve the best network performance. Figure 11 As shown in (a) and (b), the effects of learning rate and training batch size on the classification performance of the proposed network on the SV and UP datasets are similar to those on the IN dataset. Therefore, the learning rate and input batch size of FCAN_AKF are set to 5e respectively. -4 and 64.

[0183] In addition, the size of the input patch also has a significant impact on the classification performance of CNNs. Figure 12 In this paper, we explored the impact of input patch size on the classification performance of FCAN_AKF on various datasets. Overall, across all three datasets, as the input patch size increases, the network's classification performance initially increases and then decreases. Furthermore, the proposed network achieves optimal classification performance when the patch size is 9×9. Therefore, the input patch size of FCAN_AKF is set to 9×9.

[0184] C. Effectiveness analysis of the proposed module: In order to verify the effectiveness of the proposed module, this part conducts an ablation experiment on the proposed module. As shown in Table 1, 8 groups of experiments are conducted on each dataset. First, by comparing Experiment 1 and Experiment 2 FCAN_AKF (only using, it can be concluded that DPSSA can effectively improve the classification accuracy of the network on three different datasets. Specifically, on the IN dataset, the OA of FCAN_AKF using DPSSA is improved by 0.5% compared with not using DPSSA. Similarly, on the SV dataset, OA is improved by 0.45%. On the UP dataset, OA is improved by 0.31%. DPSSA can effectively improve the classification performance of the network because it can enhance important features through attention and effectively suppress unimportant features. In addition, DPSSA reduces the loss of information through multi-scale pyramid pooling. Subsequently, by comparing Experiment 1 and Experiment 4, the effectiveness of the NBR strategy is also verified. Obviously, on the IN dataset, the OA of FCAN_AKF using the NBR strategy is improved by 0.34% compared with the one without the NBR strategy. . Similarly, on the other two data sets, the classification accuracy of FCAN_AKF using NBR strategy has been improved to varying degrees compared with that without NBR strategy. This is because NBR enables the network to capture non-local spectral features by reorganizing the original spectrum, thereby establishing a mutual dependence between short-range spectral information and long-range spectral information. Not only that, the proposed FCAN_AKF combines the non-local features after spectral reorganization with the local features of the original hyperspectral image, realizing feature fusion and complementarity. Therefore, NBR can effectively improve the classification performance of the network. Finally, the effectiveness of AKF is also verified. Experiment 1 is compared with Experiment 3. The results show that compared with FCAN_AKF without AKF, the classification performance of FCAN_AKF with AKF on the three data sets has been significantly improved. This is because AKF can adaptively remove redundant information and gain important feature information.

[0185] In addition, to verify whether NBR, AKF, and DPSSA interfere with each other in FCAN_AKF, we compared Experiment 5 with Experiment 8, Experiment 6 with Experiment 8, and Experiment 7 with Experiment 8. Clearly, NBR, AKF, and DPSSA do not interfere with each other in FCAN_AKF and can effectively improve the classification accuracy of the network under all circumstances.

[0186] Table 1 Ablation experiment of the proposed module (√ indicates that the corresponding module is used, — indicates that the corresponding module is not used)

[0187]

[0188] 1 is FCAN_AKF (basic network, i.e., not using NBR, AKF, and DPSSA); 2 is FCAN_AKF (using only the DPSSA module); 3 is FCAN_AKF (using only the AKF module); 4 is FCAN_AKF (using only the NBR strategy); 5 is FCAN_AKF (using DPSSA and AKF); 6 is FCAN_AKF (using NBR and DPSSA); 7 is FCAN_AKF (using NBR and AKF); 8 is the complete FCAN_AKF;

[0189] In order to more intuitively analyze the impact of the proposed module on network performance, this paper conducts a step-by-step incremental analysis of the proposed module on each dataset. Specifically, Figure 13 The visualization of the classification results using different strategies on the IN dataset is given. Figure 13 (a) is a visualization of the classification results of the base network (i.e., FCAN_AKF with only NBR, AKF, and DPSSA modules removed). Figure 13 (b) is a visualization of the classification results obtained after adding the DPSSA module to the basic network. Obviously, Figure 13 (b) and Figure 13 Compared with (a), the same categories are more clustered and different categories are more dispersed. This shows that the DPSSA module can effectively improve the classification performance of the network. Figure 13 (c) is in Figure 13 AKF is added to the basis of (b). Similarly, (c) in 13 and Figure 13 Compared with (b), the classification performance of the network has been improved again. Figure 13 (d) is a visualization of the classification results obtained by the complete FCAN_AKF network. Compared with the previous strategies, the complete FCAN_AKF achieved the best classification performance. In addition, Figure 14-15 , the same results can be obtained on the SV dataset and the UP dataset. This proves that the three proposed modules do not interfere with each other, and their impact on network performance shows an incremental improvement as they are added.

[0190] D. Comparison of different methods: To further verify the effectiveness of the proposed network, this section compares the proposed FCAN_AKF with seven CNN-based hyperspectral image classification methods. These seven methods include: DBMA

[24] , DBDA

[25] , SSTN

[26] , HResNetAM

[27] , HPDM-SPRN

[35] , A2S2KresNet

[36] , and FECNet

[37] . Tables 2-4 show the classification results and parameter counts of different methods on the three datasets.

[0191] First, overall, the proposed FCAN_AKF achieves the best classification performance on all three datasets. Furthermore, FCAN_AKF exhibits significant advantages over other methods in terms of overall network parameter count. Specifically, Table 2 presents the classification results and parameter counts for all methods on the IN dataset. The proposed FCAN_AKF exhibits significant advantages over other methods in terms of the OA, AA, and KAPPA coefficients. In particular, the OA and KAPPA coefficients are 1%-5% higher than those of other methods. This demonstrates not only the significant overall classification performance of the proposed method, but also its ability to classify each class in a balanced manner, achieving good classification consistency. This is due in part to the complementary integration of local and non-local features, and in part to the fact that FCAN_AKF, with the help of DPSSA and AKF, effectively removes redundant information that interferes with classification. In terms of overall network parameter count, the proposed method also exhibits significant advantages on the IN dataset. Specifically, the proposed FCAN_AKF has only 59.382k parameters, approximately 6k-700k fewer than other methods. This is due to its simple network architecture, which does not introduce additional parameters. Furthermore, FCAN_AKF uses depthwise separable convolutions for feature extraction, significantly reducing the network's parameter count without compromising classification performance.

[0192] Table 3 shows the classification results and parameter counts for all methods on the SV dataset. Compared to the IN dataset, the SV dataset has a richer amount of data available for training. Therefore, all methods achieve good classification accuracy on the SV dataset. Despite this, the proposed method still demonstrates more competitive classification results than other methods. Similarly, as shown in Table 4, FCAN_AKF also has a significant advantage over other methods in classification results on the UP dataset. This further demonstrates that the proposed method has significant classification performance and good generalization ability.

[0193] Table 2 Classification results of different methods on the IN dataset.

[0194]

[0195]

[0196] Table 3 Classification results of different methods on the SV dataset.

[0197]

[0198] Table 4 Classification results of different methods on the UP dataset.

[0199]

[0200]

[0201] In order to more intuitively compare the classification performance of different methods, this paper visualizes the classification results of all methods. Figure 16-Figure 18 shown. Figure 16 The classification maps of all methods on the IN dataset are given. Figure 16 As can be seen, among all methods, the classification maps of DBMA and DBDA perform poorly compared to the other methods. This is because these two methods simply combine spatial spectral attention and fail to fully extract image features. The classification maps of the other methods, on the other hand, all have relatively clear category boundaries. In particular, compared to the classification maps of the other methods, the classification map of FCAN_AKF not only has clear category boundaries but also better distinguishes different categories. This is due to the fact that AKF can eliminate redundant information in FCAN_AKF, avoiding mutual interference between different categories. Furthermore, similar conclusions are drawn on the SV and UP datasets as on the IN dataset.

[0202] In supervised classification tasks, the number of training samples determines the amount of prior information available for classification. Therefore, the small sample size problem remains a major challenge in hyperspectral image classification. Considering the importance of training sample size in supervised classification networks, this paper analyzes the classification performance of the proposed method under different training sample sizes. Specifically, the proposed FCAN_AKF is compared with seven other methods. Figure 19 The classification accuracy of all methods using different proportions of training samples on three data sets is given. Figure 19 As shown in Figure (a), on the IN dataset, the classification accuracy of all methods shows an upward trend with increasing sample size. Furthermore, the proposed FCAN_AKF achieves the best classification accuracy compared to other methods at varying training sample ratios. In particular, the superior classification performance of FCAN_AKF becomes more pronounced with fewer training samples. This demonstrates that, compared to other methods, the proposed method can effectively classify even with limited training samples. Furthermore, the classification accuracy of FCAN_AKF steadily improves with increasing training sample size, demonstrating its robustness.

[0203] The present invention may have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

[0204] References

[0205] [1]J.Transon,R.d'Andrimont,A.Maugnard and P.Defourny,"Survey ofcurrent hyperspectral Earth observation applications from space and synergieswith Sentinel-2,"20179th International Workshop on the Analysis ofMultitemporalRemote Sensing Images(MultiTemp),Brugge,Belgium,2017,pp.1-8.

[0206] [2]Transon,J.;D’Andrimont,R.;Maugnard,A.;Defourny,P."Survey ofHyperspectral Earth Observation Applications from Space inthe Sentinel-2Context,"Remote Sens.2018,vol.10,157.

[0207] [3]Islam Abou El-Magd,Ahmed El-Zeiny,"Quantitative hyperspectralanalysis for characterization of the coastal water from Damietta to PortSaid,Egypt,"The Egyptian Journal ofRemote Sensing and Space Science,2014,vol.17,Issue 1,pp.61-76.

[0208] [4]Tingting Hou,Weiwei Sun,Chao Chen,Gang Yang,Xiangchao Meng,Jiangtao Peng,"Marine floating raft aquaculture extraction of hyperspectralremote sensing images based decision tree algorithm,"International JournalofAppliedEarthObservation and Geoinformation,2022,vol.111,102846,ISSN 1569-8432.

[0209] [5]Gang Yang,Ke Huang,Weiwei Sun,Xiangchao Meng,Dehua Mao,Yong Ge,Enhanced mangrove vegetation index based on hyperspectral images for mappingmangrove,ISPRS Journal of Photogrammetry and Remote Sensing,2022,vol.189,pp.236-254,ISSN 0924-2716.

[0210] [6]M.Teke,H.S.Deveci, and U.Sakarya,"A short surveyofhyperspectral remote sensing applications in agriculture,"20136thInternational Conference on RecentAdvances in Space Technologies(RAST),Istanbul,Turkey,2013,pp.171-176.

[0211] [7]Lu,Bing,Phuong D.Dao,Jiangui Liu,Yuhong He,and Jiali Shang."RecentAdvances ofHyperspectral Imaging Technology andApplications inAgriculture"Remote Sensing,2020,vol.12,no.16,p.2659.

[0212] [8]Anand S.Sahadevan,"Extraction of spatial-spectral homogeneouspatches and fractional abundances for field-scale agriculture monitoringusing airborne hyperspectral images,"Computers and Electronics inAgriculture,2021,vol.188,106325.

[0213] [9]Lv M,Li W,Chen T,Zhou J,Tao R."Discriminant Tensor-Based ManifoldEmbedding for Medical Hyperspectral Imagery,"IEEE J Biomed Health Inform,2021,vol.25,no.9,pp.3517-3528.

[0214]

[10] Aref,M.,Youssef,A.,Aboughaleb,I.,&El-Sharkawy,Y."Characterizationof Normal and Malignant Breast Tissues utilizing Hyperspectral Images andAssociated Differential Spectrum Algorithm,"Journal of Biomedical Photonics&Engineering,2021,vol.7,no.2,020302.

[0215]

[11] Kumar S,Ghosh J,Crawford M M."Best-bases feature ex-tractionalgorithms for classification ofhyperspectral data".IEEE Transactions onGeoscience and Remote Sensing,2001,vol.39,no.7:pp.1368-1379.

[0216]

[12] Ediriwickrema J,Khorram S."Hierarchical maximum-likelihoodclassification for improved accuracies".IEEE Transactions on Geoscience andRemote Sensing,1997,vol.35,no.4,pp.810-816.

[0217]

[13] Chen Y,Nasrabadi N M,Tran T D."Classification for hyperspectralimagery based on sparse representation".Evolution in Remote Sensing,June 14-16,2010.Reykjavik,Iceland.New York,USA:IEEE,2010.

[0218]

[14] M.E.Paoletti,J.M.Haut,J.Plaza,A.Plaza,"Deep learning classifiersfor hyperspectral imaging:A review",ISPRS Journal ofPhotogrammetry and RemoteSensing,2019,vol.158,pp.279-317.

[0219]

[15] Wenjing Lv,Xiaofei Wang,"Overview ofHyperspectral ImageClassification",Journal ofSensors,2020,vol.2020,p.13.

[0220]

[16] Z.Li,F.Liu,W.Yang,S.Peng and J.Zhou,"A Survey ofConvolutionalNeural Networks:Analysis,Applications,and Prospects,"in IEEE Transactions onNeural Networks and Learning Systems,Dec.2022,vol.33,no.12,pp.6999-7019.

[0221]

[17] Wei Hu,Yangyu Huang,Li Wei,Fan Zhang,Hengchao Li,"DeepConvolutional Neural Networks for Hyperspectral Image Classification",JournalofSensors,2015,vol.2015,p.12.

[0222]

[18] Li,Ying,Haokui Zhang,and Qiang Shen."Spectral–SpatialClassification of Hyperspectral Imagery with 3D Convolutional NeuralNetwork,"Remote Sensing,2017,vol.9,no.1,p.67.

[0223]

[19] Z.Zhong,J.Li,Z.Luo and M.Chapman,"Spectral–Spatial ResidualNetwork for Hyperspectral Image Classification:A 3-D Deep LearningFramework,"in IEEE Transactions on Geoscience and Remote Sensing,Feb.2018,vol.56,no.2,pp.847-858.

[0224]

[20] Wang,Wenju,Shuguang Dou,Zhongmin Jiang,and Liujie Sun."A FastDense Spectral–Spatial Convolution Network Framework for Hyperspectral ImagesClassification"Remote Sensing,2018,vol.10,no.7,p.1068.

[0225]

[21] J.M.Haut,M.E.Paoletti,J.Plaza,A.Plaza and J.Li,"Visual Attention-Driven Hyperspectral Image Classification,"in IEEE Transactions on Geoscienceand Remote Sensing,Oct.2019,vol.57,no.10,pp.8065-8080.

[0226]

[22] Mei,Xiaoguang,Erting Pan,Yong Ma,Xiaobing Dai,Jun Huang,Fan Fan,Qinglei Du,Hong Zheng,and Jiayi Ma."Spectral-Spatial Attention Networks forHyperspectral Image Classification"Remote Sensing,2019,vol.11,no.8,p.963.

[0227]

[23] H.Sun,X.Zheng,X.Lu and S.Wu,"Spectral–Spatial Attention Networkfor Hyperspectral Image Classification,"in IEEE Transactions on Geoscienceand Remote Sensing,May 2020,vol.58,no.5,pp.3232-3245.

[0228]

[24] Ma,Wenping,Qifan Yang,Yue Wu,Wei Zhao,and Xiangrong Zhang."Double-Branch Multi-Attention Mechanism Network for Hyperspectral ImageClassification"Remote Sensing 2019,vol.11,no.11,p.1307.

[0229]

[25] Li,Rui,Shunyi Zheng,Chenxi Duan,Yang Yang,and Xiqi Wang."Classification ofHyperspectral Image Based on Double-Branch Dual-AttentionMechanism Network"Remote Sensing 2020,vol.12,no.3,p.582.

[0230]

[26] Z.Zhong,Y.Li,L.Ma,J.Li and W.-S.Zheng,"Spectral–SpatialTransformer Network for Hyperspectral Image Classification:A FactorizedArchitecture Search Framework,"in IEEE Transactions on Geoscience and RemoteSensing,2022,vol.60,pp.1-15,Art no.5514715.

[0231]

[27] Z.Xue,X.Yu,B.Liu,X.Tan and X.Wei,"HResNetAM:Hierarchical ResidualNetwork With Attention Mechanism for Hyperspectral Image Classification,"inIEEE Journal of Selected Topics in Applied Earth Observations and RemoteSensing,2021,vol.14,pp.3566-3580.

[0232]

[28] X.Zheng,H.Sun,X.Lu and W.Xie,"Rotation-Invariant AttentionNetwork for Hyperspectral Image Classification,"in IEEE Transactions on ImageProcessing,2022,vol.31,pp.4251-4265.

[0233]

[29] S.Mei,X.Li,X.Liu,H.Cai and Q.Du,"Hyperspectral ImageClassification Using Attention-Based Bidirectional Long Short-Term MemoryNetwork,"in IEEE Transactions on Geoscience and Remote Sensing,2022,vol.60,pp.1-12,Art no.5509612.

[0234]

[30] Xiuping Jia and J.A.Richards,"Segmented principal componentstransformation for efficient hyperspectral remote-sensing image display andclassification,"in IEEE Transactions on Geoscience and Remote Sensing,Jan.1999,vol.37,no.1,pp.538-542.

[0235]

[31] Jaime Zabalza,Jinchang Ren,Jiangbin Zheng,Huimin Zhao,ChunmeiQing,Zhijing Yang,Peijun Du,Stephen Marshall,Novel segmented stackedautoencoder for effective dimensionality reduction and feature extraction inhyperspectral imaging,Neurocomputing,2016,vol.185,pp.1-10,ISSN 0925-2312.

[0236]

[32] Genyun Sun,Xuming Zhang,Xiuping Jia,Jinchang Ren,Aizhu Zhang,Yanjuan Yao,Huimin Zhao,Deep Fusion of Localized Spectral Features and Multi-scale Spatial Features for Effective Classification of Hyperspectral Images,International Journal of Applied Earth Observation and Geoinformation,2020,vol.91,102157,ISSN 1569-8432,

[0237]

[33] Wu,Haiyang,Cuiping Shi,Liguo Wang,and Zhan Jin."A Cross-ChannelDense Connection and Multi-Scale Dual Aggregated Attention Network forHyperspectral Image Classification"Remote Sensing,2023,vol.15,no.9,p.2367.

[0238]

[34] R.Liu,W.Cai,G.Li,X.Ning and Y.Jiang,"Hybrid Dilated ConvolutionGuided Feature Filtering and Enhancement Strategy for Hyperspectral ImageClassification,"in IEEE Geoscience and Remote Sensing Letters,2022,vol.19,pp.1-5,Art no.5508105.

[0239]

[35] X.Zhang,S.Shang,X.Tang,J.Feng and L.Jiao,"Spectral PartitioningResidual Network With Spatial Attention Mechanism for Hyperspectral ImageClassification,"in IEEE Transactions on Geoscience and Remote Sensing,2022,vol.60,pp.1-14.

[0240]

[36] S.K.Roy,S.Manna,T.Song and L.Bruzzone,"Attention-Based AdaptiveSpectral–Spatial Kernel ResNet for Hyperspectral Image Classification,"inIEEE Transactions on Geoscience and Remote Sensing,Sept.2021,vol.59,no.9,pp.7831-7843,.

[0241]

[37] C.Shi,D.Liao,T.Zhang and L.Wang,"Hyperspectral ImageClassification Based on Expansion Convolution Network,"in IEEE Transactionson Geoscience and Remote Sensing,2022,vol.60,pp.1-16.

Claims

1. A hyperspectral image classification method based on an adaptive filter and a feature complementary network, characterized by: The specific process of the method is: (1) Establish a feature complementary network model FCAN_AKF based on adaptive knowledge filtering; Obtain the trained feature complementary network model FCAN_AKF based on adaptive knowledge filtering; (2) Input the hyperspectral image to be tested into the trained feature complementary network model FCAN_AKF based on adaptive knowledge filtering to complete the classification of the hyperspectral image to be tested; In said (1), a feature complementary network model FCAN_AKF based on adaptive knowledge filtering is established; Obtain the trained feature complementary network model FCAN_AKF based on adaptive knowledge filtering; The specific process is: Step 1: Randomly sample hyperspectral images; The hyperspectral image training set is represented as X∈R h×w×b , label Y∈R h×w ; Where h, w, and b represent the height, width, and number of spectral bands of the hyperspectral image HSI, respectively, and R represents a real number; The training set is randomly selected from each category according to a preset ratio; Step 2: Establish a feature complementary network model FCAN_AKF based on adaptive knowledge filtering; Obtain the trained feature complementary network model FCAN_AKF based on adaptive knowledge filtering; In the step 2, a feature complementary network model FCAN_AKF based on adaptive knowledge filtering is established; Obtain the trained feature complementary network model FCAN_AKF based on adaptive knowledge filtering; The specific process is: The feature complementation network model FCAN_AKF based on adaptive knowledge filtering includes: non-local band reconstruction strategy module, non-local feature extraction module, local feature extraction module, feature complementation fusion module, double pyramid spectral space attention module, adaptive knowledge filter and classification layer; Step 2.

1. Divide the original hyperspectral image into different patches x∈R h×w×b ; Step 2. Patch x∈R h×w×b The non-local band recombination strategy module NBR of the feature complementation network model FCAN_AKF based on adaptive knowledge filtering is input with a batch size of 64. The non-local band recombination strategy module NBR first analyzes the correlation between spectral bands and groups them according to the correlation, and then the spectra in each group are recombined. Step 2 and 3: Input the NBR output x′ into the non-local feature extraction module and output the non-local features; Step 24: Patch x∈R h×w×b Input the local feature extraction module with a batch size of 64 and output local features; Step 25: Concatenate the non-local features obtained in step 23 and the local features obtained in step 24, input the concatenated features into a feature complementary fusion module, and output the fused features; Step 26: Input the fused features into the dual pyramid spectral spatial attention module, and the dual pyramid spectral spatial attention module outputs the spectral spatial attention; Step 27: Input the spectral spatial attention output of the dual pyramid spectral spatial attention module into the adaptive knowledge filter, and the adaptive knowledge filter outputs features; Step 28: Input the output features of the adaptive knowledge filter into the classification layer for classification; Step 29: Obtain the trained feature complementary network model FCAN_AKF based on adaptive knowledge filtering.

2. The hyperspectral image classification method based on adaptive filter and feature complementary network according to claim 1, characterized in that: In step 22, the patch x∈R h×w×b The non-local band recombination strategy module NBR of the feature complementation network model FCAN_AKF based on adaptive knowledge filtering is input with a batch size of 64. The non-local band recombination strategy module NBR first analyzes the correlation between spectral bands and groups them according to the correlation. Then, the spectra in each group are recombined. The specific process is as follows: Step 221: Group the spectral bands of the patch according to the correlation between the bands; the specific process is: The process of spectral correlation analysis can be expressed as Where C(i,j) represents the correlation between the i-th band and the j-th band, Cov(·) represents the covariance operation, Cov(i,j) represents the covariance operation between the i-th band and the j-th band, Cov(i,i) represents the covariance operation between the i-th band and the i-th band, Cov(j,j) represents the covariance operation between the j-th band and the j-th band, and · represents the dot product; Step 222: Reorganize the spectra within each group. The specific process is as follows: Each group was again evenly divided into B1, B2, and B3; Then non-adjacent band groups can be inserted between adjacent band groups, i.e. B1 is inserted between the adjacent groups B2 and B3; The process is described as x′=NBR(B1,B2,B3) (3) Among them, Slice -1 (·) represents the spectrum slicing operation; x C is the output of correlation grouping; is a rounding function, and b represents the number of spectral bands; n represents the number of spectral recombined groups; B n is the result of grouping; NBR(·) represents the non-local band splicing operation, which is to insert B1 between the two adjacent groups B2 and B3, and x′ represents the output of NBR.

3. The hyperspectral image classification method based on adaptive filter and feature complementary network according to claim 2, characterized in that: In the steps 2 and 3, the output x′ of NBR is input into the non-local feature extraction module to output the non-local features; The specific process is: The non-local feature extraction module includes submodule 1, submodule 2, submodule 3, depthwise separable convolution submodule, submodule 5, and submodule 6; Submodule 1 includes 3D convolution, BN, and ReLU in sequence; Submodule 2 includes 3D convolution, BN, and ReLU in sequence; Submodule 3 includes 3D convolution, BN, and ReLU in sequence; The depthwise separable convolution submodule includes 3D depthwise convolution, BN, Relu, 3D pointwise convolution, BN, Relu in sequence; Submodule 5 includes three-dimensional point-by-point convolution, BN, and ReLU in sequence; Submodule 6 includes three-dimensional point-wise convolution, BN, ReLU, deformation and compression operations in sequence; x′ passes through submodule 1 to obtain feature 1; Feature 1 passes through submodule 2 to obtain feature 2; Feature 1 and feature 2 are combined into feature 3 through submodule 3; Feature 1, feature 2 and feature 3 are processed through the depth-wise separable convolution submodule to obtain feature 4; Feature 1, feature 2, feature 3 and feature 4 are processed through submodule 5 to obtain feature 5; Feature 5 passes through submodule 6 to obtain feature 6.

4. The hyperspectral image classification method based on adaptive filter and feature complementary network according to claim 3, characterized in that: In step 24, the patch x∈R h×w×b Input the local feature extraction module with a batch size of 64 and output local features; The specific process is: The local feature extraction module includes submodule 1, submodule 2, submodule 3, depthwise separable convolution submodule, submodule 5, and submodule 6; Submodule 1 includes 3D convolution, BN, and ReLU in sequence; Submodule 2 includes 3D convolution, BN, and ReLU in sequence; Submodule 3 includes 3D convolution, BN, and ReLU in sequence; The depthwise separable convolution submodule includes 3D depthwise convolution, BN, Relu, 3D pointwise convolution, BN, Relu in sequence; Submodule 5 includes three-dimensional point-by-point convolution, BN, and ReLU in sequence; Submodule 6 includes three-dimensional point-wise convolution, BN, ReLU, deformation and compression operations in sequence; x passes through submodule 1 to obtain feature A; Feature A is converted into feature B through submodule 2; Feature A and feature B are converted into feature C through submodule 3; Feature A, feature B and feature C are processed through the depth-wise separable convolution submodule to obtain feature D; Feature A, feature B, feature C and feature D are processed through submodule 5 to obtain feature E; Feature E is converted into feature F through submodule 6.

5. The hyperspectral image classification method based on adaptive filter and feature complementary network according to claim 4, characterized in that: In the step 25, the non-local features obtained in the step 23 and the local features obtained in the step 24 are spliced, and the spliced ​​features are input into a feature complementary fusion module to output the fused features; The specific process is: The feature 6 obtained in step 2 and 3 and the feature F obtained in step 2 and 4 are concatenated and input into the fully connected layer. The output of the fully connected layer is sequentially input into the two-dimensional point-by-point convolution, BN, and ReLU to output the fused features.

6. The hyperspectral image classification method based on adaptive filter and feature complementary network according to claim 5, characterized in that: In step 26, the fused features are input into the dual pyramid spectral spatial attention module, and the dual pyramid spectral spatial attention module outputs the spectral spatial attention; The specific process is: The dual pyramid spectral-spatial attention module includes a dual pyramid spectral attention module and a dual pyramid spatial attention module; The fused features are input into the dual-pyramid spectral attention module, and the dual-pyramid spectral attention module outputs spectral attention; The dual-pyramid spectral attention module outputs spectral attention and inputs it into the dual-pyramid spatial attention module, which then outputs spatial attention. The spectral attention and spatial attention are summed to obtain the spectral spatial attention; The fused features are input into the dual-pyramid spectral attention module, and the dual-pyramid spectral attention module outputs spectral attention. The specific process is: In the dual-pyramid spectral attention module DPSpe_A, the key tensor K, query tensor Q, and value tensor V are obtained by linear transformation of the input; the process is described as Where b, h, and w represent the number of input bands, height, and width respectively; W K 、W Q and W V Represents the operation of the linear transformation layer; r(x in ) is to input x in The spatial dimension of is transformed from h×w to N, that is, N=h×w; () Γ represents transpose, · represents dot product; K′=P1(K)||P2(K)||P4(K)||P8(K) (7) V′=P1(V)||P2(V)||P4(V)||P8(V) (8) Where P1, P2, P4, and P8 represent adaptive average pooling with scales of 1, 2, 4, and 8, respectively; || represents the concatenation operation; K′ represents the key tensor after adaptive average pooling; V′ represents the value tensor after adaptive average pooling; x spe =relu(r(x in )+sf(K′·Q)·V′) (9) Among them, sf(·) represents the softmax function; relu(·) is the activation function; x spe is the spectral attention, · represents the dot product; The dual-pyramid spectral attention module outputs spectral attention and inputs it into the dual-pyramid spatial attention module, and the dual-pyramid spatial attention module outputs spatial attention. The specific process is: In the dual pyramid spatial attention module DPSpa_A, the key tensor k, query tensor q and value tensor v are obtained by linear transformation of the input; the process is described as Where b, h, and w represent the number of input bands, height, and width respectively; w k 、w q and w v Represents the operation of the linear transformation layer; r(x spe ) is to input x spe The spatial dimension of is transformed from h×w to N, that is, N=h×w; () Γ represents transpose, · represents dot product; k′=P1(k)||P2(k)||P4(k)||P8(k) v′=P1(v)||P2(v)||P4(v)||P8(v) Where P1, P2, P4, and P8 represent adaptive average pooling of scales 1, 2, 4, and 8, respectively; || represents the concatenation operation; k′ represents the key tensor after adaptive average pooling; v′ represents the value tensor after adaptive average pooling; x′ spe =relu(r(x spe )+sf(k′·q)·v′) Among them, sf(·) represents the softmax function; relu(·) is the activation function; x′ spe is the final spectral space attention, and represents the dot product.

7. The hyperspectral image classification method based on adaptive filter and feature complementary network according to claim 6, characterized in that: In step 27, the spectral spatial attention output of the dual pyramid spectral spatial attention module is input into the adaptive knowledge filter, and the adaptive knowledge filter outputs features; The specific process is: Denote the eigenvalue of the input filter as x DPSSA ; First, use the sigmiod function to calculate x DPSSA Normalize to obtain the eigenvalue X, and then input the eigenvalue X into the knowledge filter; When the eigenvalue X of the input knowledge filter is greater than or equal to T, the input eigenvalue X will be gained, that is, the eigenvalue X becomes N*X; On the contrary, when the eigenvalue X of the input knowledge filter is less than T, the input eigenvalue X will be set to 0; the process is described as Where N is the gain value, T is the threshold, and * is the multiplication sign; The filter output feature value and the double pyramid spectral spatial attention module output spectral spatial attention are added together as the final adaptive knowledge filter output feature.

8. The hyperspectral image classification method based on adaptive filter and feature complementary network according to claim 7, characterized in that: In step 29, the trained feature complementary network model FCAN_AKF based on adaptive knowledge filtering is obtained; the specific process is: Step 291: Initialize the network, i.e. set the flag of adaptive knowledge filtering to 0; Update the flag bit NFlag of N to 1, and update the flag bit TFlag of T to 0; The N update step size S and T update step size s′ are set to 1 and 0.1 respectively; Step 2912: the current prediction accuracy ROA and the best prediction accuracy BOA of the feature complementary network model FCAN_AKF based on adaptive knowledge filtering are initialized to 0; Step 2913: Start training the feature complementary network model FCAN_AKF based on adaptive knowledge filtering and save the training parameters; Step 2914: Perform verification based on the feature complementary network model FCAN_AKF based on adaptive knowledge filtering, calculate the loss value and save the current prediction accuracy ROA; Determine whether the loss value of the network model FCAN_AKF decreases 10 times continuously; if the loss value of the network model FCAN_AKF does not decrease 10 times continuously, that is, the adaptive knowledge filtering flag Flag = 0, execute step 2913 until the maximum number of training times is reached; If the loss value of the network model FCAN_AKF decreases 10 times in a row, that is, the adaptive knowledge filtering flag is set to 1, the network model FCAN_AKF will update N and T and execute steps 2913 until the maximum number of training times is reached; The specific process is: When performing the update of N and T, the network model FCAN_AKF will first determine the status of NFlag; When NFlag is 1, the network model FCAN_AKF will compare the current prediction accuracy with the best prediction accuracy. If the current prediction accuracy is greater than the best prediction accuracy, the current prediction accuracy is assigned to the best prediction accuracy, and NFlag and TFlag are set to 0 and 1 respectively. N forward updates are performed, that is, N = N + S. If the current prediction accuracy is less than or equal to the best prediction accuracy, set TFlag to 1 and perform the reverse update of N, that is, N = N-2S; When NFlag is 0, the network model FCAN_AKF will determine the status of TFlag; when TFlag is 1, the network model FCAN_AKF will compare the current prediction accuracy with the best prediction accuracy; if the current prediction accuracy is greater than the best prediction accuracy, the current prediction accuracy will be assigned to the best prediction accuracy, and TFlag and NFlag will be set to 0 and 1 respectively; the forward update of T will be executed, that is, T=T+s′; if the current prediction accuracy is less than or equal to the best prediction accuracy, NFlag will be set to 1 and the reverse update of T will be executed, that is, T=T-2s′.

Images