Hyperspectral remote sensing image classification method based on width and depth fourier network
By using a hyperspectral remote sensing image classification method based on width and depth Fourier networks, the overfitting problem caused by limited training samples is solved, achieving efficient hyperspectral image classification and improving learning efficiency and accuracy.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-27
- Publication Date
- 2026-03-17
AI Technical Summary
The limited number of training samples for hyperspectral remote sensing images makes them prone to overfitting when training models with a large number of parameters, making it difficult for existing technologies to effectively classify them.
A hyperspectral remote sensing image classification method based on width and depth Fourier networks is adopted. Features are extracted in the frequency domain through Fourier transform, and classification is performed by combining fully connected layers. The least squares method is used to learn the weights, simplifying the network training process.
It improves learning efficiency and accuracy, reduces computational load, and achieves effective classification of hyperspectral images with limited samples.
Smart Images
Figure CN114998730B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of artificial intelligence and remote sensing, and specifically to a hyperspectral remote sensing image classification method based on a width and depth Fourier network (WD-FNet). Background Technology
[0002] Classification of hyperspectral remote sensing imagery has always been a hot topic in the field of remote sensing. Over the past thirty years, many machine learning algorithms, such as decision trees, support vector machines, multilayer perceptrons, and random forests, have been used for hyperspectral remote sensing image classification. With the development of artificial intelligence, deep learning algorithms have been successfully applied to hyperspectral remote sensing image classification, such as deep convolutional neural networks, deep recurrent neural networks, long short-term memory artificial neural networks, and spectral-spatial attention mechanism networks based on recurrent neural networks and convolutional neural networks. However, the number of training samples for hyperspectral remote sensing images is usually very limited, making overfitting an unavoidable problem when training models with a large number of parameters.
[0003] Incremental learning and scalable learning can effectively classify hyperspectral remote sensing images with appropriate computational load, proving highly effective for hyperspectral image classification with limited training samples. Furthermore, wide and deep learning models can be generated in the depth direction, the width direction, or both simultaneously, allowing for better extraction of spatial spectral information and hierarchical abstract features from hyperspectral remote sensing images. Wide neural networks typically exhibit better generalization performance. Summary of the Invention
[0004] To address the problems existing in the prior art, the purpose of this invention is to provide a hyperspectral remote sensing image classification method based on width and depth Fourier networks.
[0005] To achieve the above objectives, the present invention employs the following technical solutions.
[0006] A hyperspectral remote sensing image classification method based on width and depth Fourier networks includes the following steps:
[0007] Step 1: Acquire multiple hyperspectral remote sensing images, preprocess the hyperspectral remote sensing images to obtain a training set;
[0008] Step 2: Construct width and depth Fourier networks;
[0009] Step 3: Use the training set samples to train the width and depth Fourier networks to obtain the trained width and depth Fourier networks.
[0010] Step 4: Classify the hyperspectral remote sensing images using the trained width and depth Fourier networks.
[0011] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0012] (1) The wide deep Fourier neural network of the present invention uses Fourier transform in the width direction to extract the spatial and spectral information of hyperspectral images, and adds wide Fourier layers layer by layer in the depth direction to extract the layered features of the image, thereby improving learning efficiency and learning accuracy.
[0013] (2) The present invention converts the image to the frequency domain for feature extraction, and can use fast Fourier transform to accelerate the calculation process; and since only the effective Fourier transform output components are used, the amount of output data is greatly reduced. Therefore, the width and depth Fourier networks of the present invention only require a small amount of computation.
[0014] (3) The present invention adds a fully connected layer for classification in the width and depth Fourier neural networks. The weights in the fully connected layer are learned by using the least squares method, which simplifies the network training process. Attached Figure Description
[0015] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.
[0016] Figure 1 This is a schematic diagram of the process of the present invention;
[0017] Figure 2 This is a schematic diagram of the processing flow of the width Fourier layer of the present invention. Detailed Implementation
[0018] The embodiments of the present invention will be described in detail below with reference to examples. However, those skilled in the art will understand that the following examples are only for illustrating the present invention and should not be regarded as limiting the scope of the present invention.
[0019] refer to Figure 1 A hyperspectral remote sensing image classification method based on width and depth Fourier networks includes the following steps:
[0020] Step 1: Acquire multiple hyperspectral remote sensing images, preprocess the hyperspectral remote sensing images to obtain a training set;
[0021] Specifically, principal component analysis and normalization are performed on the hyperspectral remote sensing image. The processed hyperspectral remote sensing image is then divided into M fixed-size image blocks. Finally, each image block is stretched to obtain M stretch vectors. The M stretch vectors obtained from the preprocessing of each hyperspectral remote sensing image are used as a set of training samples to obtain the training set.
[0022] Step 2: Construct width and depth Fourier networks;
[0023] Specifically, the width and depth Fourier networks consist of four consecutively connected width Fourier layers and fully connected layers; the input to the width and depth Fourier networks is M stretching vectors obtained from the preprocessing of hyperspectral remote sensing images, and the output is the classification result of the hyperspectral remote sensing images.
[0024] The input to the first wide Fourier layer is M stretching vectors obtained from the preprocessing of the hyperspectral remote sensing image; the input to the second to fourth wide Fourier layers is the output of the previous layer.
[0025] refer to Figure 2 The processing procedure for the width Fourier layers of layers 1 to 3 is as follows:
[0026] Sub-step 1: Use a sliding window to slice the input vector, resulting in N one-dimensional slices for each input vector;
[0027] Sub-step 2: Fill the other dimensions of each one-dimensional slice with zeros to obtain N sub-vectors;
[0028] Sub-step 3: Perform a discrete Fourier transform on each sub-vector, resulting in K complex vectors for each sub-vector; using the Fourier transform amplitude as the effective feature vector, calculate the modulus of the K complex vectors of each sub-vector to obtain K Fourier transform amplitude vectors; to more effectively represent the features in the frequency domain, sum the K Fourier transform amplitude vectors of each sub-vector to obtain the Fourier transform amplitude vector of each sub-vector.
[0029] Sub-step 4: Sort the Fourier transform amplitude vectors of all sub-vectors from largest to smallest;
[0030] Sub-step 5: Keep the first N S The Fourier transform amplitude vectors of the N sub-vectors are used as feature vectors of each input vector and output; where N S This represents the dimension of the subvectors in the next layer of the network.
[0031] The processing procedure for the last width Fourier layer, i.e., the 4th width Fourier layer, is as follows:
[0032] Sub-step 1: Use a sliding window to slice the input vector, resulting in N one-dimensional slices for each input vector;
[0033] Sub-step 2: Fill the other dimensions of each one-dimensional slice with zeros to obtain N sub-vectors;
[0034] Sub-step 3: Perform a discrete Fourier transform on each sub-vector, resulting in K complex vectors for each sub-vector;
[0035]
[0036] In the formula, v mnLet be the nth sub-vector in the mth stretching vector, where 1≤m≤M, 1≤n≤N; L represents the number of frequency components, 1≤l≤L-1;
[0037] Using the Fourier transform amplitude as the effective eigenvector, the modulus of the K complex vectors of each sub-vector is calculated to obtain the K Fourier transform amplitude vectors.
[0038] The Fourier transform amplitude vector is represented as:
[0039]
[0040] To more effectively represent the features in the frequency domain, the K Fourier transform amplitude vectors of each sub-vector are summed to obtain the Fourier transform amplitude vector of each sub-vector.
[0041] Sub-step 4: Sort the Fourier transform amplitude vectors of all sub-vectors from largest to smallest;
[0042] Sub-step 5: Keep the first N S The Fourier transform amplitude vectors of the N sub-vectors serve as the feature vectors of each input vector; where N S This represents the dimension of the subvectors in the next layer of the network.
[0043] Sub-step 6: Merge the feature vectors of all input vectors to obtain a merged vector and output it.
[0044] The fully connected layer classifies the merged vector output by the last wide Fourier layer and outputs the classification result.
[0045] The output of the fully connected layer is:
[0046]
[0047] In the formula, This is the output of the Q-th Fourier layer;
[0048] Assuming the truth value of the instance class is given by D, the weights can be calculated using the least squares method:
[0049]
[0050] U given below FT The pseudo-inverse calculation formula yields:
[0051]
[0052] The hyperparameters of the four width Fourier layers are as follows:
[0053] The first wide Fourier transform layer has a sliding window length of 15, a step size of 0.9, a discrete Fourier transform size of 600, and 100 outputs obtained by trimming the data.
[0054] The second wide Fourier transform layer has a sliding window length of 0.35, a step size of 0.15, a discrete Fourier transform size of 1000, and 100 outputs obtained by trimming the data.
[0055] The third wide Fourier transform layer has a sliding window length of 0.3, a step size of 0.15, a discrete Fourier transform size of 1000, and 100 outputs obtained by trimming the data.
[0056] The fourth wide Fourier transform layer has a sliding window length of 0.32, a step size of 0.15, a discrete Fourier transform size of 3000, and 300 outputs obtained by trimming the data.
[0057] Step 3: Use the training set samples to train the width and depth Fourier networks to obtain the trained width and depth Fourier networks.
[0058] Step 4: Classify the hyperspectral remote sensing images using the trained width and depth Fourier networks.
[0059] Simulation test
[0060] The Pavia University dataset was used to test the performance of wide and deep Fourier networks. The Pavia University data was acquired by the Reflective Optical Systems Imaging Spectroradiometer (ROSIS) sensor above the Pavia University campus. It has nine land cover types, and the image size is 610×610. After removing pixels with no information, the size is 610×340, and there are 103 bands. Nine datasets were obtained from the Pavia University dataset. The computer system used had an Intel-I7-8700K CPU @ 3.7GHz and 32GB of RAM.
[0061] The classification accuracy of the wide and deep Fourier networks in this embodiment is compared with that of mainstream learning models (including Multilayer Perceptron (MLP), Convolutional Neural Network (CNN), 2D CNN, 3D CNN, Dynamic Width and Depth Neural Network (DWDNN), WSWS network, and SMSB network), and overall accuracy (OA), average accuracy (AA), and Kappa coefficient are used as evaluation metrics. The comparison results are shown in Table 1.
[0062] The formula for calculating OA is as follows:
[0063]
[0064] In the formula, N c and N t These represent the total number of correctly classified and test samples, respectively.
[0065] AA is defined as:
[0066]
[0067] In the formula, N c_i and N t_i These represent the number of correctly classified and test samples of the i-th class, respectively.
[0068] Table 1. Classification accuracy (%) of different methods
[0069] Data group MLP CNN 2-D CNN 3D CNN SMSB WSWS DWDNN WD-FNet 1 97.13 96.18 98.51 98.40 99.11 99.10 99.87 99.85 2 98.43 96.69 99.54 96.91 98.97 100.00 100.00 99.96 3 95.15 80.86 84.62 97.05 98.89 93.01 96.98 98.89 4 95.05 87.21 98.04 98.84 98.74 98.37 99.29 99.18 5 99.88 99.63 100.00 100.00 100.00 99.88 99.75 99.26 6 96.35 88.30 97.10 99.32 99.87 99.97 100.00 100.00 7 90.85 95.05 95.05 98.92 99.79 99.00 98.62 100.00 8 93.21 96.39 96.39 98.33 98.99 98.33 99.59 99.50 9 99.30 99.69 99.69 99.90 98.04 98.95 99.65 99.30 OA 96.47 93.66 97.84 96.52 99.11 99.19 99.69 99.77 AA 95.04 91.65 96.56 97.47 99.16 98.51 99.31 99.55 Kappa 95.36 91.72 97.19 95.50 98.79 98.93 99.59 99.69
[0070] As can be seen from Table 1, WD-FNet in this embodiment has the best overall performance among all classification methods, with OA, AA and Kappa coefficients of 99.77%, 99.55% and 99.69%, respectively.
[0071] Although the present invention has been described in detail in this specification with general description and specific embodiments, some modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present invention are within the scope of protection claimed by the present invention.
Claims
1. A hyperspectral remote sensing image classification method based on a width and depth Fourier network, characterized in that, The method comprises the following steps: Step 1, obtaining a plurality of hyperspectral remote sensing images, and preprocessing the hyperspectral remote sensing images to obtain a training set; Step 2, establishing a width and depth Fourier network; Step 3, training the width and depth Fourier network using the training set samples to obtain a trained width and depth Fourier network; Step 4, classifying the hyperspectral remote sensing images using the trained width and depth Fourier network. The width and depth Fourier network comprises Q width Fourier layers and a full connection layer connected in sequence; the input of the width and depth Fourier network is M stretching vectors obtained by preprocessing the hyperspectral remote sensing images, and the output is a classification result of the hyperspectral remote sensing images; the width Fourier layer is used for extracting features of the hyperspectral images, and the full connection layer is used for outputting the classification result; The processing procedure of the width Fourier layer of the first to Q-1 layers is as follows: Substep 1, slicing the input vectors using a sliding window to obtain N one-dimensional slices under each input vector; Substep 2, filling other dimensions of each one-dimensional slice with zeros to obtain N sub-vectors; Substep 3, performing discrete Fourier transform on each sub-vector to obtain K complex vectors for each sub-vector; performing modulus operation on the K complex vectors of each sub-vector to obtain K Fourier transform amplitude vectors; and summing the K Fourier transform amplitude vectors of each sub-vector to obtain a Fourier transform amplitude vector of each sub-vector; Substep 4, sorting the Fourier transform amplitude vectors of all sub-vectors from large to small; Sub-step 5, reserve the Fourier transform amplitude vector of the first N S sub-vectors as the feature vector of each input vector and output; wherein N S is the dimension of the next layer network sub-vector; The processing procedure of the width Fourier layer of the Qth layer is as follows: Substep 1, slicing the input vectors using a sliding window to obtain N one-dimensional slices under each input vector; Substep 2, filling other dimensions of each one-dimensional slice with zeros to obtain N sub-vectors; Substep 3, performing discrete Fourier transform on each sub-vector to obtain K complex vectors for each sub-vector; performing modulus operation on the K complex vectors of each sub-vector to obtain K Fourier transform amplitude vectors; and summing the K Fourier transform amplitude vectors of each sub-vector to obtain a Fourier transform amplitude vector of each sub-vector; Substep 4, sorting the Fourier transform amplitude vectors of all sub-vectors from large to small; Sub-step 5, keep the Fourier transform amplitude vector of the first N S sub-vectors as the feature vector of each input vector; wherein N S is the dimension of the sub-vector of the next layer network. Substep 6, merging the feature vectors of all input vectors to obtain a merged vector and outputting the merged vector.
2. The hyperspectral remote sensing image classification method based on width and depth Fourier network according to claim 1, wherein, Step 1, specifically, performing principal component analysis and normalization processing on the hyperspectral remote sensing images, then dividing the processed hyperspectral remote sensing images into M fixed-size image blocks, and finally stretching each image block to obtain M stretching vectors; taking the M stretching vectors obtained by preprocessing each hyperspectral remote sensing image as a group of training samples to obtain a training set.