Lightweight hyperspectral image classification method and system based on intelligent optimization

Abstract

The invention discloses a lightweight hyperspectral image classification method and system based on intelligent optimization, and relates to the technical field of image processing. Comprising the following steps: preparing a data set, constructing a network model, training the network model, determining evaluation indexes, and intelligently optimizing the model. According to the technical scheme of the invention, an improved ViT module is adopted to carry out interactive splicing operation on the deep features and the shallow features of the generated image in a mode of taking a brand new multilayer interactive attention mechanism as a core, and more attention is given to the relevance of feature information between layers while feature fusion is carried out. Global spatial features with higher discrimination are extracted; and a high-frequency feature extraction branch, a low-frequency feature extraction branch and a spatial-spectral feature extraction branch are set to extract the data set, so that high-frequency features, low-frequency features and spatial-spectral features are effectively extracted, and the features are jointly extracted and fused to effectively improve the classification precision and feature extraction capability of the overall classification model.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a lightweight hyperspectral image classification method and system based on intelligent optimization. Background Art

[0002] With the rapid development of earth observation technology and hardware platforms, hyperspectral sensors can generate more hyperspectral image data to confirm different types of objects and provide potential possibilities for detailed identification of materials and accurate estimation of abundance. Currently, hyperspectral image classification technology is widely used in military applications, environmental monitoring, and medical diagnosis. However, as a three-dimensional cube data with low spatial resolution and more redundant band information, hyperspectral images still face great challenges in the actual application of hyperspectral image classification using existing classification methods. The classification network with convolutional neural network as the basic framework tends to ignore global features, making it difficult to accurately classify and identify objects in hyperspectral images. Although the network with Transformer as the main framework can extract global features, it often ignores local features. Therefore, it is of great significance to study a hyperspectral image classification model with high computational efficiency and a small number of parameters and to optimize the use of spatial and spectral information by updating the network structure.

[0003] The Chinese patent publication number is CN114118369A, and its name is "A design method for image classification convolutional neural network based on swarm intelligence optimization". This method provides a design method for image classification convolutional neural network based on swarm intelligence optimization. Compared with the models obtained by other traditional intelligent optimization methods, although the image classification accuracy and model parameter performance have been improved, it can only extract shallow feature information. At the same time, there are also problems such as high computational complexity and weak generalization ability that need to be solved.

[0004] Therefore, how to effectively solve the above problems becomes the key to improving the classification performance of hyperspectral images, which deserves further research and improvement. Summary of the invention

[0005] The technical solution of the present invention to solve the above technical problems is to provide a lightweight hyperspectral image classification method based on intelligent optimization, comprising the following steps:

[0006] S1. Prepare datasets: hyperspectral image datasets including Indian Pines dataset, Pavia University dataset and KSC dataset. Label each dataset and use it for hyperspectral image classification.

[0007] S2. Constructing the network model: extracting through high-frequency feature extraction branch, low-frequency feature extraction branch and spatial spectrum feature extraction branch; including high- and low-frequency feature separation module, deep separable convolution module, improved ViT network module, hyperspectral image band reconstruction module, lightweight spatial spectrum attention hybrid convolution module and multi-layer perception module;

[0008] S3. Training network model: Use sample data from Indian Pines dataset, Pavia University dataset and KSC dataset as training sets to train the network model;

[0009] S4. Determine the evaluation index: set the learning rate, select the network optimizer, and determine the hyperspectral image classification accuracy performance evaluation index and lightweight performance evaluation index;

[0010] S5. Intelligent optimization model: Use image datasets to train and fine-tune the model, use the new intelligent optimization model algorithm LSCOA to optimize the model, compare the classification results through evaluation indicators until the conditions are met, and output the final classification results.

[0011] Furthermore, the high- and low-frequency feature separation module includes a frequency domain feature convolution, a normalization layer, and an R-type activation function layer, which are used to decompose the hyperspectral image data into high-frequency features and low-frequency features.

[0012] Furthermore, the improved ViT network module includes image segmentation Patch, an improved multi-layer interactive attention mechanism, position encoding and Transformer encoder. The deep features and shallow features of the generated image are fused through splicing operations with the multi-layer interactive attention mechanism as the core, and more attention is paid to the correlation of information between layers, so as to effectively extract global features while reducing the complexity of calculation.

[0013] Furthermore, the hyperspectral image band reconstruction module performs spectral band matching reconstruction through correlation analysis between bands, and then performs spectral intelligent optimization learning to reconstruct hyperspectral image data with higher resolution.

[0014] Furthermore, the lightweight spatial-spectral attention hybrid convolution module includes three-dimensional convolution, CBAM attention mechanism, Dropout layer, SE attention mechanism, R-type activation function, average pooling layer, two-dimensional convolution and fully connected layer, which are used to extract spatial-spectral features of hyperspectral images; wherein, the CBAM attention mechanism includes a spectral attention module and a spatial attention module.

[0015] Furthermore, the multi-layer perception module includes an input layer, a fully connected layer, a G-type activation function layer, a hidden layer and an output layer, which are used to fuse high-frequency features, low-frequency features and spatial spectrum features, and output classification results.

[0016] Furthermore, the new intelligent optimization model algorithm LSCOA combines the sine-cosine optimization algorithm SCA and the spectrum optimization algorithm LSO to iteratively update individual positions and adaptively adjust the resolution of the search space to find the optimal solution.

[0017] In order to solve the above technical problems, the present invention also proposes a lightweight hyperspectral image classification system based on intelligent optimization, which is used to execute the above lightweight hyperspectral image classification method based on intelligent optimization, comprising:

[0018] Data preparation module, used to prepare and annotate hyperspectral image datasets;

[0019] A network model building module is used to build a lightweight hyperspectral image classification network model including a high-frequency feature extraction branch, a low-frequency feature extraction branch, and a spatial spectrum feature extraction branch;

[0020] The training module is used to select some sample data from the data set as a training set to train the network model;

[0021] An evaluation index determination module is used to set the learning rate, select the network optimizer, and determine the hyperspectral image classification accuracy performance evaluation index and lightweight performance evaluation index;

[0022] The intelligent optimization module is used to train and fine-tune the model using image datasets. The model is optimized using the new intelligent optimization model algorithm LSCOA that combines the sine-cosine optimization algorithm SCA and the spectral optimization algorithm LSO. The classification results are compared through evaluation indicators until the optimal model conditions are met and the final classification results are output.

[0023] Furthermore, the network model building module also includes a high- and low-frequency feature separation module, a deep separable convolution module, an improved ViT network module, a hyperspectral image band reconstruction module, a lightweight spatial-spectral attention hybrid convolution module and a multi-layer perception module.

[0024] Furthermore, the intelligent optimization module adopts a new intelligent optimization model algorithm LSCOA, combined with a sine-cosine optimization algorithm SCA and a spectrum optimization algorithm LSO, to iteratively update individual positions and adaptively adjust the resolution of the search space to find the optimal solution.

[0025] Compared with the prior art, the advantages and positive effects of the technical solution to be protected by the present invention are:

[0026] (1) The present invention creates a new overall hyperspectral image classification model. According to the characteristics that high-frequency feature information focuses on the important local features of hyperspectral images and low-frequency information can capture global features, three branches are constructed to effectively extract high-frequency features, low-frequency features and spatial spectrum features. By jointly extracting and fusing features, the classification accuracy and feature extraction ability of the overall classification model are effectively improved.

[0027] (2) The present invention creates a new algorithm LSCOA for intelligent optimization and updating of the model. By complementarily combining the sine-cosine optimization algorithm with strong adaptability and stability and the spectral optimization algorithm with more stable convergence performance and more suitable for hyperspectral image classification networks, the optimal hyperspectral image classification model is updated and iterated, and the classification performance of hyperspectral images is significantly improved compared with before optimization.

[0028] (3) The lightweight classification network model adopted by the present invention as a whole can still have excellent classification efficiency in scenarios with limited hyperspectral image data resources or restricted applications. Evaluation indicators show that it is easier to train and calculate the model while reducing the classification cost, greatly shortening the time spent on testing. At the same time, a lightweight spatial-spectral attention hybrid convolution module with the same excellent characteristics is created.

[0029] (4) The present invention combines the band optimization method and the band reconstruction method that can select channels with strong correlation in the spatial-spectral feature extraction branch to reconstruct a hyperspectral image with higher resolution, which has richer spectral features and spatial detail features, and facilitates the subsequent precise extraction of hyperspectral images based on spatial-spectral joint features to achieve higher classification accuracy.

[0030] (5) The present invention adopts an improved ViT module to interactively splice the generated deep features and shallow features of the image through a newly proposed multi-layer interactive attention mechanism as the core. While performing feature fusion, it pays more attention to the correlation between feature information between layers, and extracts global spatial features with higher discriminability. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0032] Figure 1 It is a flowchart of the steps of the lightweight hyperspectral image classification method based on intelligent optimization of the present invention;

[0033] Figure 2 It is a structural schematic diagram of the network model of the present invention;

[0034] Figure 3 It is a structural schematic diagram of the high and low frequency feature separation module of the present invention;

[0035] Figure 4 Schematic diagram of the structure of the depth-separable convolution module of the present invention;

[0036] Figure 5 It is a schematic diagram of the structure of the lightweight spatial-spectral attention hybrid convolution module of the present invention;

[0037] Figure 6 It is a structural schematic diagram of the improved ViT network model of the present invention;

[0038] Figure 7 Schematic diagram of the structure of the CBAM attention mechanism of the present invention;

[0039] Figure 8 Schematic diagram of the structure of the SE attention mechanism of the present invention;

[0040] Fig. 9 It is a structural schematic diagram of the multi-layer perception module of the present invention. DETAILED DESCRIPTION

[0041] The present invention proposes a lightweight hyperspectral image classification method and system based on intelligent optimization, aiming to design a lightweight hyperspectral image classification method based on intelligent optimization to improve the classification accuracy and feature extraction capability of the overall classification model.

[0042] The lightweight hyperspectral image classification method and system based on intelligent optimization proposed by the present invention will be described below in a specific embodiment:

[0043] Embodiment 1:

[0044] A lightweight hyperspectral image classification method based on intelligent optimization, such as Figure 1 As shown, the following steps are included:

[0045] S1. Prepare datasets: hyperspectral image datasets including Indian Pines dataset, Pavia University dataset and KSC dataset. Label each dataset and use it for hyperspectral image classification.

[0046] Specifically, Dataset 1 is the Indian Pines dataset, which is a commonly used remote sensing image dataset used for ground object classification and target recognition tasks. The Indian Pines dataset includes 16 ground object categories, including forests, corn, soybeans, wheat, etc. The research generally takes the remaining 200 bands after removing 20 water absorption bands as the research object, annotates the data with an image space size of 145×145 and uses it for hyperspectral image classification;

[0047] The second dataset is the Pavia University dataset. The Pavia University dataset is obtained by an airborne visible infrared imaging spectrometer sensor with a spatial resolution of 18 meters. The dataset is divided into 9 categories, and 200 bands are used as research objects. The image space size of 512×614 is annotated and used for hyperspectral image classification.

[0048] Dataset 3 is the KSC dataset, which was taken by a sensor at the Kennedy Space Center in Florida. The data contains a total of 224 bands, and 176 bands remain after removing water vapor noise. The spatial resolution is 18 meters, and there are a total of 13 categories for hyperspectral image classification;

[0049] S2. Constructing the network model: extracting through high-frequency feature extraction branch, low-frequency feature extraction branch and spatial spectrum feature extraction branch; including high- and low-frequency feature separation module, deep separable convolution module, improved ViT network module, hyperspectral image band reconstruction module, lightweight spatial spectrum attention hybrid convolution module and multi-layer perception module;

[0050] Specifically, the high-frequency feature extraction branch includes a high- and low-frequency feature separation module and a deep separable convolution module; the low-frequency feature extraction branch includes a high- and low-frequency feature separation module and an improved ViT network module; the spatial-spectral feature extraction branch includes a hyperspectral image band reconstruction module and a lightweight spatial-spectral attention hybrid convolution module; finally, the multi-layer perception module fuses the high-frequency features extracted by the high-frequency feature extraction branch, the low-frequency features extracted by the low-frequency feature extraction branch, and the spatial-spectral features extracted by the spatial-spectral feature extraction branch;

[0051] Working principle:

[0052] The dataset uses principal component analysis to perform covariance matrix eigendecomposition preprocessing on the original hyperspectral image with dimensions of W×H×D, and converts it into a new hyperspectral image with dimensions of W×H×C as the input of the two-dimensional stacking module, where W is the image width, H is the image height, D is the number of original image bands, and C is the number of bands after dimensionality reduction;

[0053] The hyperspectral image data after dimension reduction is input into the high- and low-frequency feature separation module through the PCA principal component analysis method, where H is the length of the hyperspectral image data, W is the width of the hyperspectral image data, and C is the number of bands of the hyperspectral image data. The specific composition of the high- and low-frequency feature separation module is as follows: Figure 3 As shown in the figure, the whole module consists of frequency domain feature convolution, normalization layer, and R-type activation function layer. The high- and low-frequency feature separation module first performs feature decomposition through frequency domain feature convolution, and reduces the spatial size of the low-frequency feature to eliminate the redundant information in the low-frequency information of the space. The high-frequency feature is expressed as H×W×(1-p)C, and the reduced low-frequency feature is expressed as H / 3×W / 3×(1-p)C, where p is the proportion of the selected channel number in the total channel number, indicating the spectral dimension of the hyperspectral image data to be processed.

[0054] The activation function used in the high- and low-frequency feature separation module is the R-type function, which is defined as follows:

[0055]

[0056] Among them, x represents the input feature information.

[0057] Considering that the parameter value p has a certain adverse effect on the performance of high- and low-frequency feature extraction, it is necessary to improve the frequency domain feature convolution. For the high-frequency feature frequency update and the update between low frequency and high frequency in the high-frequency feature component, an S-type function that is less affected by noise data is added after the update. The importance of the frequency update feature in the high frequency and the frequency update feature between low frequency and high frequency is further judged according to the size of the eigenvalue, and the weight value is automatically assigned to them. By adaptively adjusting the feature weight value, the proportion of the high-frequency frequency and the frequency feature is adjusted, thereby reducing the impact of parameter p on the high and low frequency components. Similarly, the S-type function is also added to the low-frequency feature component to adaptively adjust the features between low-frequency components. The S-type function is defined as follows:

[0058]

[0059] The internal feature update of the frequency domain feature convolution includes high-frequency feature update and low-frequency feature update. The high-frequency feature update process is taken as an example to illustrate. The formula for the high-frequency feature update itself and the upsampling operation from low-frequency to high-frequency feature are as follows:

[0060]

[0061] in, It is the output high-frequency feature after updating its own high-frequency frequency domain feature. is the high-frequency feature output after the low-frequency to high-frequency feature update, Upsample is the upsampling operation, X H→HX is the input feature updated from the high frequency domain to the high frequency domain. L→H M is the input feature updated from the low-frequency domain to the high-frequency domain. H is the weight generated by the convolution operation in the high-frequency domain, M L To generate the weights using convolution operation in the low-frequency domain, and The feature information is transformed into a new representation feature through the softamx function and The weight parameters s and k of the importance of the entire high-frequency feature extraction are adjusted to obtain the final high-frequency feature component output, which can be expressed as the multiplication of the weight parameter and the original high-frequency component. The formula is as follows:

[0062]

[0063] The high-frequency features and low-frequency features separated by the high- and low-frequency feature separation module are input into the high-frequency feature extraction branch and the low-frequency feature extraction branch respectively. The specific composition diagram of the depthwise separable convolution module in the high-frequency feature extraction branch is shown in the figure. Figure 4 As shown in the figure, the entire high-frequency feature extraction branch includes different two-dimensional convolutions. The deep convolutions with kernel sizes of 3×3 and 5×5 and the point convolutions with kernel size of 1×1 are combined to extract local key detail features while taking into account the lightweight requirements. The combination of deep convolution and point convolution is often used in resource-constrained scenarios such as hyperspectral image data samples, and can still achieve good classification results with limited data resources.

[0064] After being decomposed by the high- and low-frequency feature separation module, the image division Patch in the improved ViT network module can convert the input low-frequency feature information into position information. Then, an improved multi-layer interactive attention mechanism combining the hierarchical attention mechanism and the cross-attention mechanism is further adopted to enable the improved ViT network module to focus on the extraction and fusion of deep and shallow features of the image, effectively solving the problem that the traditional ViT network module cannot effectively learn feature representation, and capturing the spatial-spectral joint feature information of the hyperspectral image. The proposed improved ViT, as an image classification model that does not require decoding, often plays a greater role in the application scenario of hyperspectral image data classification with rich spatial-spectral information. The Transformer structure in the ViT unit only has an encoder component. The specific composition diagram of the improved ViT encoder is shown in the figure. Figure 6 As shown in the figure, in order to enhance the representation ability of the network and reduce the computational complexity, the image feature input feedforward network uses two-dimensional convolution with convolution kernel sizes of 1×1, 3×3 and 5×5 to further extract high-level features of the hyperspectral image.

[0065] The improved ViT network model uses the features output by the newly proposed improved multi-layer interactive attention mechanism as the input of the feedforward network in the model. By alternately applying the multi-layer attention mechanism, local and global information is captured and the diversity between layers is enhanced to further capture the low-frequency feature information of the hyperspectral image. The multi-layer interactive attention formula is as follows:

[0066]

[0067] Among them, Q is the query matrix, K is the key matrix, V is the value matrix, and d k is the dimension of the input key matrix, R Q , R K and R V are the weight matrices obtained after training, T 1 With T 2 are two sets of standardized sequences, Q n is the nth vector in the query matrix Q, is the vector Q n The transpose operation, K s is the sth vector in the key matrix K, ∑ i is a summation operation on the variable n. The value ranges of n and s are both from 1 to l, where l is the number of branches.

[0068] The hyperspectral image band reconstruction module in the spatial spectrum feature extraction branch mainly performs spectral band matching reconstruction through correlation analysis between bands, and reconstructs hyperspectral image data with higher resolution, which is convenient for achieving higher accuracy in the subsequent classification of hyperspectral images. The spectral band correlation coefficient L is used to calculate the correlation between adjacent bands. The correlation coefficient is defined as follows:

[0069]

[0070] Among them, N i and L i is the band value of the adjacent band, YesN i The average value of YesL i The average value of ∥·∥ 2 Represents the 2-norm of the vector. The closer the absolute value of the correlation coefficient L is to 1, the stronger the correlation between the bands is. The closer the absolute value of the correlation coefficient L is to 0, the weaker the correlation between the bands is.

[0071] Since the hyperspectral image dataset has many wavelengths and bands, the spectral resolution may be different. Therefore, after selecting the bands with stronger correlation according to the correlation coefficient, it is necessary to use the band matching matrix K to re-match the total set of spectral bands C, that is, the matched spectral band set can be expressed as KC. The formula for reconstructing a hyperspectral image E with higher resolution is as follows:

[0072]

[0073] Among them, Y is the hyperspectral image before reconstruction, S is the noise in the hyperspectral image, R is the spectral response function, H C is the spectral band set after matching, B is the band reconstruction spatial variation matrix, and D L is the mapping matrix corresponding to the E band of the hyperspectral image to be reconstructed.

[0074] The band matching process between the higher-resolution hyperspectral image E and the total set of spectral bands C can be achieved through the spectral band matching matrix K. The optimal solution of K can be transformed into the problem of solving the following minimum distance d:

[0075] d = argmin∥K i W C -W E ∥ 2 ;

[0076] Among them, W C and W E Respectively represent the band information of the total band set C and the hyperspectral image E to be reconstructed, ∥·∥ 2 Represents the 2-norm of the vector, i is less than the number of bands of the hyperspectral image Y before reconstruction.

[0077] The spectral band matching reconstruction is mainly carried out through correlation analysis between bands, and then spectral intelligent optimization learning is performed to reconstruct hyperspectral image data with higher resolution, so as to achieve higher accuracy in the subsequent classification of hyperspectral images;

[0078] The high-resolution hyperspectral image data is input into Figure 5In the lightweight spatial-spectral attention hybrid convolution module shown in FIG, the high-resolution hyperspectral image E obtained after band reconstruction is input into the lightweight spatial-spectral attention hybrid convolution module. The lightweight spatial-spectral attention hybrid convolution module in the spatial-spectral feature extraction branch includes three-dimensional convolution, CBAM attention mechanism, Dropout layer, SE attention mechanism, R-type activation function, average pooling layer, two-dimensional convolution and full connection layer. The CBAM attention mechanism and SE attention mechanism are inserted into the lightweight spatial-spectral attention hybrid convolution module as attention mechanisms, which can pay more attention to the key spatial-spectral features. The convolution kernel sizes of the two three-dimensional convolutions are 2×2×32 and 2×2×18 respectively. The CBAM attention mechanism module is embedded behind the first three-dimensional convolution layer. When the feature map is processed by the CBAM attention mechanism, the network training efficiency, the network's spatial-spectral feature extraction capability and nonlinearity capability can be effectively improved. The schematic diagram of the CBAM attention mechanism is shown in FIG. Figure 7 As shown in the figure, it includes two independent sub-modules, the spectral attention module and the spatial attention module, which respectively perform channel and spatial feature extraction. The spectral attention mechanism aims at the problem that the important spectral band information in the data is not paid enough attention. The spectral attention mechanism highlights the spectral information that is more valuable for hyperspectral image classification and suppresses redundant information by increasing spectral attention, thereby improving the classification accuracy of the overall model. The spectral attention mechanism adopts the R-type activation function. The spectral attention mechanism formula used in the CBAM attention mechanism is as follows:

[0079]

[0080] Where X represents the input data, Indicates the multiplication of corresponding elements, AvgPool is the average pooling operation, MaxPool is the maximum pooling operation, FC means full connection, MLP is a multi-layer perceptron with shared feature parameters, Sigmoid means the mapping from 0 to 1, and Reshape means resizing to the original size of the hyperspectral image.

[0081] The spatial attention mechanism is used to enhance the ability to extract important spatial regional feature information from hyperspectral image data and weaken the spatial features that are not related to classification, so as to solve the problem that ordinary convolution ignores important spatial information of hyperspectral images. Channel-based maximum pooling and average pooling are performed to assign weights to features of different spatial positions. Then, the two feature maps that will be obtained after pooling are convolved. The convolution window is 7×7, the convolution kernel depth is 3, and the pixel values ​​are reassigned according to the respective weights of each spatial position. The specific formula of the spatial attention mechanism module is as follows:

[0082] M s (F) = Sigmoid(f 7×7([AνgPool(F);MaxPool(F)]));

[0083] Among them, f 7×7 is a convolution operation with a convolution window size of 7×7, AvgPool is an average pooling operation, MaxPool is a maximum pooling operation, Sigmoid represents a mapping from 0 to 1, and F is an input feature.

[0084] After the CBAM attention mechanism module, an R-type activation function is introduced. The average pooling uses a three-dimensional average convolution pooling layer with a pooling step size of 2×2×2. The extracted important spatial spectrum features are compressed and the amount of calculation is reduced to meet the design of lightweight model construction. The feature output is then used as the feature input of the second three-dimensional convolution layer embedded in the SE attention mechanism module (improved). The SE attention mechanism composition diagram is shown in the figure below. Figure 8 As shown in the figure, in the improved SE module, the compression and extraction operation of the features is implemented through global average pooling and global maximum pooling, which meets the construction of the lightweight model and reduces the amount of calculation. The output data is compressed into two sets of channel description vectors with different global receptive fields through the feature channel. The data output by the three-dimensional convolution is activated to extract the correlation between each feature channel. The activation operation associates the information between the spatial features and the spectral features of the two sets of vectors obtained by the global average pooling and the global maximum pooling operations through the fully connected layer and activates them using the R-type activation function and the Sigmoid function. After the two sets of vectors are fused, a one-dimensional convolution is used to give each channel a specific weight to indicate the attention of different channels, and finally a feature data showing important information is output.

[0085] In the second three-dimensional convolution, Dropout regularization is used to randomly inactivate neurons in the hidden layer, which has the effect of preventing overfitting to a certain extent. Then, by adding activation functions and two fully connected layers to the two-dimensional convolution, the extracted spatial spectrum features are output as a one-dimensional vector, which facilitates the subsequent multi-layer perception module to integrate high-frequency features, low-frequency features and spatial spectrum features.

[0086] like Fig. 9 As shown in FIG. 1 , the multi-layer perception module includes an input layer, a fully connected layer, an activation function layer, a hidden layer, and an output layer. The activation function adopts a G-type function, and the approximate calculation formula of the G-type activation function is:

[0087]

[0088] Among them, X represents the feature input.

[0089] The classification method of the present invention finally fuses the high-frequency features extracted by the high-frequency feature extraction branch, the low-frequency features extracted by the low-frequency feature extraction branch, and the space spectrum features extracted by the space spectrum feature extraction branch through the constructed multi-layer perception module. The output categories of the classification results output by the multi-layer perception module are as follows:

[0090] Class=softmax(GELU(FC(X));

[0091] Among them, softmax represents the classification function, GELU is a nonlinear G-type activation function based on Gaussian error linear unit, and X represents the feature input.

[0092] S3. Training network model: Use sample data from Indian Pines dataset, Pavia University dataset and KSC dataset as training sets to train the network model;

[0093] Specifically, 2% of the sample data in the Indian Pines dataset is used as the training set, 1% of the sample data in the Pavia University dataset is used as the training set, and 0.5% of the sample data in the KSC dataset is used as the training set. The number of training times is set to 150, and the number of images input to the network each time is 30. The three training sets are trained separately.

[0094] S4. Determine the evaluation index: set the learning rate, select the network optimizer, and determine the hyperspectral image classification accuracy performance evaluation index and lightweight performance evaluation index;

[0095] The evaluation index of this method is determined. The learning rate of the training process is set to 0.0005. The Adam optimizer is selected as the network optimizer. Its advantages mainly lie in optimizing the network and improving the computational efficiency, making the parameters relatively stable, and being able to adapt to sparse gradients and alleviate the problem of gradient oscillation.

[0096] The overall accuracy, average accuracy and consistency coefficient are selected as the performance evaluation indicators of hyperspectral image classification accuracy, which can effectively evaluate the quality of classification and measure the role of classification network. Among them, the overall accuracy is an indicator to measure the overall classification accuracy of the classification model, and the average accuracy is an indicator to show the classification accuracy of the classification model for a certain category. The calculation formulas of overall accuracy OA and average accuracy AA are as follows:

[0097]

[0098] Among them, TP is the positive sample correctly classified by the model, FN is the positive sample incorrectly classified by the model, FP is the negative sample incorrectly classified by the model, and TN is the negative sample correctly classified by the model.

[0099] The consistency coefficient indicates the consistency between the predicted value and the true value, and is used to measure the effect of classification. The calculation formula of the consistency coefficient Kappa is as follows:

[0100]

[0101] Where C is the total number of categories, T i is the number of samples correctly classified in each category, a i is the number of true samples of each class, b i is the predicted number of samples of each category, and n is the total number of samples.

[0102] The lightweight performance evaluation index of hyperspectral image classification selects floating-point calculation amount and parameter amount. The floating-point calculation amount is used to observe the length of network execution time, and the parameter amount is used to measure the amount of overall video memory occupied. Both are indicators that indicate the classification efficiency and calculation amount of the classification model. The calculation formulas of floating-point calculation amount FLOPs and parameter amount Params are as follows:

[0103] FLOPs = C i ×k 2 ×C o ×W×H;

[0104] Params=C o ×(k w ×k h ×C i +1);

[0105] Among them, C o is the number of spectral bands of the output hyperspectral image data, C i is the number of spectral bands of the hyperspectral image data, k w represents the convolution kernel width, k h Indicates the convolution kernel height. k w ×k h ×C i Represents the number of weights of a convolution kernel, W represents the width of the output hyperspectral image data, and H represents the height of the output hyperspectral image data.

[0106] S5. Intelligent optimization model: Use image datasets to train and fine-tune the model, use the new intelligent optimization model algorithm LSCOA to optimize the model, compare the classification results through evaluation indicators until the conditions are met, and output the final classification results.

[0107] The model is trained and fine-tuned using image datasets. The widely applicable sine-cosine optimization algorithm SCA has a more adaptable and stable design for global search and local development in the optimization of hyperspectral image classification models. However, the satisfaction of the final solution fluctuates greatly around the theoretical optimal solution and the convergence of the overall algorithm needs to be improved. The spectral optimization algorithm LSO has the characteristics of fast convergence speed and stable convergence performance when applied to optimization problems. It effectively simulates the spectrum distribution and peak search process in spectral analysis. The algorithm can quickly and accurately find the optimal solution by adaptively adjusting the resolution of the search space, which can perfectly remedy the shortcomings of the sine-cosine optimization algorithm SCA. Therefore, a new intelligent optimization model algorithm LSCOA combining two intelligent optimization algorithms is proposed.

[0108] Among them, the spectral optimization algorithm LSO is inspired by the dispersion phenomenon in physics. It can effectively utilize the mathematical formulas of reflection, refraction and dispersion, and present diversity in the updating process of the hyperspectral image classification network model to maintain the diversity of the iterative population.

[0109] In the LSCOA algorithm, the position of an individual is used to represent a potential solution to the optimization problem. The entire population flies in the search space to find the global optimal solution. The current optimal individual position is recorded by calculating the fitness value of each individual. The cycle continues until the termination condition is met and the optimal solution is output. In each iteration, the optimal individual position is constantly updated. The specific complete position update iteration equation formula is as follows:

[0110]

[0111] in, is the position of the current particle i when it is updated by the new intelligent optimization model algorithm LSCOA at the k+1th iteration, is the position of the current particle i when it is updated by the kth iteration of the sine-cosine optimization algorithm SCA; is the historical individual optimal solution of the current particle, i.e., the target point. sin(·) is the sine function, cos(·) is the cosine function, and s 1 、s 2 、s 3 and 4 are four random solutions randomly selected from the current population, s 1 is the control parameter, s 2 is a random number with uniform distribution in the range of 0 to 2π, s 3 is a random number with uniform distribution in the range of 0 to 2, s 4 is a random number with uniform distribution in the range of 0 to 1, |·| represents the absolute value, is the position of the spectral band candidate solution newly generated when the current particle i is updated by the spectral optimization algorithm LSO for the kth iteration, μ is the ratio parameter of the optimized band to all bands, A vector of random numbers between 0 and 1.

[0112] Control parameters 1 The calculation formula is as follows:

[0113]

[0114] Among them, λ is a random constant, T is the maximum number of iterations, and t is the current number of iterations.

[0115] The calculation formula of the parameter μ, which is the ratio of the optimized band to all bands, is as follows:

[0116]

[0117] Among them, RW 2 is a vector of uniform random numbers generated in the range of 0 to 1. is a normally distributed random number vector with 0 as mean and 1 as standard deviation, t is the current iteration number, T max is the maximum number of iterations for the classification model.

[0118] After fine-tuning the parameters, a better hyperspectral image classification model is selected. The hyperspectral image classification network model is continuously intelligently optimized and iteratively updated. The classification results are compared through evaluation indicators to observe whether the classification model meets the optimal model conditions. If the optimal model conditions are met, the optimal solution with high fitness is output and the model is saved, and the final classification result is output; if the optimal model conditions are not met, the number of iterations is increased by one, and the classification network model continues to be intelligently optimized and updated, the network model is retrained and the parameters are automatically adjusted until the optimal model conditions are met, the classification model is saved and the final classification result is output.

[0119] In this embodiment, the control parameter s is optimized by intelligent 1 The random constant λ in is set to 3, the learning rate is set to 0.0005, and a total of 1500 rounds of iterations are performed. It is found that the lightweight hyperspectral image classification network based on intelligent optimization in this embodiment has the best classification performance, and the network model converges quickly and has stable convergence performance. High-quality classification image results are obtained by retraining, which further improves the accuracy of the classification network.

[0120] S6, save the model, and after the training is completed, solidify the fine-tuned network parameters, and wait for S5 to fine-tune the model to determine the optimal hyperspectral image classification network model after intelligent optimization and iterative update. If a hyperspectral image classification task is performed, the hyperspectral image can be directly input into the iteratively updated classification network model after preprocessing to obtain the final image classification result.

[0121] 1. Example 1 related effect proof:

[0122] The embodiments of the present invention have achieved some positive effects during the development or use process, and indeed have great advantages over the prior art. The following content is described in conjunction with data, charts, etc. of the test process.

[0123] The comparison of the related indicators of the existing technical methods and the method proposed in the present invention on the Indian Pines dataset, the Pavia University dataset and the KSC dataset is shown in Table 1, Table 2 and Table 3 respectively. The existing method 1 adopts the SSFTT hyperspectral image classification method which organically integrates the backbone CNN and Transformer structures and has good comprehensive classification performance. The existing method 2 adopts the 3D-CNN hyperspectral image classification method which is mainly based on the three-dimensional convolution kernel and can achieve accurate classification. The overall accuracy, average accuracy and consistency coefficient of the classification method proposed in the present invention on the Indian Pines dataset, the Pavia University dataset and the KSC dataset are higher, and the floating point calculation amount and parameter amount are less than other mainstream methods, indicating that the classification efficiency of the lightweight hyperspectral image classification network model based on intelligent optimization proposed in this embodiment is effectively improved and the calculation amount is reduced. These indicators show that the technical method proposed in the present invention has better classification effect and higher classification efficiency, and it does have great advantages compared with the existing technology, and has achieved satisfactory expected results.

[0124] Table 1 Comparison of classification-related indicators of the Indian Pines dataset:

[0125]

[0126]

[0127] Table 2 Comparison of classification-related indicators of Pavia University dataset:

[0128]

[0129] Table 3 Comparison of KSC dataset classification related indicators:

[0130]

[0131] Embodiment 2:

[0132] A lightweight hyperspectral image classification system based on intelligent optimization, used to execute the lightweight hyperspectral image classification method based on intelligent optimization of embodiment 1, comprising:

[0133] Data preparation module, used to prepare and annotate hyperspectral image datasets;

[0134] A network model building module is used to build a lightweight hyperspectral image classification network model including a high-frequency feature extraction branch, a low-frequency feature extraction branch, and a spatial spectrum feature extraction branch;

[0135] The training module is used to select some sample data from the data set as a training set to train the network model;

[0136] An evaluation index determination module is used to set the learning rate, select the network optimizer, and determine the hyperspectral image classification accuracy performance evaluation index and lightweight performance evaluation index;

[0137] The intelligent optimization module is used to train and fine-tune the model using image datasets. The model is optimized using the new intelligent optimization model algorithm LSCOA that combines the sine-cosine optimization algorithm SCA and the spectral optimization algorithm LSO. The classification results are compared through evaluation indicators until the optimal model conditions are met and the final classification results are output.

[0138] Furthermore, the network model building module includes a high- and low-frequency feature separation module, a deep separable convolution module, an improved ViT network module, a hyperspectral image band reconstruction module, a lightweight spatial-spectral attention hybrid convolution module and a multi-layer perception module.

[0139] Furthermore, the intelligent optimization module adopts a new intelligent optimization model algorithm LSCOA, combined with a sine-cosine optimization algorithm SCA and a spectrum optimization algorithm LSO, to iteratively update individual positions and adaptively adjust the resolution of the search space to find the optimal solution.

[0140] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A lightweight hyperspectral image classification method based on intelligent optimization, characterized in that: The following steps are involved: S1, prepare datasets: hyperspectral image datasets including Indian Pines dataset, Pavia University dataset and KSC dataset, annotate each dataset and use it for hyperspectral image classification; S2, building a network model: extracting features through high-frequency feature extraction branch, low-frequency feature extraction branch and empty spectrum feature extraction branch; It includes high- and low-frequency feature separation module, deep separable convolution module, improved ViT network module, hyperspectral image band reconstruction module, lightweight spatial-spectral attention hybrid convolution module and multi-layer perception module; S3, training network model: using sample data from the Indian Pines dataset, Pavia University dataset, and KSC dataset as training sets, the network model is trained; S4, determine the evaluation index: set the learning rate, select the network optimizer, and determine the hyperspectral image classification accuracy performance evaluation index and lightweight performance evaluation index; S5, intelligent optimization model: The new intelligent optimization model algorithm LSCOA is used to optimize the model, and the classification results are compared through evaluation indicators until the conditions are met, and the final classification results are output.

2. The lightweight hyperspectral image classification method based on intelligent optimization according to claim 1 is characterized in that: The high- and low-frequency feature separation module includes a frequency domain feature convolution, a normalization layer, and an R-type activation function layer, which is used to decompose the hyperspectral image data into high-frequency features and low-frequency features.

3. The lightweight hyperspectral image classification method based on intelligent optimization according to claim 1 is characterized in that: The improved ViT network module includes image segmentation Patch, improved multi-layer interactive attention mechanism, position encoding and Transformer encoder. The deep features and shallow features of the generated image are fused through splicing operation with the multi-layer interactive attention mechanism as the core, and more attention is paid to the correlation of information between layers, so as to effectively extract global features while reducing the complexity of calculation.

4. The lightweight hyperspectral image classification method based on intelligent optimization according to claim 1 is characterized in that: The hyperspectral image band reconstruction module performs spectral band matching reconstruction through correlation analysis between bands, and then performs spectral intelligent optimization learning to reconstruct hyperspectral image data with higher resolution.

5. The lightweight hyperspectral image classification method based on intelligent optimization according to claim 1 is characterized in that: The lightweight spatial-spectral attention hybrid convolution module includes three-dimensional convolution, CBAM attention mechanism, Dropout layer, SE attention mechanism, R-type activation function, average pooling layer, two-dimensional convolution and fully connected layer, which are used to extract spatial-spectral features of hyperspectral images; wherein, the CBAM attention mechanism includes a spectral attention module and a spatial attention module.

6. The lightweight hyperspectral image classification method based on intelligent optimization according to claim 1 is characterized in that: The multi-layer perception module includes an input layer, a fully connected layer, a G-type activation function layer, a hidden layer and an output layer, which are used to fuse high-frequency features, low-frequency features and spatial spectrum features, and output classification results.

7. The lightweight hyperspectral image classification method based on intelligent optimization according to claim 1 is characterized in that: The new intelligent optimization model algorithm LSCOA combines the sine-cosine optimization algorithm SCA and the spectrum optimization algorithm LSO, updates individual positions iteratively, and adaptively adjusts the resolution of the search space to find the optimal solution.

8. A lightweight hyperspectral image classification system based on intelligent optimization, used to execute the lightweight hyperspectral image classification method based on intelligent optimization as described in any one of claims 1 to 7, characterized in that: include: Data preparation module, used to prepare and annotate hyperspectral image datasets; A network model building module is used to build a lightweight hyperspectral image classification network model including a high-frequency feature extraction branch, a low-frequency feature extraction branch, and a spatial spectrum feature extraction branch; The training module is used to select some sample data from the data set as a training set to train the network model; An evaluation index determination module is used to set the learning rate, select the network optimizer, and determine the hyperspectral image classification accuracy performance evaluation index and lightweight performance evaluation index; The intelligent optimization module is used to train and fine-tune the model using image datasets. The model is optimized using the new intelligent optimization model algorithm LSCOA that combines the sine-cosine optimization algorithm SCA and the spectral optimization algorithm LSO. The classification results are compared through evaluation indicators until the optimal model conditions are met and the final classification results are output.

9. The lightweight hyperspectral image classification system based on intelligent optimization according to claim 8, characterized in that: The network model building module includes a high- and low-frequency feature separation module, a deep separable convolution module, an improved ViT network module, a hyperspectral image band reconstruction module, a lightweight spatial-spectral attention hybrid convolution module and a multi-layer perception module.

10. The lightweight hyperspectral image classification system based on intelligent optimization according to claim 8, characterized in that: The intelligent optimization module adopts a new intelligent optimization model algorithm LSCOA, combined with a sine-cosine optimization algorithm SCA and a spectrum optimization algorithm LSO, and iteratively updates individual positions and adaptively adjusts the resolution of the search space to find the optimal solution.

Images