Hyperspectral image classification method based on decoupling Gabor network

Abstract

The invention relates to the field of image classification, in particular to a hyperspectral image classification method based on a decoupling Gabor network. Two one-dimensional Gabor kernels along an orthogonal space axis are adopted, each one-dimensional Gabor kernel comprises a decoupling Gabor filter in the x direction and a decoupling Gabor filter in the y direction, a decoupling Gabor convolution module is constructed based on the decoupling Gabor filters, and specific redundant parameters of channels are greatly reduced while the effective characterization capability is reserved by sharing Gabor parameters and a parameter scattering scheme between the channels, so that the performance of the system is improved. And the overall complexity of the network is obviously reduced. And through a decoupling Gabor convolution module, a global average pooling layer and a multiple perceptron layer are combined to obtain a decoupling Gabor network. After input data is preprocessed, a network is trained, and finally, a hyperspectral image is put into the network to realize a classification effect. According to the method, efficient deployment is realized in a resource-limited hardware environment, the calculation cost and model parameters are remarkably reduced when high-dimensional data are processed, and the training and reasoning efficiency of the network is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of hyperspectral remote sensing image classification, and particularly relates to a hyperspectral image classification method based on a decoupled Gabor network. BACKGROUND

[0002] Hyperspectral image is a multi-dimensional data containing spatial and continuous spectral information, which records the spectral characteristics of each pixel in multiple bands and can accurately reflect the material characteristics and spectral characteristics of objects. It is widely used in the fields of earth science research, precision agriculture, climate change, environmental monitoring and national defense and military.

[0003] However, deep learning models face unique challenges in hyperspectral image classification: hyperspectral data has high-dimensional characteristics of hundreds of continuous spectral bands, and labeled samples are often scarce in practical applications, which leads to model overfitting and significantly increased computational cost of complex networks. To address these challenges, Gabor filters inspired by biological vision have become an important tool for improving the robustness and interpretability of deep learning models due to their outstanding performance in time-frequency analysis.

[0004] Gabor filtering in hyperspectral image classification mainly includes traditional Gabor filtering methods, Gabor filtering methods based on convolutional networks, and naive Gabor networks. Generally, the traditional designed Gabor filter shows strong frequency domain and spatial positioning ability in image processing, but has significant limitations. These limitations include dependence on manual parameter tuning and lack of automatic optimization, sensitivity to noise leading to reduced feature extraction, and fixed filter shape and parameters limiting adaptability to diverse image features, especially in handling scale and rotation variations. These defects weaken its applicability in complex image processing tasks, highlighting the urgency of improvement or replacement through advanced technologies. Secondly, Gabor filtering methods based on convolutional networks, although introducing Gabor into convolutional networks reduces the number of parameters and gives convolutional networks direction selectivity and frequency selectivity, still learn regular convolution kernels, and in fact use Gabor-like style kernels. These Gabor parameters are fixed and cannot be learned and updated autonomously, which may ignore nonlinear or complex features in images and also cause the possibility of redundant information. Finally, the naive Gabor network directly replaces the ordinary convolution kernel with the Gabor kernel, but may require more computational resources to train the learnable coupled Gabor kernel, increasing the complexity of the model and possibly slowing down the training process.

[0005] In hyperspectral image classification, most current methods require large parameter quantity and more network layers to extract feature information, which also consumes a lot of hardware resources, and even reduces the training speed and inference efficiency on large-scale hyperspectral data sets. To address these challenges, a method is needed that can significantly reduce network parameter quantity, improve training efficiency and speed up inference while ensuring classification effect. SUMMARY

[0006] To solve the problems of large parameter quantity, large hardware resource consumption, and even low training speed and inference efficiency on large-scale hyperspectral data sets in existing image classification methods, the present application provides a hyperspectral image classification method based on a decoupled Gabor network, which is used in the context of hyperspectral image classification to efficiently deploy in a resource-constrained hardware environment and significantly reduce computational cost and model parameters when processing high-dimensional data (such as hyperspectral images), thereby improving the training and inference efficiency of the network. The method mainly includes: S1: decomposed into two one-dimensional Gabor kernels along the orthogonal spatial axis, including a decoupled Gabor filter in the x direction and a decoupled Gabor filter in the y direction; S2: pass through the two decoupled Gabor filters in the x and y directions, and add a nonlinear layer and a batch normalization layer to obtain a decoupled Gabor convolution module; S3: introduce channel sharing parameters in the parameters of the decoupled Gabor filter, and design a scattering combination scheme for the parameters in the decoupled Gabor convolution module to further optimize the decoupled Gabor convolution module; S4: based on the optimized decoupled Gabor convolution module, combine a global average pooling layer and a multi-perception machine layer to obtain a decoupled Gabor network; S5: obtain a hyperspectral image data set, preprocess the data set, and divide it into a training set and a test set, train the decoupled Gabor network using the training set to obtain the trained network weight, and use the test set to output a prediction evaluation result through the trained decoupled Gabor network for classification of the actual obtained hyperspectral image.

[0007] A computer device includes a memory, a processor, and a computer program stored on the memory, the processor executes the computer program to implement the steps of the above method.

[0008] A computer readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the above method.

[0009] A computer program product includes a computer program or instructions, which, when executed by a processor, implements the steps of the above method.

[0010] The beneficial effects brought by the technical solutions provided by the present application are: the present application designs a brand-new decoupled Gabor filter, adopts two one-dimensional Gabor kernels along the orthogonal space axes, and contains an x-direction decoupled Gabor filter and a y-direction decoupled Gabor filter, and the reasoning complexity is significantly reduced through the cascading operation of the direction convolution. Based on the decoupled Gabor filter, the present application constructs a decoupled Gabor convolution module, which, through the inter-channel sharing of Gabor parameters and the parameter scattering scheme, significantly reduces the channel-specific redundant parameters while retaining the effective representation capability, and significantly reduces the overall complexity of the network. Through the constructed decoupled Gabor convolution module, combined with the global average pooling layer and the multiple perception machine layer, the decoupled Gabor network is obtained. After the input data is preprocessed, it is put into the network for training, and the model of different classification features can be obtained, and finally the hyperspectral image is put into the network to realize the classification effect. The decoupled Gabor network proposed in the present application has significant advantages in training efficiency, parameter size and reasoning speed compared with the existing advanced hyperspectral image classification methods. This method not only can process large-scale hyperspectral images more quickly and efficiently, but also has better performance in classification accuracy, which fully reflects its wide potential in practical applications. BRIEF DESCRIPTION OF DRAWINGS

[0011] The present application will be further described below in conjunction with the drawings and examples, wherein: Figure 1 is a flowchart of a hyperspectral image classification method based on a decoupled Gabor network in an embodiment of the present application; Figure 2 is a training flowchart of a decoupled Gabor network in an embodiment of the present application; Figure 3 is a schematic diagram of a decoupled Gabor convolution module in an embodiment of the present application; Figure 4 is a schematic diagram of a channel sharing parameter mechanism of a decoupled Gabor convolution module in an embodiment of the present application; Figure 5 is a scattering combination and scattering transformation of a decoupled Gabor convolution module in an embodiment of the present application; Figure 6 is a schematic diagram of the overall network structure of a decoupled Gabor network in an embodiment of the present application. DETAILED DESCRIPTION

[0012] In order to have a clearer understanding of the technical features, purposes and effects of the present application, the specific embodiments of the present application will be described in detail with reference to the drawings.

[0013] Example 1 Please refer to Figure 1 , Figure 1 is a flowchart of a hyperspectral image classification method based on a decoupled Gabor network in an embodiment of the application, specifically comprising: S1: design decoupled Gabor filter Two one-dimensional Gabor kernels decomposed along the orthogonal spatial axis contain a decoupled Gabor filter in the x direction and a decoupled Gabor filter in the y direction.

[0014] S2: Based on the decoupled Gabor filter, a decoupled Gabor convolution module is designed Through the two decoupled Gabor filters in the x and y directions, and by adding a nonlinear layer and a batch normalization layer, a decoupled Gabor convolution module is obtained, which can reduce the network complexity and improve the feature generalization ability at the same time.

[0015] Figure 3 is a schematic diagram of the decoupled Gabor convolution module of the application. Among them, is the patch size, m is the number of decoupled Gabor convolution modules, is the channel number output of the patch block, represents the decoupled Gabor kernel family parameters. It shows the parameter transformation of 2D Gabor after De-Gabor, and also gives the internal design structure of the decoupled Gabor convolution module. First, through the decoupled Gabor kernel in the x direction, then through the nonlinear layer and the batch normalization layer, the feature extraction in the x direction is realized, then, through the decoupled Gabor kernel in the y direction, and finally through the nonlinear layer and the batch normalization layer, the feature extraction in the y direction is realized.

[0016] S3: In order to further reduce the number of parameters in the decoupled Gabor convolution module, channel sharing parameters are introduced into the parameters in the decoupled Gabor, and a scattering combination scheme is designed for the parameters in the decoupled Gabor convolution module, obtaining an extremely efficient decoupled Gabor convolution module.

[0017] Figure 4 is an example diagram of the channel sharing parameter mechanism of the decoupled Gabor convolution module. By reducing the redundant parameters between different channels, complex parameter optimization on a large number of channels can be avoided, thereby effectively reducing the model calculation complexity and improving the training efficiency and generalization performance in high spatial resolution classification tasks.

[0018] Figure 5 Scattering combination and scattering transformation schematic diagram of decoupled Gabor convolution module. The red bubbles represent scattering coefficients, and * represents a convolution operator. The feature embedding of the nth output channel is represented. The decoupled Gabor convolution module in the application performs a scattering transform on the input features in a decoupled manner, further improving the diversity of parameters and the decoupled Gabor kernel.

[0019] S4: Based on the optimized decoupled Gabor convolution module, a global average pooling layer and a multi-perception layer are combined to obtain a decoupled Gabor network (DeGaborNet). S5: As shown in the figure, a hyperspectral image dataset is obtained, the dataset is preprocessed and divided into a training set and a test set, the training set is used to train the decoupled Gabor network, and the trained network weight is obtained; the test set is used to output a prediction evaluation result through the trained decoupled Gabor network, which is used for classifying the actual acquired hyperspectral image. Figure 2

[0020] The decoupled Gabor filter in step S1 is to decompose the phase in the naive Gabor network into two independent components along the spatial coordinate axes x and y directions and . Therefore, the two-dimensional naive Gabor filter can be decomposed into a combination of one-dimensional form, as follows:

[0021] wherein, the phase of the Gabor kernel is represented by ,

[0022] At the same time, the formula can be simplified as:

[0023] wherein

[0024] According to the above formula and the trigonometric function, we have:

[0025] wherein the subscripts and indicate that the two decoupled Gabor filters can be replaced.

[0026] According to the above derivation process, the decoupled Gabor filter can be obtained as follows:

[0027] The decoupled Gabor filter has strong advantages in computational efficiency, flexibility, information integrity, etc. The convolution formula obtained according to the decoupled Gabor filter is as follows:​

[0028] wherein, By using decoupled Gabor convolution kernels, the convolution operation can be performed in two spatial directions sequentially.

[0029] The decoupled Gabor convolution module (De-GCM) in step S2 is designed as: Firstly by decoupled Gabor convolution module about x direction, secondly by nonlinear layer and batch normalization layer, then by decoupled Gabor convolution module about y direction, and finally by nonlinear layer and batch normalization layer. Specifically, let and denote the th channel of the th kernel group in the horizontal (x) direction (De-GCM-x) and the horizontal (y) direction (De-GCM-y) of the th decoupled Gabor convolution module, respectively. Thus, the th decoupled Gabor convolution module (De-GCM) can be mathematically defined as:

[0030]

[0031]

[0032]

[0033] wherein, and and are the bias terms of the decoupled Gabor kernels in two directions, respectively. is the normalization layer, is the activation function, defined as: . In order to further reduce the number of parameters in De-GCM, the shared parameters across channels are used in step S3 to reduce the redundant parameters across channels. Let the decoupled Gabor kernels in two directions denote the convolution layers in two directions in the decoupled Gabor convolution module, and the number of output channels is set to , which means that each layer contains groups of convolution kernels. The parameter size of the decoupled Gabor kernel in the decoupled Gabor convolution module is

[0034] ​​​​Therefore, the decoupled Gabor convolutional module proposed in this invention has the potential to reduce the size of existing network parameters, thereby alleviating the difficulty of optimizing redundant parameters during training. However, due to the limitation of channel-shared parameters, this invention proposes a scattering combination scheme for the parameters in the decoupled Gabor convolutional module. This involves constraining the decoupled Gabor kernels in the x-direction and y-direction... Sharing the same values ​​allows for paired combinations between the two layers. Specifically, assume... ,and Decoupling Gabor convolution modules in Multi-scale feature information can be extracted from the scattering combination, and its mathematical expression is as follows:

[0035] On the other hand, other parameters in the decoupled Gabor cores in both directions can also be configured and updated independently, further enhancing the parameter diversity of the decoupled Gabor cores.

[0036] The decoupled Gabor network in step S4 generates deep multi-scale features by executing multiple decoupled Gabor convolutional modules. , in Indicates the first The operation of a decoupled Gabor convolution module Indicates the first Features obtained from decoupled Gabor convolutional modules Indicates the first Features obtained from decoupled Gabor convolutional modules.

[0037] In a decoupled Gabor network, the first The number of output channels of each decoupled Gabor convolution module is set to Assume the size of the input feature map is... After the first layer of the decoupled Gabor convolutional module, the size of the output feature map is... This size serves as the input to the second decoupled Gabor convolutional module. After passing through the second decoupled Gabor convolutional module, the size of the feature map is... And so on, the size of the output after the m-th decoupled Gabor convolutional module layer is .

[0038] Finally, first use a global average pooling layer (GAP) to... The input feature maps are rearranged into a single array containing... a vector of elements to reduce the number of parameters of the fully connected layers. Next are multiple perception layers, the output number of the first fully connected layer is twice the input number, and the output number of the second fully connected layer is equal to the number of predefined categories for the classification target. The specific classification category is finally obtained through the softmax function, and the mathematical expression is as follows:

[0039] wherein GAP() represents a global average pooling operation, represents the feature generated by the mthdecoupled Gabor convolution module, are trainable parameters of two fully connected layers.

[0040] Figure 6 is an overall network structure diagram of the decoupled Gabor network of the application. The diagram shows a decoupled Gabor network structure for hyperspectral image classification. The model first extracts a fixed-size local patch from the original hyperspectral image as input, gradually extracts directional and frequency perception features through multiple decoupled Gabor convolution modules, and deepens the channel number layer by layer. Then, the spatial information is compressed through global average pooling, and the extracted features are sent to the multiple perception layer for classification, and finally the probability result of the corresponding category is output. The architecture significantly improves the training efficiency and inference time while ensuring the generalization ability of feature modeling, and is suitable for efficient classification of high-dimensional remote sensing data.

[0041] Table 1 lists the time required for two networks to generate classification prediction maps on four data sets.

[0042]

[0043] The experimental results show that, compared with the naive Gabor network, the method of the application performs excellently in time efficiency. Especially in terms of computing resources and time consumption, the decoupled Gabor network achieves about 50% acceleration, which has a significant advantage for processing large-scale hyperspectral image data sets. This result verifies that the decoupled decoupled Gabor network can effectively reduce the computational complexity and improve the model training speed, especially when processing complex data sets.

[0044] Table 2 lists the parameter amount comparison of two networks under different modules.

[0045]

[0046] The experimental results show that, compared with the naive Gabor network, the parameter amount of the method of the application is reduced by about 60%, and the reduction of the parameter amount is more significant with the increase of the decoupled Gabor convolution module.

[0047] Embodiment 2 A computer device comprising a memory, a processor and a computer program stored on the memory, the processor executing the computer program to implement the steps of the above method.

[0048] Embodiment 3 A computer readable storage medium storing a computer program which, when executed by a processor, implements the steps of the above method.

[0049] Embodiment 4 A computer program product comprising a computer program or instructions which, when executed by a processor, implements the steps of the above method.

[0050] The above description is merely the preferred embodiments of the present application, and is not used to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A hyperspectral image classification method based on decoupled Gabor networks, characterized in that, include: S1: Decomposed into two one-dimensional Gabor kernels along the orthogonal spatial axes, containing a decoupled Gabor filter in the x-direction and a decoupled Gabor filter in the y-direction; S2: By using two decoupled Gabor filters in the x and y directions, and adding nonlinear layers and batch normalization layers, a decoupled Gabor convolution module is obtained; S3: Introduce channel-shared parameters into the parameters of the decoupled Gabor filter, and design a scattering combination scheme for the parameters in the decoupled Gabor convolution module to further optimize the decoupled Gabor convolution module; S4: Based on the optimized decoupled Gabor convolutional module, combined with the global average pooling layer and the multiple perceptron layer, the decoupled Gabor network is obtained; S5: Obtain the hyperspectral image dataset, preprocess the dataset, and divide it into training and test sets. Use the training set to train the decoupled Gabor network and obtain the trained network weights. Using the test set, the predicted evaluation results are output by the trained decoupled Gabor network and used to classify the acquired hyperspectral images.

2. The hyperspectral image classification method based on decoupled Gabor networks as described in claim 1, characterized in that, In S1, the decoupling Gabor filter for: in To decouple the parameters of the Gabor kernel, , In a naive Gabors network, the phase is decomposed into two independent components along the x and y axes of the spatial coordinate system. This represents a decoupling Gabor filter in the horizontal direction. This represents a decoupling Gabor filter in the vertical direction. Indicates the horizontal direction of the coordinate axes. Indicates the vertical direction of the coordinate axes. , It is the frequency value of the decoupling Gabor filter. It is the direction of the filter. This represents the width of the Gaussian envelope; The convolution formula obtained from the decoupled Gabor filter is as follows: in, By using decoupled Gabor convolution kernels, convolution operations are performed sequentially in two spatial directions.

3. The hyperspectral image classification method based on decoupled Gabor networks as described in claim 1, characterized in that, The decoupled Gabor convolution module is: in They represent the first In the first De-GCM-x and De-GCM-y, the first The first core group One channel, , Indicates the first l Number of output channels of each convolutional layer Indicates the image height. Indicates the image width, and and These are the bias terms of the decoupled Gabor convolutions in the x and y directions, respectively. It is a normalization layer. It is an activation function.

4. The hyperspectral image classification method based on a decoupled Gabor network as described in claim 2, characterized in that, The scattering combination scheme is as follows: By constraining the decoupled Gabor kernel in the x-direction and the decoupled Gabor kernel in the y-direction Sharing the same values ​​allows for paired combinations between two layers, decoupling the Gabor convolutional modules. The mathematical expression for extracting multi-scale feature information from scattering combinations is as follows: in, This represents the width of the Gaussian envelope in the horizontal direction. This represents the width of the Gaussian envelope in the vertical direction. , This represents the width of the nth Gaussian envelope, where n represents the number of Gaussian envelopes; Other parameters in the decoupled Gabor kernels in both directions can also be configured and updated independently, thus forming a scattering structure different from that of the two-dimensional Gabor kernel.

5. The hyperspectral image classification method based on a decoupled Gabor network as described in claim 1, characterized in that, S4 specifically refers to; S4.1: Generate multi-scale features by executing multiple decoupled Gabor convolutional modules: ， in Indicates the first The operation of a decoupled Gabor convolution module Indicates the first Features obtained from decoupled Gabor convolutional modules Indicates the first The features obtained from decoupled Gabor convolutional modules, where m represents the number of decoupled Gabor convolutional modules; S4.2: The obtained features are input into a global average pooling layer and a multi-perceptron layer, the extracted information is output, and then the classification result is obtained through the softmax function: Wherein, GAP() represents the global average pooling operation. This represents the feature generated by the m-th decoupled Gabor convolutional module. These are the trainable parameters of two fully connected layers.

6. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes a computer program to implement the steps of the hyperspectral image classification method based on decoupled Gabor networks as described in any one of claims 1-5.

7. A computer-readable storage medium, characterized in that, The device contains a computer program that, when executed by a processor, implements the steps of the hyperspectral image classification method based on a decoupled Gabor network as described in any one of claims 1-5.

8. A computer program product, characterized in that, Includes a computer program or instructions that, when executed by a processor, implement the steps of the hyperspectral image classification method based on a decoupled Gabor network as described in any one of claims 1-5.

Images