Hyperspectral image classification method and system based on hybrid convolutional neural network

Abstract

The invention provides a hyperspectral image classification method and system based on a hybrid convolutional neural network, and belongs to the technical field of image processing. Comprising the following steps: carrying out band selection on a historical data set, dividing the historical data set in a Patch form, and taking the divided historical data set as a prepared training sample; determining a training sample based on the proportion setting; training the hybrid convolutional neural network, that is, performing shallow feature extraction and fusion on the images in the training samples by using a context interaction network to obtain a first feature map; inputting the first feature map into a 3D-2D hybrid neural network, and performing deep feature extraction; adaptively adjusting model parameters of the hybrid convolutional neural network in feature extraction to determine an optimal weight; inputting a to-be-classified hyperspectral image into the hybrid convolutional neural network under the optimal weight to perform feature extraction again so as to obtain a new feature map; and inputting the new feature map into a softmax-fc classifier to realize the classification of the hyperspectral image. According to the invention, high-efficiency and high-precision classification of the hyperspectral image can be realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a hyperspectral image classification method and system based on a hybrid convolutional neural network. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Hyperspectral imaging is a new remote sensing technology based on imaging spectroscopy. It can capture spectral information and surface information at the same time, realizing the true combination of surface image information and spectral information for the first time. Hyperspectral image classification has many uses. For example, in agricultural research, it can accurately monitor crop conditions, especially in crop condition assessment, natural disaster monitoring and agricultural management. In addition, the application of hyperspectral imaging technology in many important fields such as geological mineral resources, lake and ocean water quality monitoring, drought monitoring, national defense construction, surveying and mapping, and military is also expanding. With the rapid rise and popularization of computer Internet technology, services and needs based on hyperspectral image information are more widely used in life. It can not only provide individuals with information about the surrounding environment, but also help enterprises obtain updates on relevant information perception.

[0004] Recently, the integrated application model of "sky, ground and depth" has been widely used in various aspects such as agricultural production, natural resource management, urban planning and management, and land and resources survey. More advanced hyperspectral image applications are emerging rapidly and are changing people's lives to a great extent. For example, in the medical field, hyperspectral classification technology can be used for tissue composition analysis, tumor detection, etc., providing higher imaging contrast and resolution; used to detect water bodies, soil pollution, ecosystem changes, etc. Hyperspectral images can also help identify different pollutants and their distribution, and provide data support for environmental protection and governance. Hyperspectral images have played a huge role in these fields. Existing urban areas use hyperspectral images for land use classification and infrastructure monitoring, assist in urban planning and management decisions, and investigate mineral exploration and geology through hyperspectral images, helping to identify surface and underground mineral deposits and other geological features, etc., all of which rely on hyperspectral images.

[0005] However, the inventors have found that there are some technical problems in the existing methods for classifying hyperspectral images, such as:

[0006] (1) Among the existing methods, machine learning methods such as support vector machines, random forests, and multinomial logistic regression are commonly used to classify hyperspectral images. However, these methods mainly focus on the extraction of spectral features, thus ignoring the spatial correlation and spatial locality in hyperspectral images, which leads to low classification accuracy.

[0007] (2) For example, a hyperspectral image classification method disclosed in the patent with patent application number 202211300601.5 relies on a large amount of sample data when analyzing the image. Although the amount of data can be reduced to a certain extent through joint dimensionality reduction processing; however, the classification accuracy is still positively correlated with the amount of data after dimensionality reduction. Therefore, the amount of sample data is still relatively high, resulting in low computational efficiency during classification. Summary of the invention

[0008] In order to overcome the above-mentioned deficiencies of the prior art, the present invention provides a hyperspectral image classification method and system based on a hybrid convolutional neural network, which can achieve efficient and high-precision classification of hyperspectral images.

[0009] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0010] A first aspect of the present invention provides a hyperspectral image classification method based on a hybrid convolutional neural network.

[0011] The hyperspectral image classification method based on hybrid convolutional neural network includes:

[0012] Acquire a historical data set, and use a PCA method to perform band selection on the historical data set;

[0013] Divide the historical data set after band selection into patches;

[0014] Using the divided historical data set as a preliminary training sample; and determining a training sample from the preliminary training sample based on a ratio setting;

[0015] The hybrid convolutional neural network is trained using the training samples, wherein the hybrid convolutional neural network includes a context interaction network and a 3D-2D hybrid neural network; specifically, the context interaction network is used to perform shallow feature extraction and fusion on the images in the training samples to obtain a first feature map; the first feature map is input into the 3D-2D hybrid neural network to extract deep features to obtain a second feature map; and the model parameters of the hybrid convolutional neural network in the feature extraction process are adaptively adjusted to determine the optimal weights;

[0016] The hyperspectral image to be classified is input into the hybrid convolutional neural network under the optimal weight to re-extract features to obtain a new feature map; and the newly obtained feature map is input into the softmax-fc classifier to realize the classification of the hyperspectral image.

[0017] Furthermore, the PCA method is used to select bands for the historical data set, including:

[0018] The data in the historical data set are standardized and the covariance matrix of the standardized data is calculated; the principal component eigenvector is selected according to the eigenvalue size; based on the operation of the covariance matrix, the original data is projected to the principal component eigenvector.

[0019] Furthermore, the historical data set is divided into patches, including:

[0020] Import historical datasets and label sets; set the patch size; then extract patches around each pixel and create a format for keras processing; fill the surrounding pixels with data using ZeroPad; finally, store the divided datasets and label sets.

[0021] Furthermore, the first feature map output by the context interaction network is a feature matrix; the feature matrix is ​​used to input the first layer of the 3D-2D hybrid neural network.

[0022] Furthermore, the softmax-fc classifier connects each node to the output node and maps the high-dimensional features to specific classification results; then, the output of the classification results is converted into probabilities to obtain the probability distribution of each category.

[0023] Furthermore, the hyperspectral image classification method based on the hybrid convolutional neural network also includes:

[0024] The classification results of the softmax-fc classifier are evaluated using the confusion matrix, evaluation parameters, and the specific classification accuracy of each type of land object; if the classification result evaluation does not meet the standard, the model parameters of the hybrid convolutional neural network in the feature extraction process are readjusted.

[0025] Furthermore, after obtaining the classification results of the softmax-fc classifier, a classification result report is generated for the name of each type of land feature and the classification results are visualized; different training sample ratios are set multiple times, and for different training sample ratios, accuracy fluctuation graphs under different ratios are obtained.

[0026] A second aspect of the present invention provides a hyperspectral image classification system based on a hybrid convolutional neural network.

[0027] The hyperspectral image classification system based on hybrid convolutional neural network includes:

[0028] The band selection module is configured to: obtain a historical data set, and perform band selection on the historical data set using a PCA method;

[0029] The data set partitioning module is configured to: partition the historical data set after band selection into patches;

[0030] The training sample selection module is configured to: use the divided historical data set as a preliminary training sample; and determine a training sample from the preliminary training sample based on a ratio setting;

[0031] The model training module is configured to: train a hybrid convolutional neural network using the training samples, wherein the hybrid convolutional neural network includes a context interaction network and a 3D-2D hybrid neural network; specifically, extract and fuse shallow features of images in the training samples using the context interaction network to obtain a first feature map; input the first feature map into the 3D-2D hybrid neural network to extract deep features to obtain a second feature map; and adaptively adjust the model parameters of the hybrid convolutional neural network during the feature extraction process to determine the optimal weights;

[0032] The image classification module is configured as follows: inputting the hyperspectral image to be classified into the hybrid convolutional neural network under the optimal weight to re-extract features to obtain a new feature map; and inputting the newly obtained feature map into the softmax-fc classifier to realize the classification of the hyperspectral image.

[0033] The third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps in the method for hyperspectral image classification based on a hybrid convolutional neural network as described in the first aspect of the present invention.

[0034] The fourth aspect of the present invention provides an electronic device, comprising a memory, a processor, and a program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the method for hyperspectral image classification based on a hybrid convolutional neural network as described in the first aspect of the present invention are implemented.

[0035] One or more of the above technical solutions have the following beneficial effects:

[0036] (1) The present invention uses a hybrid convolutional neural network to analyze and process image features, that is, a context interaction network is used to extract and fuse shallow features of images in training samples, and then a feature map composed of shallow features is used as the input of a 3D-2D hybrid neural network, and a 3D-2D hybrid neural network is used to further extract deep features from the feature map. Therefore, the present invention uses two combined networks to progressively extract features from hyperspectral images, taking into account both spatial correlation and spatial locality in hyperspectral images, and has higher classification accuracy than the prior art.

[0037] (2) The present invention firstly retains the main information on the basis of reducing the dimension of the hyperspectral image data based on the principal component analysis (PCA), including the dimensional information processing of the data; at the same time, since the present application sets the ratio of training samples multiple times, the classification results of the Softmax-fc classifier can be evaluated multiple times, and even if the number of samples is very small, the optimal weight to achieve balanced classification accuracy can be found. Therefore, the present invention can use less sample data to ensure the classification accuracy of the hyperspectral image, thereby improving the computational efficiency during classification.

[0038] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0040] Figure 1 This is a flow chart of a hyperspectral image classification method based on a hybrid convolutional neural network in Example 1 of the present invention.

[0041] Figure 2 This is a flow chart of a method for evaluating the classification results of a softmax-fc classifier in Embodiment 1 of the present invention. DETAILED DESCRIPTION

[0042] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0043] It should be noted that the terms used herein are for describing specific embodiments only and are not intended to be limiting of exemplary embodiments according to the present invention.

[0044] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0045] The overall idea proposed by the present invention is: in order to achieve efficient and high-precision classification of remote sensing image data sets of different land objects, the present invention provides a hyperspectral image classification method based on a hybrid convolutional neural network, and realizes high-precision classification of land objects in different hyperspectral data sets under the condition of few samples by building a hybrid convolutional neural network model, that is: on the basis of making full use of hundreds of spectral band information of hyperspectral images, data processing is performed on the image data set; various land object information in different data sets are classified by algorithms, and the accuracy of each type of information and the accuracy evaluation index of the entire data set are obtained, and finally a visualization result diagram of the classification range of each land object is displayed.

[0046] Embodiment 1

[0047] This embodiment discloses a hyperspectral image classification method based on a hybrid convolutional neural network.

[0048] like Figure 1 As shown, the hyperspectral image classification method based on the hybrid convolutional neural network includes:

[0049] Step S1, obtaining a historical data set, and using the PCA method to perform band selection on the historical data set;

[0050] Step S2, dividing the historical data set after band selection into patches;

[0051] Step S3, using the divided historical data set as a preliminary training sample; and determining a training sample from the preliminary training sample based on a ratio setting;

[0052] Step S4, using the training samples to train a hybrid convolutional neural network, the hybrid convolutional neural network includes a context interaction network and a 3D-2D hybrid neural network; specifically, using the context interaction network to perform shallow feature extraction and fusion on the images in the training samples to obtain a first feature map; inputting the first feature map into the 3D-2D hybrid neural network to extract deep features to obtain a second feature map; adaptively adjusting the model parameters of the hybrid convolutional neural network in the feature extraction process to determine the optimal weights;

[0053] Step S5: input the hyperspectral image to be classified into the hybrid convolutional neural network under the optimal weight to re-extract features to obtain a new feature map; and input the newly obtained feature map into the softmax-fc classifier to achieve classification of the hyperspectral image.

[0054] Based on the above process, high-efficiency and high-precision classification of hyperspectral images can be achieved. In order to facilitate the understanding of the technical solution of the present invention, the above steps in the technical solution of the present invention are further explained and illustrated.

[0055] Step S1, obtaining a historical data set, and using the PCA method to perform band selection on the historical data set. Specifically, this can be achieved by the following steps:

[0056] Step S1-1, obtaining a historical data set.

[0057] The historical data set used in this embodiment is a public data set, and such public data sets can be directly downloaded and used on relevant websites. Specifically, the historical data set downloaded in this embodiment is Indian Pines, which includes the ground truth data set Indian Pines (actual size 6.0MB) and the label set Indian Pinesgroundtruth (actual size 1.1KB) used for classification. Since the main technical points of the present invention are not involved here, no further details will be given.

[0058] Step S1-2: Use principal component analysis (PCA) to select bands from the historical data set.

[0059] The data in the historical data set are standardized and the covariance matrix of the standardized data is calculated; the principal component eigenvector is selected according to the eigenvalue size; based on the operation of the covariance matrix, the original data is projected to the principal component eigenvector.

[0060] Step S1-2-1, standardize the data in the historical data set to ensure that each feature has the same scale; calculate the covariance matrix of the processed data to better evaluate the correlation between the features.

[0061] First, the data in the historical data set is standardized so that the mean of each feature is 0 and the variance is 1. Specifically, the formula used for standardization is as follows:

[0062]

[0063] Among them, z represents the historical data set after standardization, x represents the original observation value of a pixel in a certain band (feature), μ represents the mean, and σ represents the standard deviation.

[0064] Subsequently, the covariance matrix is ​​calculated for the standardized historical data set. The covariance matrix can reflect the correlation between features. Specifically, the covariance matrix is ​​expressed as follows:

[0065]

[0066] Where C represents the covariance matrix, n represents the number of samples (each pixel can be regarded as a sample); X represents the standardized data matrix, X T is the transposed matrix of the data matrix X.

[0067] Step S1-2-2: According to the size of the eigenvalues ​​of the covariance matrix, select the eigenvectors corresponding to the first k eigenvalues ​​as the principal components.

[0068] Step S1-2-3: Project the original data onto the selected principal components through matrix calculation to achieve dimensionality reduction of the data set.

[0069] First, perform matrix calculation; specifically, the matrix calculation is implemented by the following formula, namely:

[0070] Y = XW;

[0071] Among them, Y represents the data after dimensionality reduction, and W represents the matrix composed of the selected eigenvectors.

[0072] Subsequently, the original data is projected onto the selected principal components to achieve dimensionality reduction of the data set.

[0073] Step S2: Divide the historical data set after band selection into patches.

[0074] Import historical datasets and label sets; set the patch size; then extract patches around each pixel and create a format for keras processing; fill the surrounding pixels with data using ZeroPad; finally, store the divided datasets and label sets.

[0075] Step S2-1: Build a suitable pytorch algorithm environment according to the actual usage environment, and import the applied hyperspectral dataset (historical dataset) and label set through the loadmat function.

[0076] Step S2-2: In order to better process the data information, set the appropriate patch=w size, extract the patch around each pixel, and create a format for keras processing, that is: take the width of (w-1) / 2 for processing around the pixel centered at (i, j); fill the data for the surrounding pixels through ZeroPad.

[0077] Due to the large amount of hyperspectral data, the regional division in step S2 can process and analyze the data more effectively, and train and test on a smaller data set, making the algorithm calculation more efficient. After dividing the data set into small blocks, local information can be extracted to better capture details; and these extracted local features help improve the accuracy of classification, speed up model training time, and reduce memory usage. Therefore, step S2 can further enhance the feature learning intensity, model generalization ability, and model robustness.

[0078] Step S2-3: Store the divided data sets and label sets.

[0079] It can be understood that the reason why the training set and the test set are stored separately according to the ratio in this embodiment is to facilitate the training and testing of the subsequent model.

[0080] Step S3: using the divided historical data set as a preliminary training sample; and determining a training sample from the preliminary training sample based on a ratio setting.

[0081] The preparatory training samples are divided into training samples and test samples, and the ratio of the training samples (training set) and the test samples (test set) in the preparatory training samples is set based on the ratio parameter setting formula test_ratio=p. For example, when the test parameter p=0.9 is set, it means that 99% of the data set in the entire preparatory training samples are training samples, and the other 1% of the data set in the entire preparatory training samples are test samples. It can be understood that the ratio relationship between the training samples and the test samples can be adjusted according to actual needs, and this embodiment does not impose specific restrictions on this.

[0082] Step S4, using the training samples to train the hybrid convolutional neural network; wherein the hybrid convolutional neural network includes a context interaction network and a 3D-2D hybrid neural network; specifically, using the context interaction network to perform shallow feature extraction and fusion on the images in the training samples to obtain a first feature map; inputting the first feature map into the 3D-2D hybrid neural network to extract deep features to obtain a second feature map; adaptively adjusting the model parameters of the hybrid convolutional neural network in the feature extraction process to determine the optimal weights.

[0083] Step S4-1, set a 3D convolution layer of (7,3,3), that is, the length, width, and height of the convolution kernel of the convolution layer are 7, 3, and 3 respectively; set a suitable channel and perform regularization on it; at the same time, use Relu as the activation function;

[0084] Step S4-2: Build a contextual interaction network through two layers of (1,1,1) 3D convolutional layers (l and l+1).

[0085] Furthermore, there are two 3D convolution layers. In this embodiment, the two 3D convolution layers are named as layer l and layer l+1 respectively. There is a parameter in the convolution layer, which is the convolution kernel size. The 3D convolution layer has three parameters. The convolution kernel sizes of the two layers are set to (1, 1, 1). The two layers are the context interaction network. The lth layer is used to calculate the weighted sum when sliding at each position of the input.

[0086] The data processing flow of the context interaction network is as follows: the input features pass through the lth layer and the l+1th layer, and the input data of the lth layer and the output data of the l+1th layer are added pixel by pixel. The output features obtained in step S4-1 are input into the context interaction network to obtain the output feature F1, and F1 is input into the context interaction network again to obtain the output feature F2, thereby realizing multiple feature information fusion and extraction.

[0087] The output features obtained in step S4-1 and the output features obtained in the lth layer are added pixel by pixel to fuse the features, and the fused data features are input into the 'l+1' layer. The features are fused again in the 'l+1' layer, that is, the output features obtained in the 'l' layer and the output features in the 'l+1' layer are added pixel by pixel to fuse the features again.

[0088] Step S4-3: Use a 3D-2D hybrid neural network to further extract deep features.

[0089] A hybrid convolution module is constructed by setting a (5,1,1) 3D convolution layer and a (3,3) 2D convolution layer. That is, a 3D convolution layer with a convolution kernel length, width, and height of 5, 3, and a 2D convolution layer with a convolution kernel length and width of 3, 3 are combined to obtain a 3D-2D hybrid neural network.

[0090] The feature matrix (first feature map) F2 output by the context interaction network is input into the 3D convolution layer of the 3D-2D hybrid neural network to obtain the feature matrix F3. Set appropriate input and output feature channels; input F3 into the 2D convolution layer of the hybrid neural network to obtain F4, so as to further extract deep features from the spatial features of the fused and superimposed hyperspectral images; perform regularization on the extracted features; and use Relu as the activation function. Finally, expand the obtained two-dimensional feature matrix to obtain a one-dimensional matrix.

[0091] Furthermore, the two-dimensional features are expanded through the Flatten function to obtain a one-dimensional feature sequence. For example, a 3*3 two-dimensional feature matrix is ​​constructed:

[0092]

[0093] Convert the above matrix to one dimension, i.e. [1,2,3,4,5,6,7,8,9].

[0094] Step S4-4: adaptively adjust the model parameters of the hybrid convolutional neural network during the feature extraction process to determine the optimal weights.

[0095] Put the hybrid convolutional neural network on the GPU and set the appropriate learning rate, loss function, optimization algorithm and other hyperparameters. By setting the epoch, the model is iterated multiple times on the training sample; in each iteration, forward propagation, loss function calculation, back propagation and weight update operations are performed. The curve of the training accuracy and loss of the hybrid convolutional neural network model changing with the number of iterations is obtained, and the number of iterations of the hybrid convolutional neural network model is further adjusted to ensure the optimization of the model weights.

[0096] Based on step S4, this embodiment can automatically learn features by utilizing the classification method of a hybrid convolutional neural network, greatly reducing the work of manual labeling; it can be applied to multiple different hyperspectral data sets under a small number of sample conditions, and has better application universality than the existing technology.

[0097] Step S5: input the hyperspectral image to be classified into the hybrid convolutional neural network under the optimal weight to re-extract features to obtain a new feature map; and input the newly obtained feature map into the softmax-fc classifier to achieve classification of the hyperspectral image.

[0098] In the specific implementation process, the softmax-fc classifier connects each node to the output node and maps the high-dimensional features to the specific classification results; then, the output of the classification results is converted into probabilities to obtain the probability distribution of each category, thereby realizing the classification of hyperspectral images.

[0099] Furthermore, in order to further improve the accuracy of the hybrid convolutional neural network in classifying hyperspectral images, the present invention also provides a method for evaluating the classification results of the hybrid convolutional neural network. Through this method, positive feedback can be given on whether the classification accuracy of the hybrid convolutional neural network reaches the ideal state; when it does not reach the ideal state, the model parameters are readjusted to enhance the classification performance of the convolutional neural network. Therefore, even when the number of samples is extremely small, the optimal weight for achieving balanced classification accuracy can be found; therefore, the present invention can use less sample data to ensure the classification accuracy of hyperspectral images, thereby improving the computational efficiency during classification.

[0100] like Figure 2 As shown in the figure, the evaluation of the classification results of the hybrid convolutional neural network can be achieved through the following process:

[0101] The classification results of the softmax-fc classifier are evaluated using indicators such as confusion matrix, evaluation parameters (OA, AA, Kappa) and the specific classification accuracy of each type of land feature; if the classification result does not meet the evaluation standard, the model parameters of the hybrid convolutional neural network during the feature extraction process or the classification process are readjusted.

[0102] (1) Calculate the confusion matrix, evaluation parameters (OA, AA, Kappa) and specific classification accuracy of each type of feature for the obtained results; generate a specific classification result report corresponding to the name of each type of feature. At the same time, classify the classification results of the classifier pixel by pixel, extract the classification results, and predict the pixel labels pixel by pixel to realize the visualization of the classification results; finally, generate a map and analyze its application value.

[0103] Furthermore, the confusion matrix is ​​mainly composed of four values: TP (True Positive), FP (False Positive), FalsePositive, and FN (False Negative). These values ​​are the results calculated and saved when the model was last tested. Specifically, TP means that the actual value is the same as the predicted value, FP means that the actual value is negative, but the model predicts it to be positive, TN means that the actual value and the predicted value have the same meaning, and FN means that the actual value is positive, but the model predicts it to be negative.

[0104] Furthermore, the result report is generated through the code into a txt report file, including the three parameters of OA, AA, and Kappa and the evaluation parameters of each category.

[0105] Furthermore, the classification result map is also generated by predicting the classification pixel by pixel; wherein the color represented by each classification can be determined based on the label map.

[0106] (2) The robustness of the algorithm is ensured by setting different classification samples to compare the fluctuation range of the evaluation indicators.

[0107] (3) Set different training sample ratios multiple times; repeat process (1) for different training sample ratios to obtain accuracy fluctuation graphs under different ratios, and ensure the robustness of the model through comparison. It can be understood that when the fluctuations shown in the accuracy fluctuation graphs under different ratios are smaller, the more robust the model is.

[0108] The entire classification method and the entire evaluation process of this embodiment meet the characteristics of high efficiency, high precision, less interference, rich information, strong applicability, etc., and can provide users with more accurate and reliable classification maps or reports. The entire algorithm of this embodiment is built on a computer with a GPU graphics processing unit. It is more efficient when processing large-scale hyperspectral image data, and can use hardware acceleration such as GPU for training and reasoning, enhancing the ability to process large-scale data; at the same time, an adaptive moment estimation (Adam) optimizer is used for updating, and the loss function in the training process is calculated by the cross entropy loss function (CrossEntropy Loss). The data processing process is carried out on Python.

[0109] Embodiment 2

[0110] This embodiment discloses a hyperspectral image classification system based on a hybrid convolutional neural network.

[0111] The hyperspectral image classification system based on hybrid convolutional neural network includes:

[0112] The band selection module is configured to: obtain a historical data set, and perform band selection on the historical data set using a PCA method;

[0113] The data set partitioning module is configured to: partition the historical data set after band selection into patches;

[0114] The training sample selection module is configured to: use the divided historical data set as a preliminary training sample; and determine a training sample from the preliminary training sample based on a ratio setting;

[0115] The model training module is configured to: train a hybrid convolutional neural network using the training samples, wherein the hybrid convolutional neural network includes a context interaction network and a 3D-2D hybrid neural network; specifically, extract and fuse shallow features of images in the training samples using the context interaction network to obtain a first feature map; input the first feature map into the 3D-2D hybrid neural network to extract deep features to obtain a second feature map; and adaptively adjust the model parameters of the hybrid convolutional neural network during the feature extraction process to determine the optimal weights;

[0116] The image classification module is configured as follows: inputting the hyperspectral image to be classified into the hybrid convolutional neural network under the optimal weight to re-extract features to obtain a new feature map; and inputting the newly obtained feature map into the softmax-fc classifier to realize the classification of the hyperspectral image.

[0117] Embodiment 3

[0118] The purpose of this embodiment is to provide a computer-readable storage medium.

[0119] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps in the method for hyperspectral image classification based on a hybrid convolutional neural network as described in the first embodiment of the present disclosure.

[0120] Embodiment 4

[0121] The purpose of this embodiment is to provide an electronic device.

[0122] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, the steps in the method for hyperspectral image classification based on a hybrid convolutional neural network as described in the first embodiment of the present disclosure are implemented.

[0123] The steps involved in the apparatuses of the above embodiments 2, 3 and 4 correspond to the method embodiment 1, and the specific implementation methods can refer to the relevant description part of embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.

[0124] Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0125] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A hyperspectral image classification method based on a hybrid convolutional neural network, characterized in that: include: Acquire a historical data set, and use a PCA method to perform band selection on the historical data set; Divide the historical data set after band selection into patches; Use the divided historical data set as preliminary training samples; Based on the ratio setting, determining a training sample from the prepared training samples; The hybrid convolutional neural network is trained using the training samples, wherein the hybrid convolutional neural network includes a context interaction network and a 3D-2D hybrid neural network; specifically, the context interaction network is used to perform shallow feature extraction and fusion on the images in the training samples to obtain a first feature map; the first feature map is input into the 3D-2D hybrid neural network to extract deep features to obtain a second feature map; and the model parameters of the hybrid convolutional neural network in the feature extraction process are adaptively adjusted to determine the optimal weights; The hyperspectral image to be classified is input into the hybrid convolutional neural network under the optimal weight to re-extract features to obtain a new feature map; The newly obtained feature map is then input into the softmax-fc classifier to achieve the classification of the hyperspectral image.

2. The method for hyperspectral image classification based on a hybrid convolutional neural network as claimed in claim 1, characterized in that: The PCA method is used to select bands for historical data sets, including: The data in the historical data set are standardized and the covariance matrix of the standardized data is calculated; the principal component eigenvector is selected according to the size of the eigenvalue; based on the operation of the covariance matrix, the original data is projected to the principal component eigenvector.

3. The method for hyperspectral image classification based on hybrid convolutional neural network as claimed in claim 1, characterized in that: Divide historical data sets into patches, including: Import historical datasets and label sets; set the patch size; then extract patches around each pixel and create a format for keras processing; fill the surrounding pixels with data using ZeroPad; finally, store the divided datasets and label sets.

4. The method for hyperspectral image classification based on hybrid convolutional neural network as claimed in claim 1, characterized in that: The first feature map output by the context interaction network is a feature matrix; the feature matrix is ​​used to input the first layer of the 3D-2D hybrid neural network.

5. The method for hyperspectral image classification based on hybrid convolutional neural network as claimed in claim 1, characterized in that: The softmax-fc classifier connects each node to the output node and maps the high-dimensional features to specific classification results; then, the output of the classification results is converted into probabilities to obtain the probability distribution of each category.

6. The method for hyperspectral image classification based on hybrid convolutional neural network as claimed in claim 1, characterized in that: Also includes: The classification results of the softmax-fc classifier are evaluated using the confusion matrix, evaluation parameters, and the specific classification accuracy of each type of land object; if the classification result does not meet the evaluation standards, the model parameters of the hybrid convolutional neural network in the feature extraction process are readjusted.

7. The method for hyperspectral image classification based on hybrid convolutional neural network as claimed in claim 6, characterized in that: After obtaining the classification results of the softmax-fc classifier, a classification result report is generated for each type of feature and the classification results are visualized; different training sample ratios are set multiple times, and accuracy fluctuation graphs under different training sample ratios are obtained.

8. A hyperspectral image classification system based on a hybrid convolutional neural network, characterized in that: include: The band selection module is configured to: obtain a historical data set, and perform band selection on the historical data set using a PCA method; The data set partitioning module is configured to: partition the historical data set after band selection into patches; The training sample selection module is configured to: use the divided historical data set as a preliminary training sample; Based on the ratio setting, determining a training sample from the prepared training samples; The model training module is configured to: train a hybrid convolutional neural network using the training samples, wherein the hybrid convolutional neural network includes a context interaction network and a 3D-2D hybrid neural network; specifically, extract and fuse shallow features of images in the training samples using the context interaction network to obtain a first feature map; input the first feature map into the 3D-2D hybrid neural network to extract deep features to obtain a second feature map; and adaptively adjust the model parameters of the hybrid convolutional neural network during the feature extraction process to determine the optimal weights; The image classification module is configured to: input the hyperspectral image to be classified into the hybrid convolutional neural network under the optimal weight to re-extract features to obtain a new feature map; The newly obtained feature map is then input into the softmax-fc classifier to achieve the classification of the hyperspectral image.

9. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, the steps in the hyperspectral image classification method based on a hybrid convolutional neural network as described in any one of claims 1 to 7 are implemented.

10. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the hyperspectral image classification method based on a hybrid convolutional neural network as described in any one of claims 1 to 7 are implemented.

Images