Hyperspectral classification method based on neural network architecture search

Abstract

The invention discloses a hyperspectral image classification method based on neural architecture search, which belongs to the field of image classification technology. The method sequentially performs the following steps: inputting a hyperspectral image (HSI), cutting it into samples of 32×32 pixels using a sliding window and dividing it into training samples, verification samples and test samples; constructing a neural architecture search (NAS) space using 6 preset operations, and inputting the training samples into the space; searching for candidate network structures using a gradient algorithm search strategy, and regularizing the architecture parameters using β-decay during the search process; updating the structural gradient weight using a confidence learning rate; forming basic units with the searched preset operations, and connecting them to form an optimal neural network architecture; classifying HSI using the optimal neural network architecture to obtain classification results. The invention is a hyperspectral image classification method based on neural network search, which searches for the optimal architecture in a search space constructed by 6 preset operations, and improves the generalization ability of NAS using β-decay regularization, improves the NAS operation speed using a confidence learning rate, and reduces the overfitting phenomenon, effectively improving the classification accuracy of hyperspectral images, and providing a good idea for subsequent neural architecture search algorithms in hyperspectral image classification research and related applications.

Description

Technical Field

[0001] The invention discloses a hyperspectral classification method based on a neural network architecture search, and belongs to the technical field of remote sensing image classification and intelligent interpretation. Background Art

[0002] Hyperspectral images are images acquired by hyperspectral imagers. They are rich in spectral information and spatial dimension information, so they are widely used in precision agriculture, marine hydrological monitoring, atmospheric environment monitoring and other fields. Hyperspectral image classification refers to the task of assigning a category to each pixel in the scene. Since hyperspectral images involve a wide range of fields and are far-reaching, hyperspectral image classification is a key research direction in the field of remote sensing.

[0003] In recent years, the rise of deep learning has promoted the further development of hyperspectral image classification. Traditional CNN architectures need to rely on manual design and parameter adjustment, which consumes a lot of time and effort. In this context, the method of automatically generating neural network architectures, namely Neural Architecture Search (NAS), has attracted a lot of attention. NAS relaxes the search space to make it continuously differentiable, and then learns the weight of each operation through gradient descent to optimize the performance of the structure and make the structure search more efficient. Summary of the invention

[0004] In view of the above problems, the present invention provides a hyperspectral image classification method based on neural architecture search. After the hyperspectral image to be classified is input, the HSI image is cropped and divided into training samples, verification samples and test samples using a sliding window strategy. The NAS space is constructed using 6 preset operations, and the training samples are input into the space. The candidate network structure is searched out using the search strategy of the gradient algorithm. During the search process, the architecture parameters are regularized using β-decay. The structural gradient weight is updated using CLR, and the preset operations searched are combined into basic units, and connected to form the optimal neural network architecture; finally, the HSI is classified using the optimal neural network architecture to obtain the classification result.

[0005] The object of the present invention is achieved in that:

[0006] Step a, inputting a hyperspectral image to be classified;

[0007] Step b, using a sliding window strategy to crop the HSI image and divide it into training samples, verification samples and test samples;

[0008] Step c: Use 6 preset operations to build a NAS space and input the training samples into the space;

[0009] Step d, using the search strategy of the gradient algorithm to search for candidate network structures, and using β-decay to regularize the architecture parameters during the search process;

[0010] Step e: Use CLR to update the structure gradient weight;

[0011] Step f, grouping the searched preset operations into basic units and connecting them to form an optimal neural network architecture;

[0012] Step g, using the optimal neural network architecture to classify the HSI to obtain a classification result;

[0013] The hyperspectral image classification method based on neural architecture search includes the following steps:

[0014] The hyperspectral image classification method based on neural architecture search according to claim 1 is characterized in that step b specifically includes cropping the samples by adopting a sliding window strategy, with a window size of 32×32 and an overlap rate of 50%. 300 samples are randomly selected from the samples as a training data set, 200 samples are used as a validation data set, and the remaining samples are used as a test set;

[0015] The hyperspectral image classification method based on neural architecture search according to claim 1 is characterized in that step c specifically comprises:

[0016] Step c1: Set is a set of candidate operations on directed edges, denoted by the architecture parameter α (m,n) The preset operations are obtained by parameterization. There are 6 preset operations on each edge, including None, Skip-Connect, Max pooling, SeparateConv, Dilated-Conv, and Avg pooling. The convolution kernels of each operation are 3×3, 5×5, and 7×7. The above operations are defined as the candidate operation set in the search space.

[0017] Step c2, input the training samples into the space;

[0018] The hyperspectral image classification method based on neural architecture search according to claim 1 is characterized in that step d specifically comprises:

[0019] Step d1, using the Softmax function, the discrete search space is converted into a continuous space;

[0020] In order to make the search space continuous, α o (m,n) The candidate operation set The discretized operation in is transformed into a set of Softmax algorithms of all possible operations;

[0021]

[0022] in: It represents the set of all possible operations, and the sum of all coefficients in the operation set is 1. o (m,n) It represents the architecture parameters between nodes m and n (m=0,1,…,N-1; n=0,1,…,N-1), and at most N nodes are set in each unit.

[0023] Step d2: After the Softmax operation, the set of architecture parameters β is obtained k (m,n) , for the parameter β k (m,n) Perform attenuation regularization;

[0024]

[0025] Where: β k (m,n) is the combination of the architecture parameters between node m and node n, and k is the number represented by the optional operations.

[0026] Step d3, NAS uses the Softmax-activated architecture parameter set β to represent the possibility of each operator. Attenuation regularization of β can limit its value from being too large, which can improve the robustness and architecture generalization in the architecture search process.

[0027] The hyperspectral image classification method based on neural architecture search according to claim 1 is characterized in that step e specifically comprises:

[0028] Step e1: After the Softmax operation, the architecture parameter α in all mixed operations is α = {α (m,n)} network weights w are jointly learned, and the loss value in the training process is calculated using the cross entropy formula according to the node calculation, and the loss value in the training process is obtained respectively. and

[0029] Step e2: According to and The network parameters and structural parameters are optimized bidirectionally to obtain the network weight parameter w. W is trained in the training data set to obtain the optimal value of the network parameter. After one round of training, the structural weights between each node are evenly distributed.

[0030] Step e3: perform double-layer optimization on the network parameters and structural parameters according to the loss value, use CLR to update the initial learning rate of the original architecture weight, establish a confidence gradient, and implement cyclic weight update according to this step;

[0031]

[0032] Where: t represents the current epoch number of the current training, δ represents the confidence factor, and the present invention takes δ=2,

[0033] 1. α is the learning rate of the NAS architecture, which is set to lr by default α =0.0003.

[0034] The hyperspectral image classification method based on neural architecture search according to claim 1 is characterized in that step f specifically comprises:

[0035] Step f1: After the weight is updated, the two preset operations with the largest weights before the node of each unit are taken, and the operations and nodes are connected to construct a reduction unit for downsampling and a normal unit for normal sampling;

[0036] In step f2, the step size of all preset operations in the normal unit is 1, so the size of the feature map output by it remains unchanged. The step size of all preset operations in the reduction unit is 2, and it is inserted at 1 / 3 and 2 / 3 of the network depth. The reduction unit and the normal unit are connected to obtain the optimal neural network architecture;

[0037] The hyperspectral image classification method based on neural architecture search according to claim 1 is characterized in that step g specifically comprises classifying the HSI using the optimal neural network architecture to obtain a classification result.

[0038] Beneficial effects:

[0039] Traditional CNN architectures require manual design and parameter adjustment, which consumes a lot of time and effort. The present invention is a hyperspectral image classification method based on neural network search. It searches for the optimal architecture in a search space constructed by 6 preset operations, uses β-decay regularization to improve the generalization ability of the search structure, and uses the confidence learning rate to improve the model's computing speed. At the same time, it reduces the overfitting phenomenon, effectively improves the classification accuracy of hyperspectral images, and provides a good idea for subsequent neural architecture search algorithms in hyperspectral image classification research and related applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a flow chart of the hyperspectral image classification method based on neural architecture search of the present invention.

[0041] Figure 2 It is a search process diagram of the hyperspectral image classification method based on neural architecture search of the present invention.

[0042] Figure 3 It is a schematic diagram of the β-decay regularization principle in the method of the present invention.

[0043] Figure 4 It is a schematic diagram of the CLR principle in the method of the present invention.

[0044] Figure 5 It is a schematic diagram of the structure of the normal unit searched for the Indian Pines data set in the method of the present invention.

[0045] Figure 6 It is a schematic diagram of the structure of the reduction unit searched for the Indian Pines data set in the method of the present invention.

[0046] Figure 7 It is a schematic diagram of the structure of a normal unit searched from the Pavia University data set in the method of the present invention.

[0047] Figure 8 It is a schematic diagram of the structure of the reduction unit searched for the Pavia University data set in the method of the present invention.

[0048] Fig. 9 This is a diagram of the classification results in the Indian Pines dataset in the method of the present invention.

[0049] Fig.10 It is a classification result diagram of the Pavia University dataset in the method of the present invention. DETAILED DESCRIPTION

[0050] The specific implementation modes of the present invention are further described in detail below with reference to the accompanying drawings.

[0051] The hyperspectral image classification method based on neural architecture search in this specific implementation manner is as follows: Figure 1 As shown, the network principle diagram is as follows Figure 2 As shown, the following steps are included:

[0052] Step a, inputting a hyperspectral image to be classified;

[0053] In this specific implementation, two public datasets, Indian Pines and Pavia University, are used.

[0054] (1) Indian Pines dataset: acquired by the AVIRIS sensor over the Indian Pines Agricultural Experiment Station in northwestern Indiana, USA. The original data consists of 224 bands, 4 zero bands and 20 water absorption bands (104-108, 150-163, 220) are removed, and the remaining 200 bands are used for experimental research, ranging from 0.4 to 2.5 μm, with a spatial size of 145×145 pixels.

[0055] (2) Pavia University dataset: obtained by the ROSIS-03 sensor over the University of Pavia in Italy. The dataset size is 610×340 pixels, the spatial resolution is 1.3m, the original data has 115 bands, and the spectral coverage range is 0.43 to 0.86μm. 12 noise bands are removed, and 103 bands are left for experimental research. There are 9 city categories.

[0056] Step b, using a sliding window strategy to crop the HSI image and divide it into training samples, verification samples and test samples;

[0057] The samples are cropped by using a sliding window strategy, with a window size of 32×32 and an overlap rate of 50%. 300 samples are randomly selected from the samples as the training data set and 200 samples are used as the validation data set. The training data set is used to train the weights and biases of each neuron in the model, while the architecture variables are actually optimized based on the validation data set. After obtaining the optimal architecture, all remaining labeled samples are used as test data sets to evaluate the performance of the optimal network architecture and obtain the final classification results.

[0058] Step c: Use 6 preset operations to build the NAS space and input the training samples into the space

[0059] Step c1: Set is a set of candidate operations on directed edges, denoted by the architecture parameter α (m,n) The preset operations are obtained by parameterization. There are 6 preset operations on each edge, including None, Skip-Connect, Max pooling, SeparateConv, Dilated-Conv, and Avg pooling. The convolution kernels of each operation are 3×3, 5×5, and 7×7. The above operations are defined as the candidate operation set in the search space.

[0060] Step c2, input the training samples into the search space;

[0061] Step d, using the search strategy of the gradient algorithm to search for candidate network structures, and using β-decay to regularize the architecture parameters during the search process;

[0062] Step d1, using the Softmax function, the discrete search space is converted into a continuous space;

[0063] In order to make the search space continuous, α o (m,n) The candidate operation set The discretized operation in is transformed into a set of Softmax algorithms of all possible operations;

[0064]

[0065] in: It represents the set of all possible operations, and the sum of all coefficients in the operation set is 1. o (m,n) It represents the architecture parameters between nodes m and n (m=0,1,…,N-1, n=0,1,…,N-1), and each unit can be set up to N

[0066] nodes.

[0067] Step d2: After the Softmax operation, the set of architecture parameters β is obtained k (m,n) , for the parameter β k (m,n) Perform attenuation regularization;

[0068]

[0069] Where: β k (m,n) is the combination of the architecture parameters between node m and node n, and k is the number represented by the optional operations.

[0070] When node x (m) With x (n) The final operation between Therefore, an operation coefficient is defined for each candidate operation. This coefficient is determined by the architecture parameters Parameterized as A mixture of all possible Softmax operations within . So from node x (m) to x (n) The information flow of is the weighted sum of all candidate operations.

[0071]

[0072]

[0073] Among them: the sum of all coefficients in the operation set is 1. In addition, the weights of all mixed operations between two nodes are composed of dimensions The vector α o (m,n) Parameterized.

[0074] The entire network is made up of repeated units stacked together. Each unit is a directed acyclic graph (DAG) consisting of several directed edges and N nodes, including two input nodes, one output node, and the remaining N-3 are intermediate nodes. Define each node as x (m) (m=0,1,…,N-1), which represents a feature graph. The information of the input node of the current unit is obtained from the output nodes of the two previous units, and the output node is the deep cascade of all the intermediate nodes in the unit. The directed edge in the unit represents the information from the intermediate node x (m) to x (n) The information flow consists of candidate operations. There is a preset operation o on each directed edge. (m,n) Associated with it, this candidate operation transfers information from node x (m) Transfer to node x (n) , calculate the node x based on all the nodes before it in the same unit (n) Output.

[0075]

[0076] Among them: is a set of candidate operations on a directed edge. In NAS, the default operation for each directed edge is determined by the architecture parameter α (m,n) Parameterized.

[0077] After the Softmax operation, β-decay regularization is performed. For the convenience of analysis, we start from the default setting of regularization and consider the single-step update of the structural parameter α, as shown in the following formula.

[0078]

[0079] Where: η α and are the learning rate and corresponding loss of the structure parameters, respectively.

[0080] In step d3, NAS search uses the Softmax-activated architecture parameter set β to represent the possibility of each operator. The explicit regularization of β can limit its value from being too large, which can improve the robustness and generalization of the architecture during the architecture search process.

[0081] For simplicity, the θ function (α is the independent variable) is used to represent the overall impact of β-decay regularization. In order to achieve the above β regularization similar to weight decay through α, the following unified formula can be obtained:

[0082]

[0083]

[0084] Where: F is the mapping function, which determines the influence of α on β. Therefore, a more suitable mapping function F is needed. Substituting the above formula into the single-step update formula of the structural parameter α, we can get as well as Then divide the former by the latter.

[0085]

[0086] As shown above, the mapping function determines the influence of α on β. In order to avoid invalid regularization and avoid large changes, the gradient of α is equal to the value of the F mapping function.

[0087]

[0088] After the gradient value is obtained, the β regularization loss value is as follows:

[0089]

[0090] Substitute the gradient formula of α into and The division formula captures the effect of β-decay regularization:

[0091]

[0092] Among them, it can be seen that θ decreases with the increase of α. The larger α is, the smaller θ is, and the smaller α is, the larger θ is. The value of β is constrained to be closer to its average value, thereby achieving an effect similar to weight attenuation, so it is called β-attenuation regularization.

[0093] Step e: Use CLR to update the structure gradient weight;

[0094] Step e1: After the Softmax operation, the structure is relaxed, and the neural architecture search is performed on the architecture parameters α={α (m,n)} network weights w for joint learning, and use the cross entropy formula to calculate the loss value during training, respectively and

[0095] The purpose of the search space defined above is to optimize α = {α (m,n)} matrix, and the original weight of the neural network is called w. The cross entropy formula is used to calculate the loss value during the training process, and the loss value is obtained respectively. and In order to achieve end-to-end optimization, the architecture search process hopes that the two matrices can make the results better, so a two-layer optimization is carried out.

[0096] Step e2: According to and The network parameters and structural parameters are optimized bidirectionally to obtain the network weight parameter w. w is trained in the training data set to obtain the optimal value of the network parameter. After one round of training, the structural weights between each node are evenly distributed.

[0097] In the search process, the two-layer optimization has a strict hierarchy. In order to achieve the optimization strategy of both at the same time, the process is to fix α = {α (m,n)} matrix, and then use the gradient descent w value, fix the value of w on the validation set, and then use the gradient descent α = {α (m,n)}, and repeat the process to obtain a more ideal value. The purpose of the search is to find the best architecture α * , to minimize the validation loss The specific implementation formula is as follows:

[0098]

[0099]

[0100] Where: The network weight w is obtained by minimizing the training loss NAS solves this problem using a two-level optimization, where α is a high-level variable and w is a low-level variable.

[0101] Step e3: Perform double-layer optimization on network parameters and structural parameters according to the loss value, use CLR to update structural weights, use CLR to update the initial learning rate of the original architecture weights, establish confidence gradients, calculate gradients related to network weights, and finally implement weight updates.

[0102] In order to alleviate the problem of gradient update speed being easily affected by the continuous relaxation strategy and prevent the gradient explosion, CLR is introduced. The confidence learning rate in the architecture search process should increase with the update training time of the architecture weights to alleviate the overfitting phenomenon of the model. The specific definition is as follows:

[0103]

[0104] Where: t represents the current epoch number of the current training, δ represents the confidence factor, and the present invention takes δ=2,

[0105] 1. α is the learning rate of the NAS architecture, which is set to lr by default α =0.0003.

[0106] After the validation sample is input into the model, we get use Compute the gradients with respect to the architecture weights as follows:

[0107]

[0108] The network weights w are obtained by minimizing the training loss α is the operation weight, and the original weight of the neural network is called w.

[0109] Construct confidence gradients:

[0110]

[0111] In obtaining the confidence gradient grad α After the weight α is manipulated, the search network becomes a two-level optimization problem. The overall training process of the search phase is as follows:

[0112]

[0113]

[0114] is the cross entropy loss function in the training phase, α * (w) is the fixed architecture weight α when the network weight is w.

[0115] The verification steps in the search phase are:

[0116]

[0117]

[0118] in: To verify the cross entropy loss function, w * (α) is the fixed network weight w when the architecture weight is α.

[0119] After two-layer optimization, the training phase and the verification phase are alternately used to update the weights.

[0120] Step f, grouping the searched preset operations into basic units and connecting them to form an optimal neural network architecture;

[0121] Step f1: After weight update, α = {α (m,n)} matrix, so that the edge with the largest weight is preserved, so the final unit needs to be generated after this structure converges. For each node, retain at most two edges with the largest weight. For the edge between two nodes, only retain the edge with the largest weight. Calculate and take the two largest weight operations before the node of each unit, connect the operations and nodes, and construct the reduced unit for downsampling and the normal unit for normal sampling;

[0122] In step f2, the step size of all preset operations in the normal unit is 1, so the size of the feature map output by it remains unchanged. The step size of all preset operations in the reduction unit is 2, and it is inserted at 1 / 3 and 2 / 3 of the network depth. Connecting the reduction unit and the normal unit to obtain the optimal neural network architecture

[0123] Step g: classify the HSI using the optimal neural network architecture to obtain the classification result.

[0124] In order to subjectively evaluate the classification effect, Figure 5 and Figure 6 The true value images of the Indian Pines dataset and the Pavia University dataset and the pseudo-color images of the classification results of each method are shown respectively. The order of the classification methods is the same as that of the classification methods in Table 1.

[0125] Table 1 and Table 2 show the detailed classification results of the proposed method and other comparative methods on the two datasets. It is obvious that the proposed method has better classification performance than all comparative methods, and it has the highest classification indexes on the two datasets, with OA reaching 96.60% and 97.61% respectively.

[0126] It can be seen that compared with the classical method support vector machine (EMP-SVM) and several advanced methods, including spectral-spatial residual network (SSRN), residual neural network (ResNet) and Cutout improved NAS (CNAS) method, the present invention is closer to the actual distribution of objects, and the area of ​​misclassification is greatly reduced, which further proves the effectiveness of the proposed method in hyperspectral data classification.

[0127] Table 1 Comparison of classification accuracy of different classification methods for Indian Pines dataset (%)

[0128]

[0129] Table 2 Comparison of classification accuracy of different classification methods for Pavia University dataset (%)

[0130]

Claims

1. A hyperspectral image classification method based on neural architecture search, characterized in that: The following steps are involved: Step a, inputting a hyperspectral image to be classified; Step b, using a sliding window strategy to crop the HSI image and divide it into training samples, verification samples and test samples; Step c: construct a neural architecture search (NAS) space using six preset operations and input the training samples into the space; Step d, using the search strategy of the gradient algorithm to search for candidate network structures, and using β-decay to regularize the architecture parameters during the search process; Step e: Using the confidence learning rate (CLR) to update the structural gradient weight; Step f, grouping the searched preset operations into basic units and connecting them to form an optimal neural network architecture; Step g, classifying HSI using the optimal neural network architecture to obtain a classification result; Step c is specifically as follows: Step c1: Set is a set of candidate operations on directed edges, denoted by the architecture parameter α (m,n) The preset operations are obtained by parameterization. There are 6 preset operations on each edge, namely, None, Skip-Connect, Max pooling, Separate Conv, Dilated-Conv, and Avg pooling. The convolution kernels of each operation are 3×3, 5×5, and 7×7. The above operations are defined as the candidate operation set in the search space. Step c2: input the training samples into the search space.

2. The hyperspectral image classification method based on neural architecture search according to claim 1 is characterized in that: Specifically, step b is to crop the samples by adopting a sliding window strategy, with a window size of 32×32 and an overlap rate of 50%. 300 samples are randomly selected from the samples as the training data set, 200 samples are used as the validation data set, and the remaining samples are used as the test set.

3. The hyperspectral image classification method based on neural architecture search according to claim 1 is characterized in that: Step d is specifically as follows: Step d1, using the Softmax function, the discrete search space is converted into a continuous space; In order to make the search space continuous, α (m,n) The candidate operation set The discretized operation in is transformed into a set of Softmax algorithms of all possible operations; in: It represents the set of all possible operations, and the sum of all coefficients in the operation set is 1. It represents the architecture parameters between nodes (m=0,1,…,N-1; n=0,1,…,N-1). Each unit can have at most N nodes. Step d2: After the Softmax operation, the set of architecture parameters is obtained Parameters Perform attenuation regularization; in: is the combination of the architectural parameters between node m and node n, k is the number of optional operations represented, Step d3, NAS uses the Softmax-activated architecture parameter set β to represent the possibility of each operator. Attenuation regularization of β can limit its value from being too large, which can improve the generalization of the architecture.

4. The hyperspectral image classification method based on neural architecture search according to claim 1, characterized in that: Step e is specifically as follows: Step e1: After the Softmax operation, the architecture parameter α in all mixed operations is α = {α (m,n) The network weights w are jointly learned, and the loss value in the training process is calculated using the cross entropy formula according to the node calculation, and the training loss value (L train ) and validation loss value (L val ); Step e2: According to L train and L val The network parameters and structural parameters are optimized bidirectionally to obtain the network weight parameter w. W is trained in the training data set to obtain the optimal value of the network parameter. After one round of training, the structural weights between each node are evenly distributed. Step e3: perform double-layer optimization on the network parameters and structural parameters according to the loss value, use CLR to update the initial learning rate of the original architecture weight, establish a confidence gradient, and implement cyclic weight update according to this step; Where: t represents the current epoch number of the current training, δ represents the confidence factor, δ = 2, lr α is the learning rate of the NAS architecture, which is set to lr by default α =0.0003.

5. The hyperspectral image classification method based on neural architecture search according to claim 1, characterized in that: Step f is specifically as follows: Step f1: After the weight is updated, the two preset operations with the largest weights before the node of each cell are taken, and the operations and nodes are connected to construct a reduced cell for downsampling and a normal cell for normal sampling; In step f2, the step size of all preset operations in the normal unit is 1, so the size of its output feature map remains unchanged. The step size of all preset operations in the reduction unit is 2, and it is inserted at 1 / 3 and 2 / 3 of the network depth. The reduction unit and the normal unit are connected to obtain the optimal neural network architecture.

6. The hyperspectral image classification method based on neural architecture search according to claim 1, characterized in that: Step g specifically includes using the optimal neural network architecture to classify the HSI to obtain a classification result.

Images