Hsi classification method based on noise interference inspired spatial-spectral neural architecture search

Abstract

The application belongs to the technical field of image classification and is based on a hyperspectral image (HSI) classification method of spatial-spectral neural architecture search (HSI) based on noise interference inspiration. First, the application constructs a modular search space including spatial-spectral attention convolution operators, jump connections, pooling operations and other candidate operations. Then, pixels are randomly selected from the HSI according to a certain proportion as training, verification and test sets and applied to the super network architecture search stage and the final network optimization stage. In the architecture search process, Surpernet constructs a weight sharing model including all possible operations in the network, and searches and optimizes the final operator channel and kernel size from the model architecture while keeping the consistency of the operator and the original model of Surpernet. Then, the application constructs a fair competition environment by a noisy-differentiable architecture search strategy (Noisy-DARTS) and guides the network parameters and model parameters in the training super network to derive the optimal network architecture. Finally, in the performance evaluation stage, the application integrates a label smoothing loss function and a polynomial expansion loss function to construct a fusion loss function to solve the data imbalance problem caused by the long-tail distribution of the HSI samples and automatically adjusts the parameters to design the optimal network architecture for different data sets.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image classification, specifically to a noise interference-inspired spatial-spectral neural architecture search HSI classification method. Background Technology

[0002] Hyperspectral images (HSI) simultaneously contain high spatial resolution and continuous spectral bands of different objects, possessing the unique characteristic of "image-spectrum integration." In the field of remote sensing, the continuous spectra of ground objects acquired by imaging spectrometers can comprehensively reflect the inherent spectral characteristics of ground objects and the subtle differences between them, thereby greatly improving the precision and accuracy of remote sensing image classification. Based on these characteristics, hyperspectral classification technology is widely used in fields such as military reconnaissance, mineral exploration, and urban construction.

[0003] The inherent "homogeneous heterogeneity" and "homogeneous heterogeneity" characteristics of hyperspectral image processing (HSI) significantly impact its classification accuracy. Furthermore, various fields demand high classification accuracy in HSI image processing tasks. Currently, HSI classification methods are primarily based on deep learning. Convolutional neural networks (CNNs) can extract rich spatial-spectral information from HSI end-to-end, providing a robust and reliable feature extraction mechanism for hyperspectral information. However, as model structures become more complex, hand-designed neural networks require substantial time and resources due to their complexity and large number of parameters. Simultaneously, different HSI datasets exhibit significant differences in frequency bands, spectral range, and spatial resolution, leading to varying optimal network structures for different datasets. Therefore, designing a universal classification model applicable to various HSI datasets is extremely challenging.

[0004] The goal of Neural Architecture Search (NAS) is to automate the process of designing optimal network models. This involves selecting and combining different candidate operations from a predefined search space to automatically construct high-performance neural network structures. Differentiable Architecture Search (DARTS) introduces the concept of continuous relaxation, enabling differentiable optimization of the search space. It completes the search for the entire model on a single SuperNet, avoiding the problem of repeatedly training multiple models and significantly improving the search efficiency of NAS. Summary of the Invention

[0005] The technical problems to be solved by this invention are how to construct a high-efficiency search space for HSI classification, the performance collapse problem of NAS, the low classification accuracy, and the misclassification of a few class samples.

[0006] This invention solves the above-mentioned technical problems by employing the following technical solution: a noise interference-inspired spatial-spectral neural architecture search HSI classification method includes:

[0007] Step a: Sample extraction. Pixels are randomly selected from the HSI data according to a certain proportion as the training set and validation set. The training set is divided into two parts, used for the search phase and the training phase respectively, while the remaining samples are reserved as the test set.

[0008] Step b: Construct the modular search space, the Noisy-DARTS search strategy, and the performance evaluation strategy;

[0009] Step c: Set multiple intermediate nodes in the unit as intermediate feature extractors, and guide the training of Supernet with a search strategy. Jointly optimize the search strategy based on the performance evaluation results, and continue to guide the generation of new network structures.

[0010] Step d: Based on the performance evaluation results in step c, derive the final architecture. Select the weights of the Supernet model with the highest validation accuracy to derive the final subnetwork. Based on the weights of the search model, determine the optimal operations between nodes and the internal topology of each cell in each layer.

[0011] Step e: Train the optimal architecture found by searching, and select label smoothing loss and multinomial expansion loss function as the loss function of the optimal architecture. Adjust the importance of different multinomial bases according to the target task and dataset.

[0012] This invention first considers the differences in sample distribution across different HSI datasets. It constructs a new, efficient modular search space based on lightweight convolutional blocks and a spatial-spectral attention module to enhance the representation capability of deep spatial-spectral features, adapting to the characteristics of different dataset sample distributions and thus improving the classification accuracy of HSI. Second, considering the search efficiency of gradient descent-based NAS methods and the unfair competition problem inherent in NAS, this invention employs Noisy-DARTS as the main strategy to achieve an efficient search process, and adds noise interference to skip connections to alleviate the unfair competition problem during the search process. Third, addressing the misclassification problem of minority class samples in the HSI dataset, this invention integrates a fusion loss function combining label smoothing loss function and multinomial expansion loss function as a performance evaluation strategy. By weighting the multinomial coefficients, the loss weight for minority classes is increased, thereby improving the model's classification ability for minority class samples. Finally, to better capture global contextual information, this paper uses MLP for the final classification.

[0013] In a more specific technical solution, the modular search space uses a cell unit as the basic building block, which can be viewed as a directed acyclic graph. Nodes in the graph represent layers in the network, and edges represent possible operations (such as convolution and pooling). It includes spatial-spectral attention convolution operators, skip connections, and pooling operations. The spatial-spectral attention convolution operator maintains the effectiveness of convolution while focusing the receptive field, accurately extracting highly discriminative spatial-spectral features. By applying depthwise separable convolution in a high-dimensional space, it effectively reduces computational complexity, achieving efficient and accurate feature learning. Skip connections effectively prevent gradient vanishing. Pooling operations filter redundant information and increase the nonlinearity of the model. The modular search space contains every network architecture that can be generated by the neural architecture search method.

[0014] In a more specific technical solution, the Noisy-DARTS search strategy injects compliance into skip connections. Distributed Gaussian noise suppresses the competitive advantage of skip connections. Here, μ and σ are intermediate variables. To make the search process differentiable, Noisy-DARTS continuously relaxes the search space, treating each edge as a mixture of all candidate operations (through softmax weights), transforming the network architecture selection into an optimization of the mixture probability. Simultaneously, to address the performance degradation caused by unfair competition during the search process, Noisy-DARTS adds Gaussian noise to skip connections to suppress their competitive advantage, creating a level playing field for the search process.

[0015] In a more specific technical solution, the performance evaluation strategy combines a fusion loss function of label smoothing loss and multinomial expansion loss function. Through Taylor expansion, the classification loss function is decomposed into a series of weighted multinomial bases, which allows the loss function to be more flexibly customized and adjusted for different tasks and datasets.

[0016] In a more specific technical solution, the joint optimization in the search phase is that Nosiy-DARTS simultaneously optimizes two objectives: minimizing the network loss on the training set (i.e., architecture-independent network parameters) and maximizing the mixed probability of the selected operation (i.e., the architecture search task). The joint optimization process is as follows:

[0017]

[0018]

[0019] Among them, L tra and L val These represent the training loss and validation loss, respectively.

[0020] In a more specific technical solution, the final architecture is derived by a search strategy that selects the operation with the highest probability based on the mixed probability of operations on each edge. Noisy-DARTS uses optimization algorithms such as gradient descent to update network parameters (including weights and biases) and operation weights. As training progresses, better-performing candidate operations gradually gain higher weights, while those with poorer performance are gradually eliminated. Finally, the final network architecture is composed of the best-performing candidate operations.

[0021] Beneficial effects:

[0022] This invention relates to the field of remote sensing image classification, specifically a noise-inspired spatial-spectral neural architecture search method for HSI classification. Different HSI datasets exhibit significant differences in spatial resolution and spectral bands. Therefore, no single universal model is suitable for all situations. The core idea of ​​this invention is to automatically design networks for different datasets through neural architecture search, effectively avoiding the time cost of manual design. This invention designs a series of spatial-spectral convolution operations within a modular search space, enabling the model to efficiently extract rich HSI information and achieve high-precision classification capabilities. The invention employs the Nosiy-DARTS search strategy, mitigating performance degradation while achieving efficient search. During the search process, this invention designs a fusion loss function integrating label smoothing loss and multinomial expansion loss function to reduce misclassification of minority class samples. Attached Figure Description

[0023] Figure 1This is an overall flowchart of the noise interference-inspired spatial-spectral neural architecture search HSI classification method in the present invention.

[0024] Figure 2 This is an overall structural diagram of the noise interference-inspired spatial-spectral neural architecture search HSI classification method in the present invention.

[0025] Figure 3 This is a diagram of the spatial-spectral attention convolution structure in the method of this invention.

[0026] Figure 4 This is a schematic diagram of the search process in the method of the present invention.

[0027] Figure 5 It is a pseudo-color image of the PaviaUniversity dataset in the method of this invention.

[0028] Figure 6 This is a classification result diagram of the PaviaUniversity dataset in the method of this invention. Detailed Implementation

[0029] To make the objectives, technical solutions, and features of this invention clearer, the specific embodiments of this invention will be described in further detail below with reference to the accompanying drawings.

[0030] Example 1

[0031] like Figure 1 As shown, a noise-inspired spatial-spectral neural architecture search HSI classification method is presented in this embodiment. The applicable fields of this method include, but are not limited to, remote sensing image classification. In this example, the invention divides the data into training and validation sets for the search phase to complete the search for the final architecture. The training, validation, and test sets are used for training and optimizing the final architecture, and the test set is used to evaluate the classification performance of the final network.

[0032] Example 2

[0033] like Figure 2 As shown, the noise interference-inspired spatial-spectral neural architecture search HSI classification method of the present invention includes the following main modules in this embodiment: modular search space, search strategy, and performance evaluation.

[0034] The modular search space contains all possible neural network architectures. Within this search space, the neural network is modeled as a directed acyclic graph, where each node represents a different layer and operation operator, and edges e... (i,j)The data flow between these nodes is represented by each edge, which contains all candidate operations. The search space consists of a series of cells, each representing a small, reusable network structure. During the search process, through joint optimization of search strategies, the cells are eventually stacked to form a complete neural network. To fully extract the spatial-spectral information of the HSI data, this invention designs spatial-spectral attention convolutions in the search space, such as... Figure 3 As shown, the spatial-spectral attention convolution is actually composed of channel attention and spatial attention, using global max pooling and global average pooling to obtain the attention weights for each channel and space. The end is a depthwise separable convolution module, which reduces the computational complexity of the model while maintaining the effectiveness of convolution.

[0035] In this invention, skip connections and other operations easily form residual structures, thus gaining an advantage during the search process. However, this advantage can lead to the aggregation of skip connections in the architecture, significantly squeezing the survival space of other candidate operations and ultimately causing the architecture's performance to collapse. Therefore, this invention employs noise perturbation to reduce the impact of skip connections, creating a level playing field for candidate operations.

[0036] Its output is shown in the following formula.

[0037]

[0038] In the formula, This represents an operation between two nodes. α o This represents the weight of the candidate operation. α skip Indicates the weight of the skip connection. N This indicates the addition of noise interference.

[0039] To address the imbalanced distribution of HSI data, this invention constructs a fusion loss function using label smoothing loss and multinomial expansion loss. This loss function decomposes the classification loss function into a series of weighted multinomial bases through Taylor expansion. By adjusting the weights of these multinomial bases, the function can assign higher weights to classes with fewer samples, thereby improving classification accuracy.

[0040] The specific process is shown in the following formula.

[0041] L PM =L PL +L SM

[0042] In the formula, L PM L represents the fusion loss function. PL Let L be the loss function for polynomial expansion. SM This is the label smoothing loss function.

[0043] L PL =-(1-P t ) γ log(P t )+ε1(1-P t ) 1+γ

[0044] In the formula, P t This represents the predicted probability. γ is the polynomial power shift. ε is the perturbation term.

[0045]

[0046] In the formula, y i For each class, predict the probability.

[0047] Example 3

[0048] like Figure 4 As shown, the noise-inspired spatial-spectral neural architecture search HSI classification method, in this embodiment, sets multiple intermediate nodes n within the cell. i As a feature extractor, it outputs the C of the cell by connecting intermediate nodes. l The process is shown in the following formula.

[0049]

[0050] The directed acyclic graph has node n. i The edges are determined by the architecture parameter α and the network parameter ω. α represents the weight of different candidate operations on all edges, and ω is the intrinsic parameter of the candidate operation. The search process involves a joint optimization of α and ω.

[0051] The process is shown in the following formula.

[0052]

[0053]

[0054] In the formula, L val and L tra Let represent the validation loss and training loss, respectively. The optimal α is obtained through iterative processing using the aforementioned two-layer joint optimization formula, and the operation with the largest weight is selected as the fixed operation on each edge. Then, the selected candidate operations are discretized, and the operation with the highest weight on the directed edge, o, is selected. (i,j) By discarding other operations, the final neural architecture is formed. Its discrete process is shown in the following equation.

[0055]

[0056] In the formula, O represents the search space.

[0057] Through the above process, this invention searches for the optimal architecture from a search space containing all possible architectures. This architecture is then used for the final HSI classification on the subsequent HSI validation and test sets.

[0058] Example 4

[0059] like Figure 5 As shown. The Pavia University hyperspectral dataset was used in this experiment. The Pavia University dataset was collected by the ROSIS-3 imaging spectrometer sensor. This dataset has 103 spectral bands and a resolution of 610×340 pixels. The images contain 9 classes of samples, totaling 42,776 labeled samples. The spatial resolution is 1.3 m, and the spectral region is 0.43–0.86 μm.

[0060] To verify the classification performance of this invention, experiments were conducted on a Windows 10 operating system. The classification method was implemented using Python and the PyTorch library. The experimental environment consisted of an Intel(R) Core(TM) i7-10750H CPU @ 2.60GHz / 2.59GHz processor, 16GB of RAM, and a GeForce GTX 1650Ti graphics card. The initial learning rate was 0.004, the batch size was set to 32, and the search iterations were 200.

[0061] The classification accuracy of the Pavia University dataset used in this invention is shown in Table 1. Overall accuracy, average accuracy, and Kappa coefficient were used as classification evaluation metrics. The classification results of the Pavia University dataset are shown in Table 1. Figure 6 As shown, experimental results indicate that the present invention exhibits only a small number of misclassifications, closely approximates the actual distribution of ground features, and significantly reduces the area of ​​misclassification.

[0062] Table 1. Classification accuracy of the Pavia University dataset classification method

[0063]

[0064]

Claims

1. A noise-inspired spatial-spectral neural architecture search method for HSI classification, characterized by: Step a: Sample extraction. Randomly select pixels from the HSI data according to a certain proportion as the training set, validation set, and test set. The training set is divided into two parts, used for the search phase and the training phase respectively; Step b: Construct the modular search space, the Noisy-DARTS search strategy, and the performance evaluation strategy; b1. Constructing a modular search space Using a cell as the basic building block, it is viewed as a directed acyclic graph, where nodes represent layers in the network and edges represent possible operations. The search space includes spatial-spectral attention convolution operators, skip connections, and pooling operations. The spatial-spectral attention convolution operator maintains the effectiveness of convolution while focusing the receptive field in depth, so as to accurately extract spatial-spectral features with high discriminativeness. By applying depthwise separable convolution in high-dimensional space, the computational complexity is effectively reduced, and efficient and accurate feature learning is achieved. The skip connections are used to effectively prevent gradient vanishing; the pooling operation is used to filter redundant information and increase the nonlinearity of the model; the modular search space contains every network architecture that can be generated by the neural architecture search method. b2. Design the Noisy-DARTS continuous relaxation search space to make the search process differentiable. Treat each edge as a mixture of all candidate operations. The mixing method is to superimpose softmax weights, turning the choice of network architecture into an optimization problem of mixing probability. b3. The performance evaluation strategy adopts a fusion loss function that integrates the label smoothing loss function and the multinomial expansion loss function to measure the performance of the search architecture; Step c: Set multiple intermediate nodes in the unit as intermediate feature extractors, and train and optimize Supernet under the guidance of the search strategy; jointly optimize the search strategy based on the performance evaluation results, and continue to guide the generation of new network structures; Step d: Based on the performance evaluation in step c, select the weights of the Surpernet model with the highest verification accuracy to derive the final subnetwork; determine the optimal operation between nodes and the internal topology of each cell based on the weights of the search model. Step e: Train the optimal architecture found by searching, and use the fusion loss function that integrates the label smoothing loss function and the multinomial expansion loss function as the loss function of the optimal architecture. Adjust the importance of different multinomial bases according to the target task and dataset to obtain the classification result.

2. The noise interference-inspired spatial-spectral neural architecture search HSI classification method according to claim 1, characterized in that, Step c includes: c1. Input training set Validation set ; c2. Update the network gradient according to step c, and alternately jointly optimize the network weights. and architecture parameters The optimization method is as follows: Among them, L tra and L val These represent the training loss and validation loss, respectively. c3. Validate the prediction accuracy of the network on the validation set and record the loss and validation accuracy.

3. The noise interference-inspired spatial-spectral neural architecture search HSI classification method according to claim 1, characterized in that, In step d, only the weights of the Surpernet model with the highest verification accuracy are retained to derive the final network architecture; based on the weights of the search model, the internal topology of each cell is determined.

4. The noise interference-inspired spatial-spectral neural architecture search HSI classification method according to claim 1, characterized in that, The fusion loss function constructed in step e enables the network to increase the loss weight for minority class samples through polynomial coefficient weighting, thereby improving the model's ability to classify minority class samples.

Images