A hyperspectral image classification method, system, device and medium based on proxy-assisted evolutionary convolutional attention network architecture search

Abstract

The application discloses a hyperspectral image classification method, system, device and medium based on proxy-assisted evolutionary convolution attention network architecture search, and has the following advantages: 1. By adopting the NAS technology assisted by the proxy model, the time cost and resource consumption of the evolutionary architecture search algorithm can be reduced; 2. By adopting the proxy data enhancement technology based on the chromosome base type, the problem of insufficient training data of the existing proxy model can be solved; 3. By adopting a novel proxy model training method, the training of the proxy model is divided into global and local search, so that the proxy model can well adapt to the prediction environment of the proxy model which changes with the population evolution; 4. By adopting the coding of fusing a convolutional neural network and multiple attention modules, the algorithm robustness can be improved and the calculation resources can be reasonably utilized; and 5. By adopting efficient crossover and mutation operators, the network depth and chromosome gene can be adaptively adjusted, and the algorithm search capability can be enhanced and the classification efficiency can be improved.

Description

Technical Field

[0001] This invention belongs to the field of spectral image classification technology, specifically relating to a hyperspectral image classification method, system, device, and medium based on a surrogate-assisted evolutionary convolutional attention network architecture search. Background Technology

[0002] In recent years, hyperspectral images have been widely used in geological exploration, agriculture and other fields. With the large-scale generation of hyperspectral images, the industry has put forward new challenges to the task of land cover classification.

[0003] Patent application CN109376804A proposes a hyperspectral remote sensing image classification method based on attention mechanism and convolutional neural network. This method uses a manually constructed convolutional neural network to solve the hyperspectral image classification problem. The steps are as follows: (1) Principal component analysis is used to reduce the dimensionality of the original hyperspectral image and extract sample blocks; (2) 3D convolution and pooling operations are performed on the sample blocks to obtain feature maps; (3) The feature maps are passed through spectral and spatial attention modules, and 3D convolution and pooling operations are performed again; (4) The final feature maps are input into the classifier to complete the hyperspectral image classification. This method improves the accuracy of hyperspectral image classification. However, this method uses a manually designed neural network for classification, which places high demands on the designer's model design experience and hyperspectral knowledge. For different hyperspectral images, the workload of manually designing the neural network architecture is too large, and manually designing the neural network architecture is prone to structural redundancy and human error, affecting classification efficiency and accuracy.

[0004] Patent application CN114882294A proposes a multi-objective evolutionary convolutional neural network for hyperspectral image classification. This method uses a tree-like multi-branch supernetwork structure as the basic structure of the network blocks to achieve the extraction and fusion of multi-scale features. Simultaneously, it encodes the size and spectral dimension of the input image blocks, using both classification accuracy and network complexity as optimization objectives. This achieves the search for convolutional neural networks with high classification accuracy and low network structure complexity. The specific implementation steps include: (1) initializing the architecture information and encoding the convolutional neural network to be constructed; (2) generating a dataset of individual genes and using principal component analysis (PCA) to reduce the dimensionality of the input spectral dimension; (3) constructing and training the convolutional neural network corresponding to each individual gene in the population; (4) calculating the two fitnesss of the convolutional neural network and performing crossover and mutation on the individuals in the population; (5) selecting the population with the maximum number of iterations as the optimal population, and then selecting the convolutional neural network corresponding to the individual gene with the highest classification accuracy in the optimal population as the optimal convolutional neural network for hyperspectral image classification. This technique effectively solves the problems of existing technologies, such as the inability to search for the size and spectral dimension of input image patches in convolutional neural networks, low classification efficiency, excessive human error, low classification accuracy, and complex network models. However, this method requires extensive training and evaluation of network models during multi-objective evolution, resulting in huge computational resource consumption and low search efficiency. Summary of the Invention

[0005] To overcome the shortcomings of the prior art, the present invention aims to provide a hyperspectral image classification method, system, device and medium based on surrogate-assisted evolutionary convolutional attention network architecture search. The method: (1) reduces the time cost and resource consumption of evolutionary architecture search algorithm by using surrogate model-assisted NAS technology; (2) solves the problem of insufficient training data for existing surrogate models by using surrogate data augmentation technology based on chromosome base types; (3) divides the training of surrogate models into two cases, global and local search, by using a novel surrogate model training method, which enables the surrogate model to adapt well to the prediction environment of the surrogate model that changes with population evolution; (4) improves the robustness of the algorithm and makes reasonable use of computing resources by using an encoding that integrates convolutional neural networks and multiple attention modules; (5) adaptively adjusts the network depth and chromosome genes by using efficient crossover and mutation operators, which has the advantages of enhancing the algorithm's search ability and improving classification efficiency.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A hyperspectral image classification method based on agent-assisted evolutionary convolutional attention network architecture search includes the following steps:

[0008] S1. Define the search space, which includes three sub-network blocks. Each sub-network block includes feature scale, convolution operation type, number of convolution output channels, multi-scale aggregation mode encoding, attention mechanism, skip connections, and pooling operation.

[0009] S2. After searching for the corresponding operations from the three sub-network blocks in the search space of step S1 according to the evolutionary search strategy, the three sub-network blocks are concatenated to obtain a convolutional neural network.

[0010] S3. The feature scale, convolution operation type, number of convolution output channels, multi-scale aggregation mode encoding, attention mechanism, skip connections and pooling operation of the convolutional neural network constructed in step S2 are encoded into a chromosome through a scalar hybrid encoding method. The total encoding length is 41, and the chromosome is divided into discrete basis and continuous basis.

[0011] S4. Proxy model-assisted evolutionary search: The chromosomes encoded in step S3 are randomly initialized to form a population. The Spearman index c is initialized to 0 and the fitness range r to 1. The parent and offspring populations are merged. If the conditions r ≥ 0.05 and c ≥ 0.8 are met, the proxy model is used to predict the fitness of the current population. Finally, the next generation population is selected according to the hierarchical selection strategy. If the conditions are not met, the current population is truly trained and evaluated. Finally, the roulette wheel selection strategy is used to select the next generation population.

[0012] S5. Perform crossover operation on the individuals in the population obtained in step S4, and form a crossover population from all the crossover offspring.

[0013] S6. Perform mutation operation on the individuals in the crossover population obtained in step S5, and form a mutation population from all the mutated offspring.

[0014] S7. Combine all individuals in the crossover population obtained in step S5 with all individuals in the variant population obtained in step S6 to form the offspring population.

[0015] S8. If the current iteration number is less than the maximum iteration number, increment the iteration number by 1 and then execute step S4. Otherwise, take the population of the current iteration as the optimal population, perform real training evaluation on the optimal population, and select the convolutional neural network corresponding to the individual with the highest classification accuracy from the optimal population as the optimal convolutional neural network.

[0016] S9. Input the hyperspectral image to be classified into the best-trained convolutional neural network selected in step S8 to obtain the final classification result, and calculate the classification accuracy of the convolutional neural network for all samples to be classified in the hyperspectral image, and output the corresponding classification accuracy as the result.

[0017] Specifically, the hybrid coding method in step S3 defines the structure of the chromosome as follows:

[0018] [R,I,S,T,O,T,O,T,O,G,A,J,P]

[0019] Where R represents the encoding of the input image size, with a value randomly selected from the range [19, 49]; I represents the encoding of the number of spectral channels of the input image, with a value randomly selected from the range [50, 95]; S represents the encoding of different convolution scales, with a value randomly selected from the range [16, 128]; T represents the encoding of different convolution operations, with a value randomly selected from [2D Conv, 2D SepConv, 3D Conv, 3D SepConv], where [2D Conv, 2D SepConv, 3D Conv, 3D SepConv] represent two-dimensional ordinary convolution, two-dimensional separable convolution, three-dimensional ordinary convolution, and three-dimensional separable convolution, respectively; O represents the encoding of the number of output channels of the convolution operation, with a value randomly selected from the range [16, 128]; G represents the encoding of the aggregation strategy of the sub-model, with a value randomly selected from [Sum, Weighted Sum, Concatenate], where [Sum, Weighted Sum, Concatenate] represents the number of output channels of the convolution operation, with a value randomly selected from the range [16, 128]; and G represents the encoding of the aggregation strategy of the sub-model, with a value randomly selected from [Sum, Weighted Sum, Concatenate]. [Sum, Concatenate] represent the direct summation of multi-scale features, the weighted summation of multi-scale features, and concatenation along the channel dimension, respectively. A represents the encoding of the attention mechanism, with a value randomly selected from [Spatital, Spectrum, Cross], where [Spatital, Spectrum, Cross] represent spatial attention, spectral attention, and cross attention, respectively. J represents the encoding of skip connections, with a value of 0 or 1, representing no skip connections and skip connections, respectively. P represents the encoding of pooling operations, with a value randomly selected from [Average, Max, 2×2], where [Average, Max, 2×2] represent average pooling, max pooling, and 2×2 pooling, respectively.

[0020] Specifically, step S4 is as follows:

[0021] S401. Set the maximum number of iterations to G, initialize the population size to S, and represent the parent population as P. i ={p1,p2,...,p S}, the offspring population P i ′ is set to empty, that is

[0022] S402. Initialize the Spearman metric c to 0 and the fitness range r to 1, while setting the parent population P... i and offspring population Pi 'Merge, i.e., P i "=P i ′∪P i If the conditions r ≥ 0.05 and c ≥ 0.8 are met, this stage is called the global search, and a surrogate model is used to predict the current population P. i The fitness of "" is used to obtain the level mapping R. i Mapping from level R i The highest and lowest ranked individuals are obtained and stored in the agent model's data archive A; finally, the next generation population P is selected according to the ranking-based selection strategy. i+1 ;

[0023] S403. If the conditions r≥0.05 and c≥0.8 are not met, then for the current population P... i "Perform actual training and evaluation to obtain the true classification accuracy of all chromosomes in the population {P}" i ",F i "}, using {P i ",F i Refresh proxy data archive A;

[0024] S404. Randomly sample S chromosomes from the data archive A of the surrogate model to form a dataset D, and divide the dataset D into a training set D according to a certain ratio. train and test set D test Then the training set D train and test set D test Convert to paired datasets All chromosomes are encoded into pairs and assembled. The labels of the encoded chromosomes are defined based on the classification accuracy of the corresponding networks of the preceding and following chromosomes, using the following formula:

[0025]

[0026] Among them, f i and f j Representing the classification accuracy of chromosomes i and j in data archive A, respectively, y i,j The label represents the training data after chromosome splicing. If the true evaluation accuracy of the first chromosome is higher than that of the second chromosome, the label is 1, indicating that a clear relationship has been established.

[0027] S405. Using MLP as an alternative to logistic regression, the input data is converted into binary encoding to enhance feature information and sparsity. The optimal MLP parameters are found using the training data in data archive A. When the surrogate model is retrained and tuned subsequently, the previously existing model parameters are used directly. The surrogate model is trained by calculating the cross-entropy loss function. After obtaining the relationship between the two chromosomes, the ranking of the entire population is inferred. The loss function is as follows:

[0028]

[0029] Where N represents the batch size, y i y represents the true label of sample i. i ′ represents the probability that sample i is predicted as positive, and L is the calculated loss;

[0030] S406. Supplement the training set D using a heterogeneous chromosome combination strategy. train Chromosomal loci are divided into bases representing the model's backbone structure and bases representing the size of features. The bases of the model backbone determine the model's depth, width, and structure. The individuals with the best and worst classification accuracy in the population are identified, and a large amount of pseudo-label data is generated based on their genes. The third gene of the best individual is a modifiable gene, and a new code is obtained from data randomly selected from its neighborhood. The first gene of the worst individual is a modifiable gene, and a new code is obtained from data randomly selected from its neighborhood.

[0031] S407. The data in data archive A is augmented using step 406. The augmented data archive A is then used to retrain the surrogate model. Finally, a roulette wheel selection strategy is used to select the next generation population P. i+1 .

[0032] Specifically, in step S5, the crossover operation on the population is performed as follows:

[0033] S501. From the parent population of the current iteration, use the tournament algorithm to select two gene individuals corresponding to the convolutional neural networks to be optimized.

[0034] S502. Perform crossover operation on the two selected gene individuals according to the crossover probability cp. Generate a random number that satisfies a uniform distribution of 0-1 for each selected gene individual. If the random number is less than cp, 0≤cp≤1, then perform a two-point crossover operation on the two selected gene individuals.

[0035] S503. Determine whether all gene individuals have been selected in the parent population after the current iteration. If so, form a crossover population by combining all gene individuals obtained through the above crossover operation in the parent population after the current iteration, and then execute step S6. Otherwise, execute step S501.

[0036] Specifically, in step S6, the mutation operation on the population is performed as follows:

[0037] S601. When the search mode m is a global search, perform mutation operations on the gene individuals in the crossover population obtained in step S503 according to probability; generate a random number that satisfies a uniform distribution of 0-1 for each selected pair of gene individuals. If the random number is less than mp and 0≤mp≤1, then perform a single-point mutation operation on the two selected gene individuals.

[0038] S602. Combine all the gene individuals obtained from the above mutation operation in the crossover population into a variant population.

[0039] A hyperspectral image classification system based on agent-assisted evolutionary convolutional attention network architecture search includes:

[0040] The search space definition module is used to define the search space, which includes three sub-network blocks. Each sub-network block includes feature scale, convolution operation type, number of convolution output channels, multi-scale aggregation mode encoding, attention mechanism, skip connections, and pooling operation.

[0041] The convolutional neural network building module is used to search for the corresponding operations from the three sub-network blocks in the search space according to the evolutionary search strategy, and then concatenate the three sub-network blocks to obtain the convolutional neural network.

[0042] The chromosome encoding construction module is used to encode the feature scale, convolution operation type, number of convolution output channels, multi-scale aggregation mode, attention mechanism, skip connections, and pooling operation of the constructed convolutional neural network into chromosomes through a scalar hybrid encoding method. The total encoding length is 41, and the chromosomes are divided into discrete basis and continuous basis.

[0043] The surrogate model-assisted evolutionary search module is used to randomly initialize the encoded chromosomes to form a population. The Spearman index c is initialized to 0 and the fitness range r is set to 1. The parent and offspring populations are merged. If the conditions r ≥ 0.05 and c ≥ 0.8 are met, the surrogate model is used to predict the fitness of the current population, and the next generation population is selected according to the hierarchical selection strategy. If the conditions are not met, the current population is trained and evaluated, and the next generation population is selected using the roulette wheel selection strategy.

[0044] The crossover population acquisition module is used to perform crossover operations on individuals in the population and form a crossover population from all the crossover offspring.

[0045] The mutant population acquisition module is used to perform mutation operations on individuals in the crossover population and combine all the mutated offspring into a mutant population.

[0046] The offspring population acquisition module is used to combine all individuals in the crossover population with all individuals in the mutation population to form an offspring population.

[0047] The optimal convolutional neural network acquisition module is used to implement the following: if the current iteration number is less than the maximum iteration number, the iteration number is incremented by 1 and a surrogate model is executed to assist in evolutionary search; otherwise, the population in the current iteration is taken as the optimal population, and the optimal population is trained and evaluated in real time. The convolutional neural network corresponding to the individual with the highest classification accuracy in the optimal population is selected as the optimal convolutional neural network.

[0048] The result output module is used to input the hyperspectral image to be classified into the trained optimal convolutional neural network to obtain the final classification result, calculate the classification accuracy of the convolutional neural network for all samples to be classified in the hyperspectral image, and output the corresponding classification accuracy as the result.

[0049] This invention also provides a hyperspectral image classification device based on a surrogate-assisted evolutionary convolutional attention network architecture search, comprising:

[0050] Memory: A computer-readable device that stores the computer program of the above-described hyperspectral image classification method based on agent-assisted evolutionary convolutional attention network architecture search;

[0051] Processor: Used to implement the hyperspectral image classification method based on agent-assisted evolutionary convolutional attention network architecture search when executing the computer program.

[0052] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, enables the implementation of the hyperspectral image classification method based on a surrogate-assisted evolutionary convolutional attention network architecture search.

[0053] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0054] 1) This invention employs a proxy model-assisted evolutionary neural architecture search technique, which uses evolutionary computation to optimize the structural parameters of convolutional neural networks and automatically constructs ideal model structures and related parameters. This replaces a large number of model training and evaluation stages in existing evolutionary neural architecture search techniques, reducing the time cost and resource consumption of evolutionary architecture search algorithms. It can efficiently and accurately search for neural network models suitable for hyperspectral image classification.

[0055] 2) This invention designs a proxy data augmentation technique based on chromosome base types, which solves the problem of insufficient training data for existing proxy models and has the advantages of improving and stabilizing the performance and prediction accuracy of proxy models.

[0056] 3) This invention designs a novel surrogate model training method. Through global and local search modes, the surrogate model can adapt well to the prediction environment of the surrogate model that changes with the evolution of the population.

[0057] 4) This invention designs an encoding that integrates convolutional neural networks and multiple attention modules, and combines multi-scale convolutional parameter kernel attention mechanism, which has the advantages of improving algorithm robustness and making reasonable use of computing resources.

[0058] 5) This invention designs efficient crossover and mutation operators, enabling individuals to obtain neural network models of different depths through crossover and mutation operations. It adaptively adjusts the network depth and chromosome genes to obtain suitable multi-scale features, which has the advantages of enhancing the algorithm's search capability and improving classification efficiency.

[0059] In summary, compared with existing technologies, this invention can reduce the time cost and resource consumption of evolutionary architecture search algorithms, thereby efficiently and accurately searching for neural network models suitable for hyperspectral image classification; it solves the problem of insufficient training data for existing surrogate models, improves the robustness of the algorithm and the ability to make reasonable use of computing resources, and improves classification efficiency and enhances the algorithm's search capability; this invention has the advantage of improving and stabilizing the performance and prediction accuracy of surrogate models. Attached Figure Description

[0060] Figure 1 This is the overall flowchart of the present invention.

[0061] Figure 2 This is an example diagram of the chromosome genotype and decoding network of the present invention.

[0062] Figure 3 This is a schematic diagram illustrating the data enhancement of a single gene change according to the present invention.

[0063] Figure 4 shows the classification results of the IndianPines hyperspectral image by the present invention and the prior art. Figure 4(a) is the ground truth annotation image, Figure 4(b) is the classification result of the present invention, Figure 4(c) is the classification result of EB-CNN, and Figure 4(d) is the classification result of SSRN.

[0064] Figure 5 shows the classification results of PaviaU hyperspectral images by the present invention and existing technologies. Figure 5(a) is the ground truth annotation image, Figure 5(b) is the classification result of the present invention, Figure 5(c) is the classification result of EB-CNN, and Figure 5(d) is the classification result of SSRN.

[0065] Figure 6 shows the classification results of the Houston2013 hyperspectral image by the present invention and the prior art. In Figure 6(a), the ground truth annotation is shown. Figure 6(b) shows the classification results of the present invention. Figure 6(c) shows the classification results of EB-CNN. Figure 6(d) shows the classification results of SSRN. Detailed Implementation

[0066] The technical solution adopted by the present invention will be further described below with reference to the accompanying drawings.

[0067] like Figure 1 As shown, a hyperspectral image classification method based on agent-assisted evolutionary convolutional attention network architecture search includes the following steps:

[0068] S1. Define the search space, which includes three sub-network blocks. Each sub-network block includes feature scale, convolution operation type, number of convolution output channels, multi-scale aggregation mode encoding, attention mechanism, skip connections, and pooling operation.

[0069] S2. After searching for the corresponding operations from the three sub-network blocks in the search space of step S1 according to the evolutionary search strategy, the three sub-network blocks are concatenated to obtain a convolutional neural network.

[0070] S3. The feature scale, convolution operation type, number of convolution output channels, multi-scale aggregation mode encoding, attention mechanism, skip connections and pooling operation of the convolutional neural network constructed in step S2 are encoded into a chromosome through a scalar hybrid encoding method. The total encoding length is 41, and the chromosome is divided into discrete basis and continuous basis.

[0071] Specifically, the hybrid coding method in step S3 defines the structure of the chromosome as follows:

[0072] [R,I,S,T,O,T,O,T,O,G,A,J,P]

[0073] Where R represents the encoding of the input image size, with a value randomly selected from the range [19, 49]; I represents the encoding of the number of spectral channels of the input image, with a value randomly selected from the range [50, 95]; S represents the encoding of different convolution scales, with a value randomly selected from the range [16, 128]; T represents the encoding of different convolution operations, with a value randomly selected from [2D Conv, 2D SepConv, 3D Conv, 3D SepConv], where [2D Conv, 2D SepConv, 3D Conv, 3D SepConv] represent two-dimensional ordinary convolution, two-dimensional separable convolution, three-dimensional ordinary convolution, and three-dimensional separable convolution, respectively; O represents the encoding of the number of output channels of the convolution operation, with a value randomly selected from the range [16, 128]; G represents the encoding of the aggregation strategy of the sub-model, with a value randomly selected from [Sum, Weighted Sum, Concatenate], where [Sum, Weighted Sum, Concatenate] represents the number of output channels of the convolution operation, with a value randomly selected from the range [16, 128]; and G represents the encoding of the aggregation strategy of the sub-model, with a value randomly selected from [Sum, Weighted Sum, Concatenate]. [Sum, Concatenate] represent the direct summation of multi-scale features, the weighted summation of multi-scale features, and concatenation along the channel dimension, respectively. A represents the encoding of the attention mechanism, with a value randomly selected from [Spatital, Spectrum, Cross], where [Spatital, Spectrum, Cross] represent spatial attention, spectral attention, and cross attention, respectively. J represents the encoding of skip connections, with a value of 0 or 1, representing no skip connections and skip connections, respectively. P represents the encoding of pooling operations, with a value randomly selected from [Average, Max, 2×2], where [Average, Max, 2×2] represent average pooling, max pooling, and 2×2 pooling, respectively. Based on the numerical properties of the searchable range and the function of each gene in the corresponding network model, chromosomal genes are divided into discrete and continuous bases. These bases not only reflect the influence of different bases on model performance in genetic operations but can also be used as part of the feature engineering of alternative prediction models. Discrete basis includes convolution scale, convolution operation, aggregation strategy, attention mechanism, skip connection and pooling operation, while continuous basis includes input image size, number of spectral channels of input image and number of output channels of convolution operation.

[0074] like Figure 2As shown, in S4, a surrogate model-assisted evolutionary search is used. The chromosomes encoded in step S3 are randomly initialized to form a population. The Spearman index c is initialized to 0 and the fitness range r is initialized to 1. The parent and offspring populations are merged. If the conditions r ≥ 0.05 and c ≥ 0.8 are met, the surrogate model is used to predict the fitness of the current population. Finally, the next generation population is selected according to the hierarchical selection strategy. If the conditions are not met, the current population is truly trained and evaluated. Finally, the roulette wheel selection strategy is used to select the next generation population.

[0075] Specifically, the steps for training the agent model in step S4 are as follows:

[0076] S401. Set the maximum number of iterations to G, initialize the population size to S, and represent the parent population as P. i ={p1,p2,...,p S Since evolution has not yet begun, for subsequent processing, the offspring population P... i ′ is set to empty, that is

[0077] S402. Initialize the Spearman metric c to 0 and the fitness range r to 1, while setting the parent population P... i and offspring population P i 'Merge, i.e., P i "=P i ′∪P i If the conditions r ≥ 0.05 and c ≥ 0.8 are met, this stage is called global search, indicating that the current surrogate model is capable of performing the optimization task of the current population. Therefore, the surrogate model is used to predict the current population P. i The fitness of "" is used to obtain the level mapping R. i Mapping from level R i The system obtains the highest and lowest ranked individuals and stores them in the agent model's data archive A; finally, it selects the next generation population P based on a ranking-based selection strategy. i+1 ;

[0078] S403. If the conditions r≥0.05 and c≥0.8 are not met, then for the current population P... i "Perform actual training and evaluation to obtain the true classification accuracy of all chromosomes in the population {P}" i ",F i "}, using {P i ",F i Refresh proxy data archive A;

[0079] S404. Randomly sample S chromosomes from the data archive A of the surrogate model to form a dataset D, and divide the dataset D into a training set D according to a certain ratio.train and test set D test Then the training set D train and test set D test Convert to paired datasets Specifically, all chromosomes are encoded into pairs and combined. If there are n data individuals in the file, then after pairwise encoding, there are a total of n(n-1) / 2 training data. The labels of the encoded chromosomes are defined based on the classification accuracy of the corresponding networks before and after the chromosomes, and the formula is as follows:

[0080]

[0081] Among them, f i and f j Representing the classification accuracy of chromosomes i and j in data archive A, respectively, y i,j The label represents the training data after chromosome splicing. If the true evaluation accuracy of the first chromosome is higher than that of the second chromosome, the label is 1, indicating that a clear relationship has been established.

[0082] S405. This invention uses MLP as an alternative to logistic regression, converting input data into binary encoding to enhance feature information and sparsity, thereby improving the model's feature extraction capabilities. The training of the proxy model differs in each scenario during the optimization process. This invention uses training data in data archive A to find the optimal MLP parameters. When the proxy model is subsequently retrained and adjusted, this invention directly uses the previously existing model parameters. This invention completes the training of the proxy model by calculating the cross-entropy loss function. After obtaining the relationship between the two chromosomes, the ranking of the entire population can be inferred. The loss function is as follows:

[0083]

[0084] Where N represents the batch size, y i y represents the true label of sample i. i ′ represents the probability that sample i is predicted as positive, and L is the calculated loss;

[0085] S406. Supplement the training set D using a heterogeneous chromosome combination strategy. trainThis invention proposes a heterogeneous chromosome combination strategy to expand data and alleviate the overfitting problem of MLP. Specifically, this invention divides chromosomal loci into bases representing the model's backbone structure and bases representing feature sizes. The bases of the model backbone determine the model's depth, width, and structure, which has a profound impact on the performance of the feature extractor. The real-number coding search space constructed in this invention shows that, without changes in backbone genes, the final classification accuracy of the model will not be significantly affected by a random change in a single base, which represents the feature size of one of the two chromosomes in an adjacent region. Based on the above theory, the two individuals with the greatest differences in the population, i.e., the individual with the best and worst classification accuracy, can be found, and a large amount of pseudo-label data can be generated based on their genes. Figure 3 As shown, the third gene of the best individual is a modifiable gene, so a new code is obtained from randomly selected data in its neighborhood; the first gene of the worst individual is a modifiable gene, so a new code is obtained from randomly selected data in its neighborhood. The classification accuracy corresponding to these two new codes is difficult to predict and accurate, but since their skeletal genes come from the best and worst individuals, the relationship between them remains unchanged, thus obtaining new pseudo-label data;

[0086] S407. The data in data archive A is augmented using step 406. The augmented data archive A is then used to retrain the surrogate model. Finally, a roulette wheel selection strategy is used to select the next generation population P. i+1 .

[0087] S5. Perform crossover operation on the individuals in the population obtained in step S4, and form a crossover population from all the crossover offspring.

[0088] Specifically, in step S5, the crossover operation on the population is performed as follows:

[0089] S501. From the parent population of the current iteration, use the tournament algorithm to select two gene individuals corresponding to the convolutional neural networks to be optimized.

[0090] S502. Perform crossover operation on the two selected gene individuals according to the crossover probability cp. Generate a random number that satisfies a uniform distribution of 0-1 for each selected gene individual. If the random number is less than cp, 0≤cp≤1, then perform a two-point crossover operation on the two selected gene individuals.

[0091] S503. Determine whether all gene individuals have been selected in the parent population after the current iteration. If so, form a crossover population by combining all gene individuals obtained through the above crossover operation in the parent population after the current iteration, and then execute step S6. Otherwise, execute step S501.

[0092] S6. Perform mutation operation on the individuals in the crossover population obtained in step S5, and form a mutation population from all the mutated offspring.

[0093] Specifically, in step S6, the mutation operation on the population is performed as follows:

[0094] S601. When the search mode m is a global search, perform mutation operations on the gene individuals in the crossover population obtained in step S503 according to probability; in order to explore a wider search space. Generate a random number that satisfies a uniform distribution of 0-1 for each selected pair of gene individuals. If the random number is less than mp, 0≤mp≤1, then perform a single-point mutation operation on the two selected gene individuals.

[0095] S602. Combine all the gene individuals obtained from the above mutation operation in the crossover population into a variant population.

[0096] S7. Combine all individuals in the crossover population obtained in step S5 with all individuals in the variant population obtained in step S6 to form the offspring population.

[0097] S8. If the current iteration number is less than the maximum iteration number, increment the iteration number by 1 and then execute step S4. Otherwise, take the population of the current iteration as the optimal population, perform real training evaluation on the optimal population, and select the convolutional neural network corresponding to the individual with the highest classification accuracy from the optimal population as the optimal convolutional neural network.

[0098] S9. Input the hyperspectral image to be classified into the best-trained convolutional neural network selected in step S8 to obtain the final classification result, and calculate the classification accuracy of the convolutional neural network for all samples to be classified in the hyperspectral image, and output the corresponding classification accuracy as the result.

[0099] Application prospects of this invention

[0100] With the widespread application of hyperspectral images in military and civilian fields, the demand for hyperspectral image classification is increasing. Traditional manual design of neural network models is no longer sufficient to meet the model design requirements of massive hyperspectral images. In addition, as the industry's requirements for hyperspectral image classification become increasingly stringent, existing technologies can no longer simultaneously meet the industry's requirements for high accuracy and high efficiency. The advantages of this invention are: (1) It adopts an evolutionary neural architecture search technology assisted by a surrogate model, uses evolutionary computation to optimize the structural parameters of the convolutional neural network, and automatically constructs an ideal model structure and related parameters, replacing a large number of model training and evaluation stages in the existing evolutionary neural architecture search technology, reducing the time cost and resource consumption of the evolutionary architecture search algorithm, and enabling efficient and high-precision search for neural network models suitable for hyperspectral image classification; (2) It designs a surrogate data augmentation technology based on chromosome base types, ensuring the diversity of surrogate model training data, solving the problem of insufficient training data for existing surrogate models, and has the advantages of improving and stabilizing the ability of the surrogate model to identify structural features and predict accuracy during the evolution process; (3) It designs a novel surrogate model training method, dividing the training of the surrogate model into two cases: global search and local search. In the global search, individuals with the highest and lowest ranks are obtained through rank mapping and added to the surrogate data archive; in the local search, the classification accuracy of the real trained gene individuals is used to refresh the surrogate data archive. With the evolutionary iteration process, the surrogate model can adapt well to the prediction environment of the surrogate model that changes with the population evolution in this way. (4) Design an encoding that integrates convolutional neural networks and multiple attention modules, and mixes multi-scale convolutional parameter kernel attention mechanism, which has the advantages of improving the robustness of the algorithm and making reasonable use of computing resources; (5) Design efficient crossover and mutation operators, so that individuals can obtain neural network models of different depths through crossover and mutation operations, adaptively adjust the network depth and chromosome genes, thereby obtaining suitable multi-scale features, which has the advantages of enhancing the algorithm's search ability and improving classification efficiency.

[0101] The effects of this invention will be further illustrated below with simulation experiments:

[0102] 1. Simulation experimental conditions:

[0103] The hardware platform for the simulation experiment of this invention is an Nvidia GeForce RTX 3090 GPU model.

[0104] The software platform for the simulation experiment of this invention is: Ubuntu 22.10 system, Python 3.9, PyTorch 1.13.

[0105] The input images used in the simulation experiments of this invention are: IndianPines hyperspectral dataset, PaviaU hyperspectral dataset, and Houston2013 hyperspectral dataset.

[0106] This invention sets the crossover probability (cp) and mutation probability (mp) to 0.75 and 0.25, respectively, the population size (S) to 10, and the maximum number of iterations (G) to 50. The invention uniformly sets the training parameters for each chromosome, and all models undergo 120 rounds of training. The optimizer used is Adam, with a momentum of 0.9 and a learning rate of 0.001. To accommodate the training phase, the batch size is set to 16, and L2 regularization with weight decay of 0.0003 is employed to reduce model overfitting.

[0107] 2. Simulation content and result analysis:

[0108] The simulation experiment of this invention uses the present invention, along with existing technologies EB-CNN and SSRN, to classify labeled samples in the IndianPines hyperspectral dataset, PaviaU hyperspectral dataset, and Houston2013 hyperspectral dataset, respectively.

[0109] The existing technology EB-CNN refers to a convolutional neural network structure search method proposed by Z. Liu et al. in "Evolving Block-Based Convolutional Neural Network for Hyperspectral Image Classification, IEEE Trans. Geosci. Remote Sens., vol. 60, pp. 1-21, 2022, Art no. 5525921" for solving the problem of land cover classification in hyperspectral images.

[0110] The existing technology SSRN refers to an artificially designed end-to-end spectral spatial residual network proposed by Zhong et al. in “Spectral–Spatial Residual Network for Hyperspectral Image Classification: A 3-D Deep Learning Framework, IEEE Transactions on Geoscience and Remote Sensing 56(2018):847-858.” for solving the problem of land cover classification in hyperspectral images.

[0111] The simulation content of this invention is as follows:

[0112] Simulation Experiment 1 uses the present invention and existing technologies EB-CNN and SSRN to simulate Indian Pines hyperspectral image data. Indian Pines is the earliest test data used for hyperspectral image classification. It was imaged in 1992 by an airborne visible infrared imaging spectrometer (AVIRIS) on an Indian pine forest in Indiana, USA. The imaging wavelength range is 0.4-2.5μm. The ground features were continuously imaged in 220 consecutive bands. Since bands 104-108, 150-163, and 220 cannot be reflected by water, 200 bands were retained after correction as the research object. Then, pixels of size 145×145 were cropped and labeled as hyperspectral image classification test data. The actual labeled image used in the simulation experiment of the present invention is shown in Figure 4(a). Figure 4(a) shows a total of 16 types of land features. The regions in the image, from darkest to lightest, represent categories 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, and 16, with a total of 10249 labeled samples. Simulation Experiment 1 used 5% of the labeled samples from this image to form a training set, 5% to form a validation set, and 100% to form a test set. The simulation yielded the classification results of this invention and existing technologies EB-CNN and SSRN for each sample in the test set. The classification results obtained by this invention and existing technologies EB-CNN and SSRN are plotted in Figures 4(b), 4(c), and 4(d), respectively.

[0113] As can be seen from the comparison between the classification results shown in Figures 4(b), 4(c), and 4(d) and the actual labeled map shown in Figure 4(a), although there are a few misclassified samples in some categories in the classification results obtained by the method of the present invention, the classification results of the vast majority of samples are correct and very close to the actual labeled map shown in Figure 4(a). In contrast, in the classification results obtained by the prior art EB-CNN and SSRN, in addition to a large number of small-area misclassified samples, there are also a large number of concentrated misclassified samples of some types of ground objects, with more obvious errors and wider distribution.

[0114] To verify the effectiveness of simulation experiment 1 of this invention, the classification results of the method of this invention and the existing technologies EB-CNN and SSRN were quantified using three indicators: classification accuracy of each type of land cover, overall classification accuracy, and average classification accuracy.

[0115] Using the calculation formulas of the following three indicators, the various types of ground targets in the simulation experiment 1 were quantified, and the results are shown in Table 1.

[0116]

[0117]

[0118]

[0119] Table 1. Quantitative Results of Simulation Experiment 1

[0120]

[0121] Table 1 lists the calculated classification accuracy, average classification accuracy, and overall classification accuracy of the present invention compared to existing technologies EB-CNN and SSRN. As shown in Table 1, the overall accuracy of the present invention is improved by 1.42% and 4.33% compared to existing technologies EB-CNN and SSRN, respectively. The average accuracy is reduced by 0.28% compared to existing technologies EB-CNN and improved by 6.31% compared to existing technologies SSRN. In the comparison of classification accuracy for each land cover category, the present invention's method does not improve the classification accuracy of land cover categories 1, 4, 5, 9, and 11 compared to existing technologies EB-CNN. Categories 8, 13, and 15 achieve the best classification accuracy using the present invention's method, while the classification accuracy of other land cover categories is improved compared to existing technologies EB-CNN. Compared to existing technologies SSRN, the present invention's method only shows no improvement in classification accuracy for land cover category 11; the classification accuracy of other land cover categories is improved compared to existing technologies SSRN.

[0122] Simulation Experiment 2 uses the present invention and existing technologies EB-CNN and SSRN to simulate PaviaU data. PaviaU data is a portion of the hyperspectral data captured in Pavia, Italy in 2003 by a German airborne reflectance optical spectral imager. The original PaviaU data has 115 bands, a spectral range of 430-860 nm, and a spatial resolution of 1.3 m. In the experiment, the original image was preprocessed, removing 12 low signal-to-noise ratio bands from the original 115 bands, leaving 103 bands. Furthermore, some samples containing no information were removed from the original image, resulting in hyperspectral image classification data of size 610×340, the actual labeled map of which is shown in Figure 5(a). Figure 5(a) shows a total of 9 land cover types. The regions in the image, from darkest to lightest color, represent categories 1, 2, 3, 4, 5, 6, 7, 8, and 9, with a total of 42776 labeled samples. In simulation experiment 2, 200 and 100 labeled data points were extracted from the ground features in the image as training and validation sets, respectively. The entire labeled data set was used as the test set for simulation. The classification results of each sample in the validation set were obtained by the present invention and the existing technologies EB-CNN and SSRN. The classification results obtained by the present invention and the existing technologies EB-CNN and SSRN are plotted in Figures 5(b), 5(c), and 5(d), respectively.

[0123] As can be seen from the comparison between the classification results shown in Figures 5(b), 5(c), and 5(d) and the actual labeled map shown in Figure 5(a), the vast majority of samples in the classification results obtained by the method of the present invention have achieved the correct classification results, and the overall effect is closer to the actual labeled map shown in Figure 5(a). In contrast, in the classification results obtained by the existing technologies EB-CNN and SSRN, there are still a large number of misclassified samples for some types of land features.

[0124] To verify the effectiveness of simulation experiment 2 of this invention, the classification results of the method of this invention and the existing technologies EB-CNN and SSRN were quantified using three indicators: classification accuracy of each type of land cover, overall classification accuracy, and average classification accuracy.

[0125] Using the same calculation formulas for the three indicators as in Simulation Experiment 1, the various types of ground targets in the results of Simulation Experiment 2 were quantified, and the results are shown in Table 2.

[0126] Table 2. Quantitative Results of Simulation Experiment 2

[0127]

[0128] Table 2 lists the classification accuracy, average classification accuracy, and overall classification accuracy of each land cover category compared to the present invention and existing technologies EB-CNN and SSRN. As shown in Table 2, the overall accuracy of the present invention is improved by 6.13% and 8.34% compared to existing technologies EB-CNN and SSRN, respectively, while the average accuracy is improved by 11.92% and 12.98% compared to existing technologies EB-CNN and SSRN, respectively. In the comparison of classification accuracy for each land cover category between the present invention and existing technologies EB-CNN, the classification accuracy for all land cover categories is improved compared to existing technologies EB-CNN, except for category 7. In the comparison of classification accuracy for each land cover category between the present invention and existing technologies SSRN, the classification accuracy for all land cover categories is significantly improved compared to existing technologies SSRN.

[0129] Simulation Experiment 3 uses the present invention and existing technologies EB-CNN and SSRN to simulate the Houston2013 dataset. The Houston2013 dataset was acquired by the ITRES CASI-1500 sensor on the University of Houston campus and surrounding urban areas during the 2013 IEEE GRSS Data Fusion Competition. Houston2013 contains 144 bands with a spectral range of 380-1050nm, a spatial resolution of 2.5m, and an image size of 349×1905. Its actual annotation map is shown in Figure 6(a). In Figure 6(a), there are 15 land cover types. The colors of the regions in the image, from dark to light, represent categories 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, and 15. The total number of annotated samples is 15029. In simulation experiment 3, 200 and 100 labeled data points were extracted from the ground features in the image as training and validation sets, respectively. The entire labeled data set was used as the test set for simulation. The classification results of the present invention and the existing technologies EB-CNN and SSRN for each sample in the validation set were obtained. The classification results obtained by the present invention and the existing technologies EB-CNN and SSRN are plotted in Figures 6(b), 6(c), and 6(d), respectively.

[0130] As can be seen from the comparison between the classification results shown in Figures 6(b), 6(c), and 6(d) and the actual labeled map shown in Figure 6(a), in the classification results obtained by the method of the present invention, except for a small number of classification errors in some types of land cover samples, most samples are classified correctly, and the overall results are close to the actual labeled map shown in Figure 6(a). In contrast, in the classification results obtained by the prior art EB-CNN and SSRN, there are a large number of samples with obvious classification errors in some types of land cover, and the number of samples is relatively large and the distribution is wide.

[0131] To verify the effectiveness of simulation experiment 3 of this invention, the classification results of the method of this invention and the existing technologies EB-CNN and SSRN were quantified using three indicators: classification accuracy of each type of land cover, overall classification accuracy, and average classification accuracy.

[0132] Using the same calculation formulas for the three indicators as in Simulation Experiment 1, the various types of ground features in the results of Simulation Experiment 3 were quantified, and the results are shown in Table 3.

[0133] Table 3. Quantitative Results of Simulation Experiment 3

[0134]

[0135]

[0136] Table 3 lists the classification accuracy, average classification accuracy, and overall classification accuracy of each land cover category compared to the present invention and existing technologies EB-CNN and SSRN. As shown in Table 3, the overall accuracy of the present invention is improved by 2.79% and 12.72% compared to existing technologies EB-CNN and SSRN, respectively, while the average accuracy is improved by 3.87% and 11.80% compared to existing technologies EB-CNN and SSRN, respectively. In the comparison of classification accuracy for each land cover category between the present invention and existing technologies EB-CNN, the classification accuracy for categories 2, 4, 5, 10, and 12 did not improve. Categories 14 and 15 achieved the best classification accuracy using the present invention method, while the classification accuracy for other land cover categories improved compared to existing technologies EB-CNN. In the comparison of classification accuracy for each land cover category between the present invention and existing technologies SSRN, the classification accuracy for categories 2, 10, and 12 did not improve, while the classification accuracy for other land cover categories improved compared to existing technologies SSRN.

[0137] Simulation Experiment 4 is a simulation experiment that uses the network architecture search time cost and overall accuracy of the present invention and the existing technology EB-CNN on three types of data.

[0138] Table 4. Quantitative Results of Simulation Experiment 4

[0139]

[0140] Table 4 lists the overall accuracy and GPU days of the present invention and the prior art EB-CNN on three datasets. As shown in Table 4, the present invention's method improves the overall accuracy on IndianPines by 1.43% compared to the prior art, and reduces the GPU days by 2.6654 days. On PaviaU, the present invention's method improves the overall accuracy by 1.75% compared to the prior art, and reduces the GPU days by 2.3529 days. On Houston2013, the present invention's method improves the overall accuracy by 2.05% compared to the prior art, and reduces the GPU days by 1.2039 days. Therefore, the present invention outperforms the prior art EB-CNN in both search time cost and classification accuracy.

[0141] The simulation experiments above demonstrate that the attention module constructed by the method of this invention, which integrates the search space and multi-scale structure, can significantly improve the high classification accuracy of ground cover samples. The surrogate-assisted evolutionary search method proposed in this invention uses an MLP as the predictor in traditional NAS, reducing the burden on traditional NAS and significantly improving the efficiency of convolutional neural networks in hyperspectral image ground cover classification tasks while maintaining high classification accuracy. The surrogate data augmentation method proposed in this invention improves the performance and prediction accuracy of the surrogate model while ensuring search efficiency.

[0142] In summary, compared with the prior art, the beneficial effects of the present invention are as follows:

[0143] (1) Currently, most hyperspectral image classification models are designed manually, which places high demands on the designer's model design experience and hyperspectral knowledge. Furthermore, the errors introduced by manual design are detrimental to improving the classification accuracy of the model. To address this issue, this invention proposes a neural network structure search method that automatically designs neural network structures on hyperspectral images, thereby reducing the requirements and errors associated with manual design.

[0144] (2) In neural network architecture search, the performance of a large number of network models needs to be evaluated, which consumes a huge amount of computing resources. As the industry's requirements for hyperspectral image classification models become increasingly demanding, the search efficiency of models is receiving more and more attention. This invention designs a surrogate-assisted neural network architecture search method that, while ensuring high classification accuracy of the model, minimizes model evaluation and directly predicts model performance, thereby reducing costs and saving computing resources.

[0145] (3) Existing methods for surrogate-assisted neural architecture search suffer from insufficient surrogate model data and a lack of diversity, which directly leads to deviations in the search direction and affects the final model accuracy. This invention designs a surrogate data augmentation technique based on chromosome base types to solve the problem of insufficient training data for existing surrogate models, thereby improving the performance of surrogate models.

[0146] (4) Existing surrogate models suffer from training difficulties and inaccurate predictions, which directly leads to significant deviations in the accuracy of the predicted models. This invention designs a novel surrogate model training method that adapts well to the prediction environment of surrogate models that change with population evolution, thereby ensuring the accuracy of predictions and the correctness of the evolutionary direction.

[0147] (5) Existing neural network architecture search methods for hyperspectral image classification cannot search for the model's input resolution. The input resolution determines how much effective information the input image provides to the model. An unsuitable input resolution cannot provide the amount of information required by the model, resulting in low classification accuracy. This invention designs a neural network architecture search method based on evolutionary computation, which incorporates the input resolution, which is in discrete space, into the search space, and designs crossover and mutation operators to improve the algorithm's network search capability.

[0148] This invention also provides a hyperspectral image classification system based on surrogate-assisted evolutionary convolutional attention network architecture search, comprising:

[0149] The search space definition module is used to implement the search space defined in step S1. The search space includes three sub-network blocks. Each sub-network block includes feature scale, convolution operation type, number of convolution output channels, multi-scale aggregation mode encoding, attention mechanism, skip connections and pooling operations.

[0150] The convolutional neural network construction module is used to implement the process in step S2, which involves searching for the corresponding operations from the three sub-network blocks in the search space of step S1 according to the evolutionary search strategy, and then concatenating the three sub-network blocks to obtain the convolutional neural network.

[0151] The chromosome encoding construction module is used to encode the feature scale, convolution operation type, number of convolution output channels, multi-scale aggregation mode, attention mechanism, skip connections and pooling operation of the convolutional neural network constructed in step S2 into chromosomes through a scalar hybrid encoding method. The total encoding length is 41, and the chromosomes are divided into discrete basis and continuous basis.

[0152] The surrogate model-assisted evolutionary search module is used to randomly initialize the chromosomes encoded in step S3 to form a population in step S4, initialize the Spearman index c to 0 and the fitness range r to 1, merge the parent and offspring populations, and if the conditions r ≥ 0.05 and c ≥ 0.8 are met, the surrogate model is used to predict the fitness of the current population, and the next generation population is selected according to the hierarchical selection strategy; if the conditions are not met, the current population is truly trained and evaluated, and the next generation population is selected using the roulette wheel selection strategy.

[0153] The crossover population acquisition module is used to perform crossover operations on individuals in the population obtained in step S4 in step S5, and to form a crossover population from all the crossover offspring.

[0154] The mutant population acquisition module is used to perform mutation operations on individuals in the crossover population obtained in step S5 in step S6, and to form a mutant population from all the mutated offspring.

[0155] The offspring population acquisition module is used to combine all individuals in the crossover population obtained in step S5 with all individuals in the mutation population obtained in step S6 to form an offspring population in step S7.

[0156] The optimal convolutional neural network acquisition module is used to implement step S8. If the current iteration number is less than the maximum iteration number, the iteration number is incremented by 1 and step S4 is executed. Otherwise, the population of the current iteration is taken as the optimal population, and the optimal population is subjected to real training evaluation. The convolutional neural network corresponding to the individual with the highest classification accuracy in the optimal population is selected as the optimal convolutional neural network.

[0157] The result output module is used to input the hyperspectral image to be classified into the optimal convolutional neural network selected in step S8 in step S9 to obtain the final classification result, calculate the classification accuracy of the convolutional neural network for all samples to be classified in the hyperspectral image, and output the corresponding classification accuracy as the result.

[0158] This invention also provides a hyperspectral image classification device based on a surrogate-assisted evolutionary convolutional attention network architecture search, comprising:

[0159] Memory: A computer-readable device that stores the computer program of the above-described hyperspectral image classification method based on agent-assisted evolutionary convolutional attention network architecture search;

[0160] Processor: Used to implement the hyperspectral image classification method based on agent-assisted evolutionary convolutional attention network architecture search when executing the computer program.

[0161] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, enables the implementation of the hyperspectral image classification method based on a surrogate-assisted evolutionary convolutional attention network architecture search.

Claims

1. A hyperspectral image classification method based on surrogate-assisted evolutionary convolutional attention network architecture search, characterized in that: Includes the following steps: S1. Define the search space, which includes three sub-network blocks. Each sub-network block includes feature scale, convolution operation type, number of convolution output channels, multi-scale aggregation mode encoding, attention mechanism, skip connections, and pooling operation. S2. After searching for the corresponding operations from the three sub-network blocks in the search space of step S1 according to the evolutionary search strategy, the three sub-network blocks are concatenated to obtain a convolutional neural network. S3. The feature scale, convolution operation type, number of convolution output channels, multi-scale aggregation mode encoding, attention mechanism, skip connections and pooling operation of the convolutional neural network constructed in step S2 are encoded into chromosomes through a scalar hybrid encoding method, and the chromosomes are divided into discrete basis and continuous basis. S4. Proxy model-assisted evolutionary search: Randomly initialize the chromosomes encoded in step S3 to form the population, and initialize the Spearman index. 0 and fitness range If the value is 1, merge the parent and offspring populations if the condition is met. ≥0.05 and If the fitness is ≥ 0.8, a surrogate model is used to predict the fitness of the current population, and the next generation population is selected based on a hierarchy-based selection strategy. If the condition is not met, the current population is trained and evaluated, and the next generation population is selected based on a roulette wheel selection strategy. S5. Perform crossover operation on the individuals in the population obtained in step S4, and form a crossover population from all the crossover offspring. S6. Perform mutation operation on the individuals in the crossover population obtained in step S5, and form a mutation population from all the mutated offspring. S7. Combine all individuals in the crossover population obtained in step S5 with all individuals in the variant population obtained in step S6 to form the offspring population. S8. If the current iteration number is less than the maximum iteration number, increment the iteration number by 1 and then execute step S4. Otherwise, take the population of the current iteration as the optimal population, perform real training evaluation on the optimal population, and select the convolutional neural network corresponding to the individual with the highest classification accuracy from the optimal population as the optimal convolutional neural network. S9. Input the hyperspectral image to be classified into the best-trained convolutional neural network selected in step S8 to obtain the final classification result, and calculate the classification accuracy of the convolutional neural network for all samples to be classified in the hyperspectral image, and output the corresponding classification accuracy as the result.

2. The hyperspectral image classification method based on surrogate-assisted evolutionary convolutional attention network architecture search according to claim 1, characterized in that: The hybrid coding method in step S3 defines the structure of the chromosome as follows: in, The encoding representing the size of the input image, whose value is in A number randomly selected from the range, The encoding representing the number of spectral channels of the input image, whose value is in A number randomly selected from the range, The encoding represents different convolutional scales, and its value is in A number randomly selected from the range, The encoding representing different convolution operations, whose value is in A value randomly selected from the data. These represent two-dimensional ordinary convolution, two-dimensional separable convolution, three-dimensional ordinary convolution, and three-dimensional separable convolution, respectively. The encoding representing the number of output channels of a convolution operation, its value is in A number randomly selected from the range, This represents the aggregation strategy encoding of the sub-model, and its value is in A value randomly selected from the data. These represent the direct summation of multi-scale features, the weighted summation of multi-scale features, and the concatenation of them along the channel dimension, respectively. The encoding representing the attention mechanism, whose value is in A value randomly selected from the data. These represent spatial attention, spectral attention, and cross attention, respectively. This represents the encoding for skip connections, with a value of 0 or 1, indicating no skip connection and skip connection respectively. This represents the encoding of the pooling operation, and its value is in A value randomly selected from the data. They represent average pooling, max pooling, and... Small pooling.

3. The hyperspectral image classification method based on surrogate-assisted evolutionary convolutional attention network architecture search according to claim 1, characterized in that: Step S4 is as follows: S401, Set the maximum number of iterations to... Initialize the population size to The parent population is represented as offspring population Set to empty, that is ; S402. Initialize Spearman metrics 0 and fitness range Set to 1, and simultaneously set the parent population to 1. and offspring population Merging, i.e. If the conditions are met ≥ 0.05 and If the value is ≥ 0.8, this stage is called global search, and a surrogate model is used to predict the current population. The fitness level is used to obtain the level mapping. From the level mapping The highest and lowest level individuals are obtained and stored in the agent model's data archive. Ultimately, the next generation of the population is selected based on a hierarchy-based selection strategy. ; S403, if the conditions are not met ≥ 0.05 and If ≥ 0.8, then for the current population Perform actual training and evaluation to obtain the true classification accuracy of all chromosomes in the population. ,use Refresh proxy data archive ; S404, Data archive from the agent model Random sampling A dataset composed of chromosomes Divide the dataset according to a certain ratio Divided into training set and test set Then the training set and test set Convert to paired datasets All chromosomes are encoded into pairs and assembled. The labels of the encoded chromosomes are defined based on the classification accuracy of the corresponding networks of the preceding and following chromosomes. The formula is as follows: in, and They represent data archives respectively. mesochromosome and Classification accuracy, The label represents the training data after chromosome splicing. If the true evaluation accuracy of the first chromosome is higher than that of the second chromosome, the label is 1, indicating that a clear relationship has been established. S405. Using MLP as an alternative to logistic regression, the input data is converted into binary encoding to enhance feature information and sparsity, and data archiving is used. The training data is used to find the optimal MLP parameters. When the surrogate model is retrained and tuned later, the previously existing model parameters will be used directly. The surrogate model is trained by calculating the cross-entropy loss function. After obtaining the relationship between the two chromosomes, the ranking of the entire population is inferred. The loss function is as follows: in, Indicates the batch size. Indicates sample The true label, Indicates sample The probability of predicting a positive class. To calculate the loss; S406. Supplement the training set using a heterogeneous chromosome combination strategy. Chromosomal loci are divided into bases representing the model's backbone structure and bases representing the size of features. The bases of the model backbone determine the model's depth, width, and structure. The individuals with the best and worst classification accuracy in the population are identified, and a large amount of pseudo-label data is generated based on their genes. The third gene of the best individual is a modifiable gene, and a new code is obtained from data randomly selected from its neighborhood. The first gene of the worst individual is a modifiable gene, and a new code is obtained from data randomly selected from its neighborhood. S407. Archive the data through step 406. The data in the database is augmented, and the augmented data is then archived. The agent model was retrained, and finally a roulette wheel selection strategy was used to select the next generation of the population. .

4. The hyperspectral image classification method based on surrogate-assisted evolutionary convolutional attention network architecture search according to claim 1, characterized in that: In step S5, the crossover operation on the population is specifically performed as follows: S501. From the parent population of the current iteration, use the tournament algorithm to select two gene individuals corresponding to the convolutional neural networks to be optimized. S502, Based on the crossover probability Perform a crossover operation on the two selected gene individuals, and generate a set of genes that satisfy the following conditions for each selected gene pair: A uniformly distributed random number, if the random number is less than , Then, a two-point crossover operation is performed on the two selected gene individuals; S503. Determine whether all gene individuals have been selected in the parent population after the current iteration. If so, form a crossover population by combining all gene individuals obtained through the above crossover operation in the parent population after the current iteration, and then execute step S6. Otherwise, execute step S501.

5. The hyperspectral image classification method based on surrogate-assisted evolutionary convolutional attention network architecture search according to claim 1, characterized in that: In step S6, the specific steps for performing the mutation operation on the population are as follows: S601, when searching mode For global search, mutation operations are performed on the gene individuals in the crossover population obtained in step S503 according to probability; a set of genes is generated for each selected pair of gene individuals that satisfies... A uniformly distributed random number, if the random number is less than , Then, a single-point mutation operation is performed on the two selected gene individuals; S602. Combine all the gene individuals obtained from the above mutation operation in the crossover population into a variant population.

6. A hyperspectral image classification system based on a surrogate-assisted evolutionary convolutional attention network architecture for search, as described in any one of claims 1 to 5, characterized in that: include: The search space definition module is used to define the search space, which includes three sub-network blocks. Each sub-network block includes feature scale, convolution operation type, number of convolution output channels, multi-scale aggregation mode encoding, attention mechanism, skip connections, and pooling operation. The convolutional neural network building module is used to search for the corresponding operations from the three sub-network blocks in the search space according to the evolutionary search strategy, and then concatenate the three sub-network blocks to obtain the convolutional neural network. The chromosome encoding construction module is used to encode the feature scale, convolution operation type, number of convolution output channels, multi-scale aggregation mode, attention mechanism, skip connections and pooling operations of the constructed convolutional neural network into chromosomes through a scalar hybrid encoding method, and divide the chromosomes into discrete basis and continuous basis; The surrogate model-assisted evolutionary search module is used to randomly initialize the encoded chromosome population and initialize the Spearman index. 0 and fitness range If the value is 1, merge the parent and offspring populations if the condition is met. ≥0.05 and If the fitness is ≥ 0.8, a surrogate model is used to predict the fitness of the current population, and the next generation population is selected based on a hierarchy-based selection strategy. If the condition is not met, the current population is trained and evaluated, and the next generation population is selected based on a roulette wheel selection strategy. The crossover population acquisition module is used to perform crossover operations on individuals in the population and form a crossover population from all the crossover offspring. The mutant population acquisition module is used to perform mutation operations on individuals in the crossover population and combine all the mutated offspring into a mutant population. The offspring population acquisition module is used to combine all individuals in the crossover population with all individuals in the mutation population to form an offspring population. The optimal convolutional neural network acquisition module is used to implement the following: if the current iteration number is less than the maximum iteration number, the iteration number is incremented by 1 and a surrogate model is executed to assist in evolutionary search; otherwise, the population in the current iteration is taken as the optimal population, and the optimal population is trained and evaluated in real time. The convolutional neural network corresponding to the individual with the highest classification accuracy in the optimal population is selected as the optimal convolutional neural network. The result output module is used to input the hyperspectral image to be classified into the trained optimal convolutional neural network to obtain the final classification result, calculate the classification accuracy of the convolutional neural network for all samples to be classified in the hyperspectral image, and output the corresponding classification accuracy as the result.

7. A hyperspectral image classification device based on a surrogate-assisted evolutionary convolutional attention network architecture for search, characterized in that: include: Memory: A computer program for a hyperspectral image classification method based on agent-assisted evolutionary convolutional attention network architecture search as described in any one of claims 1-5, and is a computer-readable device; Processor: Used to implement the hyperspectral image classification method based on agent-assisted evolutionary convolutional attention network architecture search as described in any one of claims 1-5 when executing the computer program.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program that, when executed by a processor, enables the implementation of the hyperspectral image classification method based on surrogate-assisted evolutionary convolutional attention network architecture search as described in any one of claims 1-5.

Images