Evolutionary deep learning method for remote sensing scene classification based on random forest

Abstract

The application discloses a remote sensing scene classification evolution deep learning method based on a random forest, which comprises the following steps: initializing a population by adopting a two-stage variable-length particle swarm strategy; updating the population by adopting a particle swarm algorithm; predicting the classification accuracy of individuals by adopting a random forest; adopting an early stopping strategy during training to make the training evaluation more efficient; enriching the population and updating the random forest model during evolution; and selecting the best individual to perform accurate evaluation on a test set after evolution is completed. The application uses a two-stage particle swarm strategy to represent the connection and parameters of each architecture, adopts a particle architecture variable-length strategy to make the population more diverse, adopts a random forest as an agent model to make the network have high performance, robustness and convenience, improves the accuracy in remote sensing scene classification relative to a manually designed network without any pre-training and data enhancement, and can find the optimal network faster relative to an automatically optimized network.

Description

Technical Field

[0001] This invention relates to an evolutionary deep learning method for remote sensing scene classification, specifically an evolutionary deep learning method for remote sensing scene classification based on random forest. Background Technology

[0002] Scene classification is an important application in the field of remote sensing imagery. Its purpose is to assign specific semantic labels (such as airplanes, rivers, highways, farms, etc.) to each unlabeled aerial image based on its semantic content. With the rapid development of ground and space observation technologies, accurate classification of remote sensing images has become a crucial research topic in the field of ground observation, attracting increasing attention. In recent years, the availability of increasingly high-resolution remote sensing images has made scene classification, change detection, land use classification, and automatic target identification even more important.

[0003] However, remote sensing scene images typically contain various types of land cover or features, differing in shape, scale, orientation, and spatial distribution. Scene images with the same semantic label may exhibit high differences in geometric structure and spatial patterns, or show little spatial similarity. For example, urban areas may consist of many geographical structures, such as residential areas, roads, and commercial areas, which belong to different semantic classes. Furthermore, the imbalance in the distribution of various land cover types is also prevalent in high-resolution remote sensing images. Small sample sizes often result in low classification accuracy and poor classification performance, posing a challenge to existing classification techniques. Therefore, effectively and accurately extracting useful features from high-resolution remote sensing images for identification has become an important task for researchers in remote sensing scene classification.

[0004] Generally, remote sensing scene classification methods can be categorized into low-level, mid-level, and high-level methods. Low-level methods use handcrafted features such as spectral density, texture, and structure to distinguish different scene categories. The performance of mid-level methods heavily relies on the extraction of handcrafted local features, such as local structural features, color histograms, and texture features. High-level methods include deep learning algorithms that can learn hierarchical features at different levels of abstraction. Deep convolutional neural networks are considered the most successful and widely used deep learning algorithms, consisting of convolutional layers, pooling layers, and fully connected layers, and can be effectively trained in a fully supervised manner.

[0005] In recent years, a series of hand-crafted deep convolutional networks, such as ResNet, have achieved good results in remote sensing images. However, these methods require extensive manual hyperparameter tuning and strong domain knowledge, such as setting hyperparameters including the total number of layers in the neural network, the number of convolutional and pooling layers, connection methods, and the size of the convolutional kernels. To address this, researchers have attempted to design automated convolutional neural networks, giving rise to a new research area called neural network architecture search. Specifically, neural network architecture search can reduce the time spent on manually tuning network models, automatically optimize some hyperparameters in the neural network, and try more novel network architectures to find the optimal architecture with better generalization ability.

[0006] In the process of realizing this invention, the inventors discovered the following technical problems in the prior art:

[0007] In the process of searching for neural network architectures, the evaluation of each found architecture typically takes several hours to several days. Therefore, with limited computing resources, the expensive neural network model evaluation may prevent the discovery of potentially promising neural network model architectures. In fact, neural network evaluation itself is a computationally expensive problem. This can be solved using surrogate-assisted algorithms. Inexpensive approximate regression and classification models, or radial basis function networks, can be used to replace expensive neural network model evaluations. This involves collecting a small amount of real performance evaluation data of expensive neural network models, training a surrogate model, and then using the trained surrogate model to predict the performance of newly generated neural network models during the search process, thereby accelerating the optimization process. Summary of the Invention

[0008] To address the aforementioned issues, this invention provides a novel evolutionary deep learning method for remote sensing scene classification based on random forests. Random forests are used as a performance predictor for architecture evaluation, and by combining multiple decision trees, highly accurate prediction results are generated. Without any pre-training or data augmentation methods, a convolutional neural network model for scene classification can be found more quickly through automatic architecture search.

[0009] The technical solution adopted in this invention is as follows:

[0010] An evolutionary deep learning method for remote sensing scene classification based on random forest includes the following steps:

[0011] S1, Initialize the population;

[0012] S2, evaluate the fitness of the population and record the historical best individual and the global best individual for each population;

[0013] S3 uses the particle swarm optimization algorithm to evolve the population;

[0014] S4: Determine if the algebra is greater than 5. If it is less than 5, go to S1; if it is greater than 5, go to S5.

[0015] S5, update the random forest model;

[0016] S6 uses the particle swarm optimization algorithm to update the population;

[0017] S7, using random forest to predict the classification accuracy of individual algorithms;

[0018] S8, after the evolution is complete, select the best individuals to test on the test set.

[0019] Furthermore, the specific steps of S2 are as follows:

[0020] S21: Input the image that needs to be classified;

[0021] S22: Divide the images into an evolutionary training set, an evolutionary test set, and a test set. The test set images are used as the data for training the final optimal architecture.

[0022] S23: Use an early stopping strategy to terminate the training of individual algorithms on remote sensing scene image data;

[0023] S24: Record the obtained neural network architecture parameters, classification accuracy, the best individual in the population history, and the best individual globally.

[0024] Furthermore, the specific steps of S23 are as follows:

[0025] S231: The neural network obtained after decoding the individual architecture is trained on the evolutionary training set;

[0026] S232: The classification accuracy of the neural network obtained after decoding the computational architecture on the evolutionary test set;

[0027] S233: If the number of iterations on the evolutionary test set satisfies the formula:

[0028] epoch > epoch best +λ

[0029] Or the classification accuracy satisfies the formula:

[0030] acc < acc best -ξ

[0031] Then the neural network training is terminated; in the formula, λ and ξ are thresholds.

[0032] Furthermore, the specific steps of S5 are as follows:

[0033] S51: First, load the encoding of the first 5 generations of individuals as training data features for the random forest, using the individual classification accuracy as the label, and establish the random forest training dataset; as the generations progress, continuously update the random forest model training database.

[0034] S52: Train decision trees to form a random forest;

[0035] S53: Save the trained random forest regression model.

[0036] Furthermore, the specific steps of S52 are as follows:

[0037] S521: Prepare individual coding data and classification labels;

[0038] S522: Select appropriate features for training the model;

[0039] S523: Construct multiple decision trees, and train each decision tree;

[0040] S524: Integrate multiple decision trees into a single random forest;

[0041] S525: Use the average of multiple decision trees as the final prediction result.

[0042] Furthermore, the specific steps of S6 are as follows:

[0043] S61: Initialize the position and velocity of the particles;

[0044] S61: Initialize the position and velocity of the particles;

[0045] S62: Update speed and position based on historical best and global best assessments using S2;

[0046] S63: Update the particles according to the following velocity and position update formulas, where the velocity update formula is:

[0047] V i (t+1) =ω·V i (t) +c1·r1·(Pbest i -X i (t) )+c2·r2·(Gbest-X i (t) )

[0048] The position update formula is:

[0049] X i (t+1) =X i (t) +V i(t+1)

[0050] In the formula, V i (t+1) V is the velocity of particle i at the next moment. i t ω is the velocity of particle i at the current moment; ω is the inertia weight, usually a positive number less than 1, used to balance global and local searches; c1 and c2 are acceleration coefficients, usually positive numbers, used to control the interaction between individuals and the group; r1 and r2 are randomly generated values ​​in the range [0,1], introducing randomness; Pbest i It is the best historical position of particle i itself; X i (t) X is the position of particle i at the current moment. i (t+1) Gbest is the position of particle i at the next moment; Gbest is the best position in the history of the entire particle swarm.

[0051] Furthermore, the specific steps of S7 are as follows:

[0052] S71: Read the individual coded data updated by S6;

[0053] S72: Input the individual coded data into the random forest prediction model obtained in S5;

[0054] S73: Output the classification accuracy predicted by the random forest model;

[0055] S74: Add the offspring population and corresponding classification accuracy generated by S73 to the random forest dataset to update the random forest model.

[0056] Furthermore, the specific steps of S8 are as follows:

[0057] S81: Input the test set partitioned by S22;

[0058] S82: Read the parameters required for the test and the globally optimal individual;

[0059] S83: Train the globally optimal individual on the test set;

[0060] S84: Obtain the final classification accuracy and computation time.

[0061] The beneficial effects of this invention are:

[0062] 1. The introduction of random forests makes the evaluation process faster and saves evaluation time;

[0063] 2. It has the advantages of simple implementation, fast speed, high classification accuracy and few adjustable parameters;

[0064] 3. Combining surrogate models and neural network architecture search can quickly find architectures with high accuracy;

[0065] 4. In remote sensing scene classification, the architecture is automatically designed, and the model parameters are relatively small. Attached Figure Description

[0066] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0067] The invention will now be further described with reference to the accompanying drawings.

[0068] like Figure 1 As shown, this invention is an evolutionary deep learning method for remote sensing scene classification based on random forest, comprising the following steps:

[0069] S1, Initialize the population;

[0070] S2, assess the fitness of the population and record the historical best individual and the global best individual for each population. The specific steps are as follows:

[0071] S21: Input the image that needs to be classified;

[0072] S22: Divide the images into an evolutionary training set, an evolutionary test set, and a test set. The test set images are used as the data for training the final optimal architecture.

[0073] S23: Employ an early stopping strategy to terminate the training of individual algorithms on remote sensing scene image data. The specific steps are as follows:

[0074] S231: The neural network obtained after decoding the individual architecture is trained on the evolutionary training set;

[0075] S232: The classification accuracy of the neural network obtained after decoding the computational architecture on the evolutionary test set;

[0076] S233: If the number of iterations on the evolutionary test set satisfies the formula:

[0077] epoch > epoch best +λ

[0078] Or the classification accuracy satisfies the formula:

[0079] acc < acc best -ξ

[0080] Then the neural network training is terminated; in the formula, λ and ξ are thresholds.

[0081] S24: Record the obtained neural network architecture parameters, classification accuracy, the best individual in the population history, and the best individual globally.

[0082] S3 uses the particle swarm optimization algorithm to evolve the population;

[0083] S4: Determine if the algebra is greater than 5. If it is less than 5, go to S1; if it is greater than 5, go to S5.

[0084] S5, update the random forest model, the specific steps are as follows:

[0085] S51: First, load the encoding of the first 5 generations of individuals as training data features for the random forest, using individual classification accuracy as the label, to establish the random forest training dataset. As the generations progress, continuously update the random forest model training database;

[0086] S52: Train decision trees to form a random forest.

[0087] In S52, surrogate models are a method in machine learning and data science used to build a less computationally expensive model to replace the original, costly evaluation model, thereby reducing computational costs, increasing training speed, or providing better performance in specific application scenarios. The general steps for using a surrogate model are as follows: First, select an original model with higher computational cost or complexity, such as a deep neural network or a complex ensemble model. Second, prepare training data, including features and target variables. Typically, the surrogate model will use the same dataset as the original model. Then, select a less computationally expensive model as the surrogate model, such as a linear regression model. Next, train the surrogate model using the same training data. Finally, evaluate the surrogate model's performance on the test set using the same evaluation metrics as the original model. The performance of the surrogate model should be within an acceptable range while maintaining lower computational cost.

[0088] Common surrogate models include linear regression, logistic regression, decision trees, support vector machines, K-nearest neighbors, and random forests. These surrogate models can be selected based on specific problems and needs to provide better performance in terms of reducing computational costs, improving efficiency, or protecting privacy. Here, we choose random forests primarily because they typically possess excellent predictive capabilities, producing highly accurate predictions across a variety of problems. It is a powerful ensemble learning method that reduces the risk of overfitting and improves generalization ability by combining multiple decision trees. In terms of robustness, random forests are robust to noise and outliers in the data because they can reduce the impact of individual tree errors through voting among multiple trees. Furthermore, random forests are easy to implement and tune, requiring minimal hyperparameter tuning, typically providing good default performance. Moreover, each tree can be built independently, making parallel processing easy and accelerating the training process. In conclusion, due to its combination of high performance, robustness, and convenience, the random forest model has wide applications in various fields and problems.

[0089] The specific steps of S52 implemented using random forest in this invention are as follows:

[0090] S521: Prepare individual coding data and classification labels;

[0091] S522: Select appropriate features for training the model;

[0092] S523: Construct multiple decision trees, and train each decision tree;

[0093] S524: Integrate multiple decision trees into a single random forest;

[0094] S525: Use the average of multiple decision trees as the final prediction result.

[0095] S53: Save the trained random forest regression model.

[0096] S6, the population is updated using the particle swarm optimization algorithm. The specific steps are as follows:

[0097] S61: Initialize the position and velocity of the particles;

[0098] S61: Initialize the position and velocity of the particles;

[0099] S62: Update speed and position based on historical best and global best assessments using S2;

[0100] S63: Update the particles according to the following velocity and position update formulas. The velocity update formula is:

[0101] V i (t+1) =ω·Vi (t) +c1·r1·(Pbest i -X i (t) )+c2·r2·(Gbest-X i (t) )

[0102] The position update formula is:

[0103] X i (t+1) =X i (t) +V i (t+1)

[0104] In the formula, V i (t+1) V is the velocity of particle i at the next moment. i t ω is the velocity of particle i at the current moment; ω is the inertia weight, usually a positive number less than 1, used to balance global and local searches; c1 and c2 are acceleration coefficients, usually positive numbers, used to control the interaction between individuals and the group; r1 and r2 are randomly generated values ​​in the range [0,1], introducing randomness; Pbest i It is the best historical position of particle i itself; X i (t) X is the position of particle i at the current moment. i (t+1) Gbest is the position of particle i at the next moment; Gbest is the best position in the history of the entire particle swarm.

[0105] S7, using random forest to predict classification accuracy for individual algorithms, the specific steps are as follows:

[0106] S71: Read the individual coded data updated by S6;

[0107] S72: Input the individual coded data into the random forest prediction model obtained in S5;

[0108] S73: Output the classification accuracy predicted by the random forest model;

[0109] S74: Add the offspring population and corresponding classification accuracy generated by S73 to the random forest dataset to update the random forest model.

[0110] S8, after evolution is complete, the best individuals are selected for testing on the test set. The specific steps are as follows:

[0111] S81: Input the test set partitioned by S22;

[0112] S82: Read the parameters required for the test and the globally optimal individual;

[0113] S83: Train the globally optimal individual on the test set;

[0114] S84: Obtain the final classification accuracy and computation time.

[0115] To further illustrate the superiority of this invention in remote sensing scene classification, Table 1 shows the classification accuracy and running time (GPU days) of the method of this invention compared with representative artificially designed neural networks ResNet34, GoogleNet, and the automatic search neural network method EPCNAS on remote sensing scene classification data of UCM and AID.

[0116] Table 1 Comparison of experimental results on UCM and AID datasets.

[0117]

[0118] Table 1 presents the experimental results on remote sensing scene classification datasets. The UCM dataset has 21 classes, with 100 images per class, each 256×256 pixels. The AID dataset has 30 classes, totaling 10,000 images, each 600×600 pixels. The experiments were conducted on a GPU equipped with an RTX 6000 processor. The population size was set to 20, and the number of generations to 10. Without any pre-training, the manually designed network was run independently 10 times, and the proposed method and the EPCNAS method were run independently 5 times. Table 1 shows the maximum accuracy and runtime (GPU days). The comparison shows that the proposed evolutionary deep learning method for remote sensing scene classification based on random forest exhibits superior classification performance. Specifically, compared to manually designed networks, it improves the accuracy of remote sensing scene classification; compared to automatically optimized networks, it can find the optimal neural network faster. These results also demonstrate that the proposed method can find a good neural network architecture in remote sensing scene classification in less time, proving its effectiveness in classification tasks.

[0119] In summary, the novel evolutionary deep learning method for remote sensing scene classification based on random forest proposed in this invention can effectively process remote sensing scene classification datasets, quickly find a good neural network architecture, and obtain high classification performance.

[0120] Neural network architecture search can reduce the time spent on manual network tuning by automatically optimizing neural network hyperparameters and architecture to find the optimal architecture. Evolutionary algorithms have been widely proven to possess strong global search capabilities, effectively finding globally optimal or near-optimal solutions in complex spaces. Combining the two allows for a faster and more accurate discovery of the optimal neural network architecture.

[0121] This invention introduces a random forest as an alternative to the performance evaluator, which effectively reduces runtime and computational cost. The random forest model exhibits outstanding performance, high robustness, and ease of application. Most importantly, the random forest model is easy to use and tune, enabling rapid model building and achieving excellent performance without extensive hyperparameter tuning. In summary, in the field of remote sensing scene classification, this invention provides a novel evolutionary deep learning method based on random forests. This method can automatically and quickly find the optimal neural network architecture and offers significant performance advantages over other networks.

[0122] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A remote sensing scene classification evolutionary deep learning method based on random forest, characterized in that, Includes the following steps: S1, Initialize the population; S2, assess the fitness of the population and record the historical best individual and the global best individual for each population; the specific steps are as follows: S21: Input the image that needs to be classified; S22: Divide the images into an evolutionary training set, an evolutionary test set, and a test set. The test set images are used as the data for training the final optimal architecture. S23: Employ an early stopping strategy to terminate the training of individual algorithms on remote sensing scene image data; the specific steps are as follows: S231: The neural network obtained after decoding the individual architecture is trained on the evolutionary training set; S232: The classification accuracy of the neural network obtained after decoding the computational architecture on the evolutionary test set; S233: If the number of iterations on the evolutionary test set satisfies the formula: Or the classification accuracy satisfies the formula: Then the neural network training is terminated; where, For the threshold; S24: Record the obtained neural network architecture parameters, classification accuracy, historical best individual and global best individual; S3 uses the particle swarm optimization algorithm to evolve the population; S4: Determine if the algebra is greater than 5. If it is less than 5, go to S1; if it is greater than 5, go to S5. S5, Update the Random Forest model; the specific steps are as follows: S51: First, load the encoding of the first 5 generations of individuals as training data features for the random forest, using the individual classification accuracy as the label, and establish the random forest training dataset; as the generations progress, continuously update the random forest model training database. S52: Train decision trees to form a random forest; the specific steps are as follows: S521: Prepare individual coding data and classification labels; S522: Select appropriate features for training the model; S523: Construct multiple decision trees, and train each decision tree; S524: Integrate multiple decision trees into a single random forest; S525: Use the average of multiple decision trees as the final prediction result; S53: Save the trained random forest regression model; S6 uses the particle swarm optimization algorithm to update the population; S7, use random forest to predict the classification accuracy of individual algorithms; the specific steps are as follows: S71: Read the individual coded data updated by S6; S72: Input the individual coded data into the random forest prediction model obtained in S5; S73: Output the classification accuracy predicted by the random forest model; S74: Add the offspring population and corresponding classification accuracy generated by S73 to the random forest dataset to update the random forest model; S8, after the evolution is complete, select the best individuals to test on the test set.

2. The remote sensing scene classification evolutionary deep learning method based on random forest according to claim 1, characterized in that, The specific steps of S6 are as follows: S61: Initialize the position and velocity of the particles; S62: Update speed and position based on historical best and global best assessments using S2; S63: Update the particles according to the following velocity and position update formulas. The velocity update formula is: The position update formula is: In the formula, It is the velocity of particle i at the next moment. It is the velocity of particle i at the current moment; c1 and c2 are inertia weights, usually positive numbers less than 1, used to balance global and local searches; c1 and c2 are acceleration coefficients, usually positive numbers, used to control the interaction between individuals and the group; r1 and r2 are randomly generated values ​​in the range [0,1], introducing randomness; Pbest i It is the best historical position of particle i itself; X i (t) It is the position of particle i at the current moment. Gbest is the position of particle i at the next moment; Gbest is the best position in the history of the entire particle swarm.

3. The remote sensing scene classification evolutionary deep learning method based on random forest according to claim 1, characterized in that, The specific steps of S8 are as follows: S81: Input the test set partitioned by S22; S82: Read the parameters required for the test and the globally optimal individual; S83: Train the globally optimal individual on the test set; S84: Obtain the final classification accuracy and computation time.

Images