Image classification method based on residual neural network

Abstract

The invention provides an image classification method based on a residual neural network, and the method comprises the following steps: collecting high-resolution satellite remote sensing image data, and guaranteeing the coverage of a plurality of ground feature types; preprocessing the image, namely adjusting the image to be smaller in size, and performing pixel value normalization and data enhancement; dividing the processed data set into a training set, a verification set and a test set; a model architecture based on a residual neural network is adopted, a plurality of residual blocks are used, and input is directly added to output through jump connection so as to relieve the gradient disappearance problem; a converter module is connected behind the residual neural network, and a self-attention mechanism is utilized to process the output of a convolutional layer so as to capture global features; a cross entropy loss function and an Adam optimizer are used for model compiling, and an early stop strategy is adopted in the training process to prevent overfitting; and performing performance evaluation on the trained model by using the test set, calculating the accuracy rate, the recall rate and the F1-score, and performing result visualization to optimize the classification effect.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an image classification method based on a residual neural network. Background Art

[0002] With the rapid development of remote sensing technology, satellite remote sensing images, with their advantages of wide coverage, short data acquisition cycle, and rich information, are playing an increasingly important role in resource and environmental monitoring, disaster prevention and mitigation, and dynamic monitoring of land use. Satellite remote sensing images contain geometric and radiometric information of objects, and classifying and identifying objects is the key to fully mining and utilizing satellite image information.

[0003] Traditional satellite image classification methods are mainly based on manual visual interpretation, which is time-consuming and labor-intensive and cannot meet the needs of large-scale data processing. In recent years, machine learning technology has been widely used in the field of remote sensing image classification. Early methods mostly used traditional machine learning models such as support vector machines (SVMs), which artificially designed features such as texture, shape, and spectrum, and then coupled classifiers for classification. This type of method has achieved certain results, but the expressive power of artificial features is limited, the design process relies on expert experience, and the extracted features are often low-level, making it difficult to describe the semantic information of objects in complex scenes.

[0004] With the rapid development of remote sensing technology in recent years, the rise of deep learning technology has brought revolutionary changes to satellite remote sensing image classification. Satellite remote sensing images are playing an increasingly important role in resource and environmental monitoring and land use. Traditional convolutional neural networks (CNNs) still have limitations when processing complex remote sensing data. Combining the powerful feature extraction capabilities of the Transformers model can provide a new solution for satellite remote sensing image classification, thereby improving classification performance and model robustness. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention provides an image classification method based on a residual neural network, and adopts a ResNet-50 model to classify satellite images. The residual learning mechanism of ResNet is the key. By introducing residual blocks and jump connections, the network can be very deep without the problem of gradient vanishing, thereby extracting higher-level and more abstract features, significantly improving the accuracy of satellite image classification, achieving high-precision automatic classification, and saving a lot of manual labeling costs.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] An image classification method based on residual neural network includes the following steps:

[0008] Collect high-resolution satellite remote sensing image data to ensure coverage of a variety of landform types;

[0009] Use professional annotation tools to annotate images at the pixel level to ensure that each type of feature is accurately identified;

[0010] Preprocess the images, including resizing them to a smaller size, normalizing pixel values, and enhancing data;

[0011] Divide the processed data set into training set, validation set and test set;

[0012] Adopt a model architecture based on residual neural network, use multiple residual blocks, and add input directly to output through skip connection to alleviate the gradient vanishing problem;

[0013] A transformer module is connected after the residual neural network to process the convolutional layer output using the self-attention mechanism to capture global features.

[0014] Use the cross entropy loss function and Adam optimizer to compile the model, and adopt an early stopping strategy during training to prevent overfitting;

[0015] Use the test set to evaluate the performance of the trained model, calculate the accuracy, recall, and F1-score, and visualize the results to optimize the classification effect.

[0016] Preferably, the image preprocessing step includes data enhancement technology, and the data enhancement technology includes random rotation, flipping, translation and adding Gaussian noise.

[0017] Preferably, in the data preprocessing step, pixel value normalization is performed using the formula:

[0018]

[0019] Among them, x′ is the normalized pixel value, x is the original pixel value, μ is the mean of the pixel values, and σ is the standard deviation of the pixel values.

[0020] Preferably, the residual neural network part includes a plurality of residual blocks, each residual block directly adding the input to the output via a skip connection.

[0021] Preferably, the converter module calculation formula is:

[0022]

[0023] Among them, Q is the query matrix, K is the key matrix, V is the value matrix, and d k is the dimension of the key vector.

[0024] Preferably, the formula of the loss function is:

[0025]

[0026] Where: L is the loss value; N is the number of samples; y i is the true category label; p i The class probabilities predicted by the model.

[0027] Preferably, the multi-spectral bands are combined to highlight the spectral characteristics of different objects, and the combination method includes calculating the vegetation index NDVI, the formula is:

[0028]

[0029] Among them, NIR is the pixel value of the near-infrared band, and Red is the pixel value of the red band.

[0030] Preferably, an early stopping strategy is set during the model training process, and training is stopped when the validation set loss no longer decreases.

[0031] Preferably, when the test set is used to evaluate the performance of the model, the calculated indicators include accuracy, recall and F1-score, and the confusion matrix is ​​displayed by visualization technology to help analyze the classification effect, wherein the calculation formula of F1-score is:

[0032]

[0033] Among them, Precision is the accuracy rate, which means the proportion of positive class predictions that are actually positive;

[0034] Recall is the recall rate, which indicates the proportion of actually positive classes that are correctly predicted to be positive.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] 1. The present invention adopts the ResNet-50 model to classify satellite images. The residual learning mechanism of ResNet is the key. By introducing residual blocks and jump connections, the network can be very deep without the problem of gradient vanishing, thereby extracting higher-level and more abstract features, significantly improving the accuracy of satellite image classification, achieving high-precision automatic classification, and saving a lot of manual labeling costs.

[0037] 2. Several steps in the data preprocessing of the present invention: resizing of large-size images, normalization of pixel values, data enhancement, and multispectral band combination, etc. These preprocessing can reduce the dependence on large-scale annotated data, and can also achieve good classification results on small and medium-scale data sets, thus improving the applicability of the method.

[0038] 3. The entire classification process of the present invention is highly automated, and the model structure and hyperparameters can be flexibly adjusted to continuously optimize the model performance, making the classification process more intelligent and efficient.

[0039] 4. The present invention can automatically, accurately and efficiently classify satellite images, and can provide important information support for many fields such as land use surveys, resource and environmental monitoring, and disaster assessment, and has broad application prospects. DETAILED DESCRIPTION

[0040] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art can think of other obvious variations. The basic principles of the present invention defined in the following description can be used for other embodiments, variations, improvements, equivalents, and other technical solutions that do not deviate from the spirit and scope of the present invention.

[0041] The present invention provides an image classification method based on residual neural network, comprising the following steps:

[0042] Step 1: Collect high-resolution satellite remote sensing image data to ensure coverage of various types of land features;

[0043] Step 2: Preprocess the image, including resizing the image to a smaller size, normalizing pixel values, and performing data enhancement;

[0044] Step 3: Divide the processed data set into training set, validation set and test set;

[0045] Step 4: Adopt a model architecture based on residual neural network, use multiple residual blocks, and add the input directly to the output through skip connections to alleviate the gradient vanishing problem;

[0046] Step 5: Connect the transformer module after the residual neural network and use the self-attention mechanism to process the convolutional layer output to capture global features;

[0047] Step 6: Use the cross entropy loss function and Adam optimizer to compile the model, and adopt the early stopping strategy during training to prevent overfitting;

[0048] Step 7: Use the test set to evaluate the performance of the trained model, calculate the accuracy, recall, and F1-score, and visualize the results to optimize the classification effect.

[0049] The first step of this application is data collection, which involves collecting high-resolution satellite remote sensing image data to ensure coverage of a variety of landform types (such as farmland, cities, forests, etc.). The data can be obtained from remote sensing satellites or related data provision platforms.

[0050] Step 2 of this application is to preprocess the collected image data, and the preprocessing steps include:

[0051] Adjust the image size and resize the high-resolution image to a smaller size such as 256x256 to reduce the computational cost;

[0052] Normalize pixel values: Normalize pixel values ​​to fall within the range of 0-1, which is beneficial for model training.

[0053] Data augmentation, applying data augmentation techniques to expand the training set, expand the training set;

[0054] Multispectral band combination, combining multispectral bands to highlight certain features.

[0055] It should be noted that the normalization process is to normalize each pixel value, using the formula:

[0056]

[0057] Among them, x′ is the normalized pixel value, x is the original pixel value, μ is the mean of the pixel values, and σ is the standard deviation of the pixel values.

[0058] It should be noted that data enhancement is the application of data enhancement technology to expand the training set. Common methods include:

[0059] Random rotation (such as 0°, 90°, 180°, 270°);

[0060] Horizontal and vertical flip;

[0061] Randomly translate horizontally and vertically by a certain number of pixels;

[0062] Add Gaussian noise to the image.

[0063] It should be noted that multispectral band combination: combining red, green, blue and near-infrared bands to form a composite image, highlighting the spectral characteristics of different objects. The vegetation index can be calculated using NDVI (Normalized Difference Vegetation Index), the formula is:

[0064]

[0065] in:

[0066] NIR is the pixel value of the near infrared band;

[0067] Red is the pixel value of the red band.

[0068] Step three of this application is data set division, which is to divide the processed data set into training set, validation set and test set, generally in a ratio of 70:15:15, to ensure that the proportion of each category in different data sets is consistent so that the model can learn comprehensive features.

[0069] Step 4 of this application is to use the residual neural network model architecture to alleviate the gradient vanishing problem. Prior to this, this application first builds and trains the model.

[0070] 1. Choose the model architecture:

[0071] A hybrid model architecture combining CNN and transformation learning is adopted, which is divided into two main parts:

[0072] Convolutional layer: ResNet-50 is used as the feature extractor;

[0073] Transformer part: A self-attention mechanism is connected after the output of the convolutional layer to capture global features.

[0074] 2. Residual blocks and skip connections:

[0075] Multiple residual blocks are used in the convolution part, and skip connections are introduced in the design to alleviate the gradient vanishing problem and ensure network depth and training effect.

[0076] 3. Converter module:

[0077] The transformer module is introduced after the output of the convolutional layer, and the self-attention mechanism is used to process the features. The calculation formula for each transformer layer is:

[0078]

[0079] Where: Q is the query matrix, K is the key matrix, V is the value matrix, d k is the dimension of the key vector.

[0080] 4. Global average pooling layer:

[0081] A global average pooling layer is applied after all convolutional and transformer modules to reduce the feature dimension and control overfitting.

[0082] 5. Fully connected layer:

[0083] The output of the global average pooling layer is connected to the fully connected layer for final classification, and the output corresponds to the probability of each object category.

[0084] 6. Output layer:

[0085] Finally, the output of the fully connected layer is converted into a probability distribution through the Softmax activation function, which represents the predicted probability of each category.

[0086] 7. Model compilation:

[0087] Use the cross entropy loss function (CategoricalCrossentropy) and Adam optimizer to compile the model. The formula of the loss function is:

[0088]

[0089] Where: L is the loss value; N is the number of samples; y i is the true category label; p i The class probabilities predicted by the model.

[0090] 8. Model training:

[0091] The model is trained using the training set and the validation set is used to evaluate the model performance at the end of each epoch, with an early stopping strategy set to prevent overfitting.

[0092] Furthermore, the above model evaluation includes:

[0093] 1. Model testing: Use the test set to evaluate the model performance and calculate indicators such as accuracy, recall, and F1-score. The calculation formula for F1-score is:

[0094]

[0095] in:

[0096] Precision is the accuracy rate, which indicates the proportion of positive class predictions that are actually positive;

[0097] Recall is the recall rate, which indicates the proportion of actually positive classes that are correctly predicted to be positive.

[0098] 2. Visualization results: Visualize the confusion matrix to show the performance of the model in each category and help analyze the classification effect.

[0099] Furthermore, the above model optimization and adjustment include:

[0100] 1. Hyperparameter adjustment: According to the model evaluation results, adjust the learning rate, batch size, number of training rounds and other hyperparameters for retraining;

[0101] 2. Model fusion: Use ensemble learning methods, such as voting mechanisms, to combine the prediction results of multiple models to improve the overall classification accuracy;

[0102] 3. Model saving: Save the trained model as a file for subsequent use and deployment.

[0103] Steps 4 to 7 of the present application are a model architecture that uses a residual neural network, including: first adopting a model architecture based on a residual neural network, using multiple residual blocks, and adding the input directly to the output through jump connections to alleviate the gradient vanishing problem; then connecting a transformer module after the residual neural network, and using a self-attention mechanism to process the convolutional layer output to capture global features; then using the cross entropy loss function and Adam optimizer to compile the model, and using an early stopping strategy during training to prevent overfitting; finally, using a test set to evaluate the performance of the trained model, calculating the accuracy, recall rate, and F1-score, and visualizing the results to optimize the classification effect.

[0104] It should be noted that the present invention uses the ResNet-50 model to classify satellite images. The residual learning mechanism of ResNet is the key. By introducing residual blocks and skip connections, the network can be very deep without the gradient vanishing problem, thereby extracting higher-level and more abstract features.

[0105] It should be noted that the data preprocessing steps of the present invention include: resizing of large-size images, normalization of pixel values, data enhancement, and multispectral band combination, etc. These preprocessing steps can reduce computational costs, accelerate convergence, expand training sets, and highlight discriminative features, thereby improving model performance.

[0106] It should be noted that the present invention reasonably divides the training set and the test set and ensures the consistency of their category distribution. By training on a large-scale training set and evaluating the generalization performance on a reserved test set, a more robust model can be obtained.

[0107] It should be noted that the present invention uses a training set to train the ResNet-50 model to learn the characteristic patterns of satellite images. Pre-trained models such as weights pre-trained on ImageNet can be used for transfer learning to improve model performance.

[0108] Those skilled in the art should understand that the embodiments of the present invention shown in the above description are only examples and do not limit the present invention. The purpose of the present invention has been fully and effectively achieved. The functions and structural principles of the present invention have been demonstrated and explained in the embodiments, and the embodiments of the present invention may be deformed or modified in any way without departing from the principles.

Claims

1. An image classification method based on residual neural network, characterized by: The following steps are involved: Collect high-resolution satellite remote sensing image data to ensure coverage of a variety of landform types; Preprocess the images, including resizing them to a smaller size, normalizing pixel values, and enhancing data; Divide the processed data set into training set, validation set and test set; Adopt a model architecture based on residual neural network, use multiple residual blocks, and add input directly to output through skip connection to alleviate the gradient vanishing problem; A transformer module is connected after the residual neural network, and the self-attention mechanism is used to process the output of the convolution layer to capture global features. Use the cross entropy loss function and Adam optimizer to compile the model, and adopt an early stopping strategy during training to prevent overfitting; Use the test set to evaluate the performance of the trained model, calculate the accuracy, recall, and F1-score, and visualize the results to optimize the classification effect.

2. The image classification method based on residual neural network according to claim 1, characterized in that: The image preprocessing step includes data enhancement technology, and the data enhancement technology includes random rotation, flipping, translation and adding Gaussian noise.

3. The image classification method based on residual neural network according to claim 1, characterized in that: In the data preprocessing step, pixel value normalization is performed using the formula: Among them, x′ is the normalized pixel value, x is the original pixel value, μ is the mean of the pixel values, and σ is the standard deviation of the pixel values.

4. The image classification method based on residual neural network according to claim 1, characterized in that: The residual neural network part includes multiple residual blocks, each residual block directly adds the input to the output through a skip connection.

5. The image classification method based on residual neural network according to claim 1, characterized in that: The converter module calculation formula is: Among them, Q is the query matrix, K is the key matrix, V is the value matrix, and d k is the dimension of the key vector.

6. The image classification method based on residual neural network according to claim 1, characterized in that: The formula of the loss function is: Where: L is the loss value; N is the number of samples; y i is the true category label; p i The class probabilities predicted by the model.

7. The image classification method based on residual neural network according to claim 1, characterized in that: The multispectral bands are combined to highlight the spectral characteristics of different objects. The combination method includes calculating the vegetation index NDVI, and the formula is: Among them, NIR is the pixel value of the near-infrared band, and Red is the pixel value of the red band.

8. The image classification method based on residual neural network according to claim 1, characterized in that: An early stopping strategy is set during the model training process, and training is stopped when the validation set loss no longer decreases.

9. The image classification method based on residual neural network according to claim 1, characterized in that: When the test set is used to evaluate the performance of the model, the calculated indicators include accuracy, recall and F1-score, and the confusion matrix is ​​displayed through visualization technology to help analyze the classification effect. The calculation formula of F1-score is: Among them, Precision is the accuracy rate, which means the proportion of positive class predictions that are actually positive; Recall is the recall rate, which indicates the proportion of actually positive classes that are correctly predicted to be positive.

Images