Unmanned aerial vehicle image vegetation classification method based on improved DeepLabV3 +

Abstract

The invention discloses an unmanned aerial vehicle image vegetation classification method based on improved DeepLabV3 +, and the method comprises the steps: preparing an unmanned aerial vehicle image data set, carrying out the Python programming, carrying out the horizontal overturning, vertical overturning and diagonal mirror image expansion of a sample, dividing a training set according to a ratio of 7: 2: 1, cutting an image, constructing a vegetation feature project, extracting texture features through a gray-level co-occurrence matrix, and carrying out the classification of the image vegetation through employing the gray-level co-occurrence matrix. A specific visible light vegetation index is selected, a VDVI index is screened out through a ReliefF algorithm and fused with GEntropy, then a vegetation classification model is constructed, a backbone network is replaced with MobileNetV2, the voidage of an ASPP module is adjusted, an ECBAM is added, features are fused for model training, an SGD optimizer and a cosine annealing algorithm are used for adjusting the learning rate until network convergence is achieved, and finally a test set is used for testing. And carrying out vegetation classification layer-by-layer experiments. The method effectively solves the problem of unmanned aerial vehicle image vegetation classification, and improves the classification precision and model performance through multi-aspect improvement.

Description

Technical Field

[0001] The present invention relates to the field of vegetation classification in drone images, and in particular to a vegetation classification method for drone images based on an improved DeepLabV3+. Background Art

[0002] With the rapid development of fields such as ecological and environmental monitoring and precision agriculture, vegetation classification using drone imagery has become an important means of obtaining vegetation information. Drones, with their flexibility and ability to capture high-resolution imagery, provide a wealth of data for vegetation research. However, traditional deep learning-based vegetation classification methods, particularly those using the original DeepLabV3+ model, struggle to meet the growing demand for high-precision classification and exhibit significant shortcomings in practical applications.

[0003] The primary shortcoming of existing technologies lies in imperfect data processing. Traditional methods lack efficient data augmentation strategies and rely solely on limited raw datasets, resulting in a shortage of sample sizes and insufficient diversity. This prevents the model from fully learning the characteristics of various vegetation types during training. Faced with complex and ever-changing real-world scenarios, the model's generalization ability is weak, resulting in a significant reduction in classification accuracy. Furthermore, its relatively simple feature extraction method fails to fully exploit the multi-dimensional information of vegetation in drone imagery, such as texture and spectrum, and cannot effectively distinguish between different vegetation types, limiting the improvement of classification results.

[0004] Another major drawback lies in deficiencies in model construction and optimization. The original DeepLabV3+ model utilizes a complex Xception backbone network structure and requires extensive computation. Given limited hardware resources, training efficiency is extremely low, making rapid deployment difficult. Furthermore, the model's ASPP module and attention mechanism are inadequate for capturing vegetation features at varying scales when processing high-resolution drone imagery. This inability to accurately process vegetation edges and details leads to biased classification results. Furthermore, traditional training optimization strategies are limited in their simplicity and lack flexibility in learning rate adjustment, which can easily lead the model into local optimal solutions, causing overfitting and severely impacting its stability and reliability. Summary of the Invention

[0005] In order to overcome the shortcomings and deficiencies of the existing technology, the present invention provides a vegetation classification method for drone images based on improved DeepLabV3+.

[0006] The technical solution adopted by the present invention is a UAV image vegetation classification method based on improved DeepLabV3+, comprising the following steps: Step S1: Obtain the original UAV image dataset and implement an automated data augmentation strategy using Python programming. Perform sample expansion operations on the dataset by horizontal, vertical, and diagonal mirror transformations. The expanded dataset is divided into training, validation, and test sets in a fixed ratio of 7:2:1. The training images are cropped to a uniform size of 1024 × 1024 pixels. Step S2: Construct a vegetation feature engineering system. Based on the principle of gray-level co-occurrence matrix, eight statistical quantities such as mean and variance are constructed in the red, green, and blue bands respectively, and 24 texture features are extracted. Seven specific indices are selected from the visible light vegetation index set, and the ReliefF algorithm is used to perform feature evaluation and sorting. The VDVI index and G_Entropy are selected from them and integrated with the remote sensing data sample set through band superposition and fusion technology. Step S3: Build an optimized vegetation classification model architecture, replacing the Xception backbone feature extraction network in the DeepLabV3+ network with the MobileNetV2 network. For the ASPP module, add 3×3 convolutional layers with dilation rates of 2 and 4 before the convolutional layer with dilation rate of 6, and remove the convolutional layer with dilation rate of 18. In the decoding stage of the DeepLabV3+ network, introduce the improved CBAM module ECBAM. Step S4: Input the fused feature data into the constructed model. Relying on a single NVIDIA A100 40G computing device, based on the PyTorch deep learning framework, using the SGD optimizer, and using the cosine annealing algorithm to dynamically adjust the learning rate, the network is brought to a convergence state through iterative training. Step S5: Input the fuzzy processed image data into the trained model for inference operation, use the drone image test set to carry out testing, and implement a step-by-step progressive experimental operation of vegetation classification.

[0007] Furthermore, in step S1, the sample expansion operation of performing horizontal mirror transformation, vertical mirror transformation and diagonal mirror transformation on the data set is implemented by calling the OpenCV library function in the Python environment, and the corresponding transformation operation is performed on each image in the original data set by writing an automated script to generate an expanded data set.

[0008] Furthermore, in step S2, when extracting texture features based on the grayscale co-occurrence matrix principle, the texture features are calculated under different parameter combinations by adjusting the grayscale quantization level, offset direction and distance parameters to obtain multiple sets of texture feature data, and then the multiple sets of data are subjected to dimensionality reduction processing through the principal component analysis method to obtain the final 24 texture features.

[0009] Furthermore, in step S3, when the MobileNetV2 network replaces the Xception backbone feature extraction network, the convolution kernel size and number of channels of the depthwise separable convolutional layer in the MobileNetV2 network are adjusted to match the input and output dimensions of the subsequent structure of the DeepLabV3+ network, thereby ensuring the consistency of feature extraction and transmission.

[0010] Furthermore, in step S3, after the ASPP module is improved, an adaptive weight adjustment strategy is adopted during the network training process to dynamically adjust the weight ratio of the newly added convolutional layer and the original convolutional layer according to the changes in the loss function at different training stages, thereby optimizing the network's learning ability for features of different scales.

[0011] Furthermore, in step S3, after performing the Maxpool and Avgpool operations and the dilation coefficient 2 dilation convolution operation, the ECBAM module performs feature splicing on the acquired feature map, performs channel compression through a 1×1 convolution layer, and then inputs it into the subsequent 3×3 depth-separable convolution layer to further extract spatial and channel features.

[0012] Furthermore, in step S4, when the learning rate is dynamically adjusted using the cosine annealing algorithm, combined with the early stopping strategy, the training is terminated early when the validation set loss no longer decreases for multiple consecutive training cycles to avoid overfitting, and the current optimal model parameters are recorded at the same time.

[0013] Furthermore, in step S5, the image is blurred by using a Gaussian blur algorithm, and multiple groups of images with different blur levels are generated by setting different Gaussian kernel sizes and standard deviation parameters to test the robustness of the model to blurred images.

[0014] Furthermore, in step S5, the layer-by-layer progressive experimental operation of vegetation classification is to perform a complete training and testing process on the model after each layer of improvement operation is completed, and record different evaluation indicators such as classification accuracy, recall rate, F1 value, etc. of the model on the test set after each improvement to form comparative analysis data.

[0015] Furthermore, in step S2, the seven specific indices are selected from the visible light vegetation index set by calculating the mutual information value between each index and the vegetation category label, and selecting the top seven indices with the mutual information value, to ensure that the selected indices have a strong ability to distinguish vegetation classification.

[0016] Beneficial Effects: This paper proposes a vegetation classification method for drone imagery based on an improved DeepLabV3+. At the data processing level, this method uses Python programming to implement an automated data augmentation strategy for horizontal, vertical, and diagonal mirroring of the original dataset, expanding the sample size and enriching data diversity. The training, validation, and test sets are scientifically divided into a 7:2:1 ratio, and the training image sizes are standardized, laying a solid data foundation for subsequent model training. In terms of feature engineering, 24 texture features are extracted based on the gray-level co-occurrence matrix. After parameter adjustment and principal component analysis optimization, these features are combined with seven selected visible light vegetation indices. The ReliefF algorithm and mutual information calculation are used to select key indices. Through band superposition and fusion, vegetation features are effectively extracted, enhancing the model's ability to capture vegetation information. In model construction, the Xception backbone network is replaced with MobileNetV2, and convolutional layer parameters are adjusted to ensure consistent feature transfer. The ASPP module is improved, with optimized void ratio settings and weight adjustment strategies, enhancing the ability to learn features at different scales. The ECBAM module is introduced to improve the feature extraction process and enhance the network's extraction of spatial and channel features. During model training and testing, the SGD optimizer is combined with the cosine annealing algorithm and early stopping strategy to dynamically adjust the learning rate, avoid overfitting, and ensure the efficiency and quality of model training. The Gaussian blur algorithm is used to generate images with different blur levels for testing. Combined with layer-by-layer progressive experiments, the model performance is comprehensively evaluated, making this method more accurate, robust, and adaptable in vegetation classification using drone images, and able to meet the needs of vegetation classification in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a flow chart of the method steps of the present invention; Figure 2 This is the improved DeepLabV3+ model structure diagram of the present invention; Figure 3 This is the MobileNetV2 network structure diagram of the present invention; Figure 4 This is the improved ASPP structure diagram of the present invention; Figure 5 This is the ECBAM diagram of the improved attention mechanism module of the present invention. DETAILED DESCRIPTION

[0018] It should be noted that, unless there is a conflict, the embodiments and features in the embodiments of this application can be combined with each other. The present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0019] like Figure 1 As shown in the figure, a vegetation classification method for UAV images based on improved DeepLabV3+ includes the following steps: Step S1: Obtain the original UAV image dataset and implement an automated data augmentation strategy using Python programming. Perform sample expansion operations on the dataset by horizontal, vertical, and diagonal mirror transformations. The expanded dataset is divided into training, validation, and test sets in a fixed ratio of 7:2:1. The training images are cropped to a uniform size of 1024 × 1024 pixels. Specifically, we first need to obtain a dataset of raw drone imagery, which contains vegetation information from various scenarios. To increase the number and diversity of samples in the dataset, we use Python programming and its rich library functions to implement an automated data augmentation strategy. Specifically, we perform sample augmentation operations such as horizontal, vertical, and diagonal mirroring, flipping the original images at multiple angles to generate new sample data. The augmented dataset is then strictly divided into training, validation, and test sets in a fixed ratio of 7:2:1. The training set is used to learn model parameters, the validation set is used to monitor model performance and adjust hyperparameters during training, and the test set is used to evaluate the generalization ability of the final trained model. Finally, the training images are cropped and resized to a uniform 1024×1024 pixel size to ensure consistent image size for the input model, facilitating unified model processing.

[0020] The 7:2:1 dataset split ratio was chosen, a relatively reasonable allocation method verified by extensive experiments. The 70% training set provides sufficient samples for the model to learn parameters, while the 20% validation set effectively monitors changes in model performance during training, promptly identifying overfitting or underfitting so that hyperparameters can be adjusted. The 10% test set objectively evaluates the model's generalization ability on unknown data. Cropping the training images to 1024×1024 pixels takes into account both the resolution of the drone imagery and the computational power of the model. This size preserves sufficient vegetation detail while ensuring computational efficiency, while avoiding excessive memory usage and insufficient computing resources due to large image size.

[0021] Data augmentation expanded the number of dataset samples and enriched the data diversity, enabling the model to learn vegetation characteristics from a wider range of angles and scenarios, effectively improving the model's generalization capabilities. Reasonable dataset partitioning ensured the scientific nature and effectiveness of the model during training, validation, and testing, enabling accurate evaluation of model performance. Unifying the training image size and standardizing the model's input format improved the stability and efficiency of model training, laying a solid data foundation for accurate vegetation classification in subsequent models.

[0022] Step S2: Construct a vegetation feature engineering system. Based on the principle of gray-level co-occurrence matrix, eight statistical quantities such as mean and variance are constructed in the red, green, and blue bands respectively, and 24 texture features are extracted. Seven specific indices are selected from the visible light vegetation index set, and the ReliefF algorithm is used to perform feature evaluation and sorting. The VDVI index and G_Entropy are selected from them and integrated with the remote sensing data sample set through band superposition and fusion technology. Specifically, when constructing the vegetation feature engineering system, based on the principle of gray-level co-occurrence matrices, eight statistical measures—mean, variance, homogeneity, contrast, dissimilarity, entropy, second-order moment, and correlation—were constructed for the red, green, and blue bands, respectively. This allowed the extraction of 24 texture features. Simultaneously, seven specific indices were selected from the visible light vegetation index set, including the visible light band difference vegetation index and the red-green ratio index. These features were evaluated and ranked using the ReliefF algorithm. By calculating the correlation between each feature and the vegetation class label, the two most discriminative features, the VDVI index and G_Entropy, were selected. Finally, using band overlay fusion technology, the selected features were integrated with the remote sensing data sample set to form a feature dataset containing multi-dimensional information.

[0023] Eight texture features were extracted from three bands because different bands reflect distinct spectral characteristics of vegetation. By extracting texture features from multiple bands, a more comprehensive capture of vegetation texture information is achieved. Seven visible vegetation indices were selected and the ReliefF algorithm was used for feature selection. This approach aims to identify the features that contribute most to vegetation classification from among the numerous vegetation indices, thereby reducing the interference of redundant features on the model, lowering the model's computational complexity, and improving model training efficiency and classification accuracy. The VDVI index and G_Entropy were chosen because they have been shown in experimental validation to have high discriminatory power for vegetation classification and are effective in distinguishing different types of vegetation.

[0024] By extracting rich texture features, screening key vegetation indices, and fusing them, the model is provided with feature data containing multi-dimensional information such as vegetation spectrum and texture. These features more accurately describe vegetation characteristics, enabling the model to better distinguish different types of vegetation, improving the accuracy and reliability of vegetation classification, and enhancing the model's adaptability to complex vegetation scenarios.

[0025] Step S3: Build an optimized vegetation classification model architecture, replacing the Xception backbone feature extraction network in the DeepLabV3+ network with the MobileNetV2 network. For the ASPP module, add 3×3 convolutional layers with dilation rates of 2 and 4 before the convolutional layer with dilation rate of 6, and remove the convolutional layer with dilation rate of 18. In the decoding stage of the DeepLabV3+ network, introduce the improved CBAM module ECBAM. Specifically, to construct the optimized vegetation classification model architecture, the Xception backbone feature extraction network in the DeepLabV3+ network was first replaced with the MobileNetV2 network. During this replacement process, the kernel size and number of channels of the depthwise separable convolutional layers in the MobileNetV2 network were adjusted to match the input and output dimensions of the subsequent DeepLabV3+ network structure. Next, the ASPP module was improved by adding 3×3 convolutional layers with dilation rates of 2 and 4 before the convolutional layer with dilation rate of 6, and removing the convolutional layer with dilation rate of 18. Finally, the improved CBAM module ECBAM was introduced in the decoding stage of the DeepLabV3+ network. After performing Maxpool and Avgpool operations and a dilated convolution with a dilation factor of 2, the feature maps obtained were concatenated, channel-compressed through a 1×1 convolutional layer, and then fed into a subsequent 3×3 depthwise separable convolutional layer to further extract spatial and channel features.

[0026] MobileNetV2 was chosen to replace Xception because it uses lightweight structures such as depthwise separable convolution. While maintaining feature extraction capabilities, it can significantly reduce the model's parameter count and computational complexity, improving the model's operational efficiency and making it more suitable for deployment in environments with limited hardware resources. The ASPP module's void ratio was adjusted to account for the scale variations and edge complexity of vegetation in drone imagery. By setting different void ratios, the network can capture contextual information in different receptive fields and enhance its processing of vegetation details and edges. The ECBAM module was introduced and a series of operations were performed to better focus on key vegetation features, suppress irrelevant information, and enhance the network's ability to extract spatial and channel features, thereby improving the model's classification accuracy.

[0027] Improvements to the model architecture have effectively reduced model complexity, improving operational efficiency and deployment flexibility. The optimized ASPP and ECBAM modules enhance the model's ability to capture vegetation features at different scales and focus on key features, enabling the model to more accurately extract vegetation features, thereby improving vegetation classification accuracy and adaptability to complex scenarios, providing more efficient and accurate model support for vegetation classification using drone imagery.

[0028] Step S4: Input the fused feature data into the constructed model. Relying on a single NVIDIA A100 40G computing device, based on the PyTorch deep learning framework, using the SGD optimizer, and using the cosine annealing algorithm to dynamically adjust the learning rate, the network is brought to a convergence state through iterative training. Specifically, the feature data fused in step S2 is input into the model constructed in step S3. A single NVIDIA A10040G is selected as the computing device, and its powerful computing power is used to accelerate the model training process. The training environment is built based on the PyTorch deep learning framework, and the SGD optimizer is used to optimize the model parameters. The cosine annealing algorithm is used to dynamically adjust the learning rate. A higher learning rate is set at the beginning of training to make the model converge quickly. As the training progresses, the learning rate is gradually reduced to prevent the model from skipping the optimal solution when it is close to the optimal solution. Through continuous iterative training, the parameters of the network are continuously optimized until the network reaches a convergence state, that is, the loss function of the model no longer decreases significantly, and the performance of the model tends to be stable.

[0029] The NVIDIA A10040G was chosen as the computing device due to its powerful GPU computing capabilities, which can quickly handle the large number of matrix operations required during deep learning model training, significantly reducing training time. The PyTorch deep learning framework was adopted due to its flexible dynamic computational graph mechanism, which facilitates model building, debugging, and modification. The SGD optimizer combined with the cosine annealing algorithm for learning rate adjustment was chosen because this combination has been widely proven effective in deep learning training. The SGD optimizer is simple and efficient, while the cosine annealing algorithm dynamically adjusts the learning rate based on training progress, balancing the model's convergence speed and accuracy and preventing the model from becoming trapped in local optimal solutions.

[0030] Leveraging high-performance computing equipment and optimized training strategies, we significantly improved model training efficiency, shortened training time, and reduced R&D costs. Dynamically adjusting the learning rate enables the model to more stably converge to the global optimal solution, improving training quality and generalization capabilities, ensuring that the trained model can accurately classify vegetation in drone imagery in real-world applications.

[0031] Step S5: Input the fuzzy processed image data into the trained model for inference operation, use the drone image test set to carry out testing, and implement a step-by-step progressive experimental operation of vegetation classification.

[0032] Specifically, the image data after blurring is input into the trained model for inference operation. The blurring process uses the Gaussian blur algorithm. By setting different Gaussian kernel sizes and standard deviation parameters, multiple sets of images with different blur levels are generated to simulate the image blur that may occur in actual applications. The test work was carried out using the drone image test set of Gong County, Yibin City, Sichuan Province provided by the Sichuan Provincial Forestry Bureau. This test set is representative and authoritative and can objectively evaluate the performance of the model in actual scenarios. During the test process, a step-by-step progressive experimental operation of vegetation classification was implemented. That is, after each layer of improvement operation was completed, the model was subjected to a complete training and testing process. The classification accuracy, recall rate, F1 value and other evaluation indicators of the model on the test set after each improvement were recorded to form comparative analysis data to evaluate the impact of each improvement on the model performance.

[0033] The Gaussian blur algorithm was used to blur the images, and multiple sets of blurred images were generated using different parameters. This was done to comprehensively test the model's robustness to varying degrees of blur and assess its adaptability in complex real-world environments. A test dataset of drone imagery from Gong County, Yibin City, Sichuan Province, was chosen because it encompasses a wide variety of local vegetation types and complex geographical environments, providing a true reflection of the model's classification capabilities in real-world application scenarios. This step-by-step experimental approach clearly demonstrated the contribution of each improvement measure to model performance, facilitating analysis of the effectiveness and rationality of model improvements.

[0034] By testing fuzzy images, the model's adaptability and robustness in complex real-world environments were evaluated, ensuring its reliability in practical applications. By utilizing an authoritative test set and a scientific, step-by-step experimental approach, the model's performance was accurately assessed, clearly demonstrating the impact of each improvement on the model's classification results. This provided data support and theoretical basis for further model optimization, helping to promote the development and application of vegetation classification technology using drone imagery.

[0035] like Figure 2As shown, the improved DeepLabV3+ network structure of the present invention is mainly an Encoder-Decoder structure. In the Encoder part, it mainly includes two major parts: backbone and ASPP. The present invention uses MobileNetV2, the first part of the backbone output is sent to the improved ASPP module, and the second part is sent to the Decoder module. In the Decoder part, the low-level feature map from the backbone middle layer and the output from the ASPP module are received as input. First, the low-level feature map is subjected to channel dimensionality reduction using 1×1 convolution, and then the output feature map is obtained through the improved ECBAM module. The feature map from ASPP is interpolated and upsampled to obtain a feature map of the same size as the low-level feature map. The two feature maps obtained are spliced ​​and sent to a group of 3*3 convolution blocks for processing. Finally, linear interpolation upsampling is performed to obtain a predicted map with the same resolution as the original image.

[0036] Preferably, in step S1, the sample expansion operation of performing horizontal mirror transformation, vertical mirror transformation and diagonal mirror transformation on the data set is implemented by calling the OpenCV library function in the Python environment, and the corresponding transformation operation is performed on each image in the original data set by writing an automated script to generate an expanded data set.

[0037] Specifically, the OpenCV library function is called in the Python environment to perform sample expansion operations such as horizontal mirror transformation, vertical mirror transformation, and diagonal mirror transformation of the dataset. Specifically, by writing an automated script, the corresponding transformation operation can be accurately performed on each image in the original dataset. The OpenCV library provides a wealth of efficient image processing functions, which can be used to accurately control the flip direction and angle of the image to generate an expanded dataset. The significance of this is that data enhancement is achieved in an automated manner, which greatly improves the efficiency of data expansion and avoids the tediousness and possible errors of manual operations. At the same time, operations based on reliable library functions ensure the stability and consistency of the data enhancement process, providing sufficient and reliable sample data for subsequent model training, which helps to improve the model's ability to learn vegetation features from different perspectives and enhance the generalization performance of the model.

[0038] Preferably, in step S2, when extracting texture features based on the grayscale co-occurrence matrix principle, the texture features are calculated under different parameter combinations by adjusting the grayscale quantization level, offset direction and distance parameters, and multiple groups of texture feature data are obtained. The multiple groups of data are then subjected to dimensionality reduction processing by the principal component analysis method to obtain the final 24 texture features.

[0039] Specifically, when extracting texture features, the grayscale quantization level, offset direction, and distance parameters are adjusted to calculate texture features under different parameter combinations, thereby obtaining multiple sets of texture feature data. The grayscale quantization level affects the fineness of the texture features, while the offset direction and distance parameters determine the texture pattern captured by the texture features. After obtaining multiple sets of data, the principal component analysis method is used to reduce their dimensionality, remove redundant information, and obtain the final 24 texture features. The significance of this operation method is that by calculating texture features through multi-parameter combinations, the texture information of vegetation in the image can be more comprehensively mined, and different parameter combinations can capture texture patterns of different scales and directions. The principal component analysis dimensionality reduction process, while retaining key texture features, reduces the data dimension, reduces the computational complexity of the model, improves the efficiency of model training, and enables the model to focus on the most discriminative texture features, thereby improving the accuracy of vegetation classification.

[0040] Preferably, in step S3, when the MobileNetV2 network replaces the Xception backbone feature extraction network, the convolution kernel size and the number of channels of the depthwise separable convolution layer in the MobileNetV2 network are adjusted to match the input and output dimensions of the subsequent structure of the DeepLabV3+ network, thereby ensuring the consistency of feature extraction and transmission.

[0041] Specifically, such as Figure 3 As shown in the figure, the overall architecture of MobileNetV2 first uses 1×1 convolution to increase dimensionality, then 3×3 DW convolution, and finally 1×1 convolution to reduce dimensionality. The dimensionality increase consists of 1×1 point-by-point convolution, batch normalization, and ReLU6 activation. The next step is 3×3 depthwise separable convolution, batch normalization, and ReLU6 activation. The dimensionality reduction consists of 1×1 point-by-point convolution and batch normalization, without the ReLU6 activation function. This is because the ReLU6 activation function loses a lot of information in low-dimensional space, but retains more useful information in high-dimensional space.

[0042] During the replacement process, the convolution kernel size and number of channels of the depthwise separable convolution layer in the MobileNetV2 network are adjusted accordingly. The convolution kernel size determines the network's perception range of local features, while the number of channels affects the dimension and quantity of features extracted by the network. By reasonably adjusting these parameters to match the input and output dimensions of the subsequent structure of the DeepLabV3+ network, the consistency of feature transmission from the MobileNetV2 network to the subsequent layers of the DeepLabV3+ network is guaranteed. The significance of this operation lies in the fact that the MobileNetV2 network has a lightweight structure. By adjusting parameters to adapt to the original network, while reducing the overall number of model parameters and computational complexity, the effectiveness and integrity of feature extraction are ensured, so that the model has faster running speed and better hardware adaptability while maintaining high classification accuracy, making it more suitable for deployment in resource-constrained environments.

[0043] Preferably, in step S3, after the ASPP module is improved, an adaptive weight adjustment strategy is adopted during the network training process to dynamically adjust the weight ratio of the newly added convolutional layer and the original convolutional layer according to the changes in the loss function at different training stages, thereby optimizing the network's learning ability for features of different scales.

[0044] Specifically, such as Figure 4 As shown in the figure, based on the characteristics of high resolution of UAV images, irregular vegetation edges, complex shapes, different sizes, and uneven distribution, the present invention adjusts the original ASPP structure. The original ASPP uses void convolution rates of 6, 12, and 18. The present invention adds two 3×3 convolutions with void rates of 2 and 4 before the convolution with a void rate of 6, and removes the convolution with a void rate of 18.

[0045] After improving the ASPP module, an adaptive weight adjustment strategy was adopted during network training. This strategy dynamically adjusts the weight ratio of the newly added convolutional layers to the original convolutional layers based on the changes in the loss function at different training stages. In the early stages of training, the model's learning of various features is in the exploratory stage. At this time, appropriately increasing the weight of the newly added convolutional layers helps the network quickly learn the newly introduced features of different scales. As training progresses, the weights are gradually adjusted based on the feedback of the loss function, allowing the network to balance the learning of features of different scales and optimize the network's ability to learn vegetation features of different scales. The significance of this strategy lies in its ability to fully leverage the advantages of the improved ASPP module, avoid the network's over-reliance on the feature extraction mode of the original convolutional layer, enable the network to more flexibly adapt to the learning needs of different stages during training, and improve the network's ability to capture and utilize features of different scales in complex vegetation scenes, thereby improving the model's classification performance and adaptability to complex environments.

[0046] Preferably, in step S3, after performing Maxpool and Avgpool operations and a dilation convolution operation with a dilation coefficient of 2, the ECBAM module performs feature splicing on the acquired feature map, performs channel compression through a 1×1 convolution layer, and then inputs it into a subsequent 3×3 depth-separable convolution layer to further extract spatial and channel features.

[0047] Specifically, such as Figure 5 As shown in the figure, since the attention mechanism module CBAM uses Maxpool and Avgpool in the channel attention submodule, pooling downsampling will cause a large amount of information loss. In addition, the output information of the channel attention module is directly used as the input information of the spatial attention module, which will cause the spatial attention module to be overly dependent on the channel attention module. To address the above problems, the present invention proposes an improved attention module ECBAM. In order to reduce the feature information loss caused by pooling, while performing Maxpool and Avgpool operations, a dilated convolution operation with a dilation factor of 2 is added. The dilated convolution can expand the receptive field to compensate for the information loss caused by the pooling operation. In order to adapt to feature map inputs of different sizes, the present invention replaces the original two fully connected layers FC with 1×1 convolution layers, which can not only accept inputs of different sizes but also reduce the amount of computation. The obtained three-way feature map is added element by element through the Add module and then a Sigmoid operation is performed to obtain the channel attention feature map. In order to reduce the direct dependence of the spatial attention module on the channel attention module feature map Mc(F), effectively integrate global features and local features of different scales, and improve feature extraction capabilities, a 3×3 depth-separable convolution is added between the channel attention module and the spatial attention module. The feature map Fc is obtained through the convolution layer and input into the spatial attention module. The channel dimension is compressed to 1 through the Maxpool and Avgpool operations to retain spatial information. The pooled features are concat-convered, and features are extracted through a convolution layer. At the same time, the channel dimension is reduced to 1, and finally, after Sigmoid activation, the spatial attention feature map Ms(Fc) is obtained. Fc and Ms(Fc) are multiplied element by element to obtain the final attention module feature map F out .

[0048] After performing Maxpool and Avgpool operations, as well as a dilated convolution with a dilation factor of 2, the ECBAM module concatenates the acquired feature maps. Maxpool and Avgpool operations extract key feature information from different perspectives, respectively, while the dilated convolution expands the network's receptive field without increasing computational overhead. After concatenation, a 1×1 convolutional layer performs channel compression, reducing the number of channels in the feature map and lowering computational complexity. The compressed feature map is then fed into a subsequent 3×3 depthwise separable convolutional layer for further extraction of spatial and channel features. The significance of this series of operations lies in the comprehensive and efficient extraction of spatial and channel information from the feature map, focusing on key vegetation features while suppressing irrelevant information. This enhances the network's ability to represent vegetation features, enabling the model to more accurately distinguish between different vegetation types during classification, improving classification accuracy and reliability.

[0049] Preferably, in step S4, when dynamically adjusting the learning rate using the cosine annealing algorithm, combined with the early stopping strategy, when the validation set loss no longer decreases for multiple consecutive training cycles, the training is terminated in advance to avoid overfitting, and the current optimal model parameters are recorded at the same time.

[0050] Specifically, when dynamically adjusting the learning rate using the cosine annealing algorithm, an early stopping strategy is combined. The cosine annealing algorithm allows the learning rate to change according to the law of the cosine function during training. Maintaining a high learning rate in the early stages of training allows for rapid model convergence, while gradually reducing the learning rate as training progresses to prevent the model from skipping the optimal solution when it approaches it. The early stopping strategy terminates training early when the validation set loss stops decreasing for multiple consecutive training cycles. During training, the model may overfit, resulting in stagnant or even degraded performance on the validation set. In this case, the early stopping strategy allows for timely termination of training to avoid wasted resources and to record the current optimal model parameters. The significance of this combined strategy lies in that it ensures efficient model training and convergence to a good solution while effectively preventing overfitting. This improves the model's generalization and stability, ensuring that the trained model has reliable classification performance in practical applications.

[0051] Preferably, in step S5, the image is blurred by using a Gaussian blur algorithm, and multiple groups of images with different blur levels are generated by setting different Gaussian kernel sizes and standard deviation parameters to test the robustness of the model to blurred images.

[0052] Specifically, when blurring an image, a Gaussian blur algorithm is used. By setting different Gaussian kernel sizes and standard deviation parameters, multiple sets of images with varying degrees of blur are generated. The Gaussian kernel size determines the range of the blur, while the standard deviation parameter controls the degree of blur. By adjusting these two parameters, it is possible to simulate different degrees of image blur caused by factors such as shooting conditions and atmospheric environment in real applications. Using these blurred images to test the model can assess the model's robustness to blurred images. This is significant because blurred images are inevitable during actual drone image acquisition. By testing the model on blurred images, we can understand the model's classification capabilities when faced with blurred images, thereby determining the model's adaptability in complex real-world environments. This ensures the model's reliable application in real-world scenarios and makes the model more practical and stable.

[0053] Preferably, in step S5, the layer-by-layer progressive experimental operation of vegetation classification is to perform a complete training and testing process on the model after each layer of improvement operation is completed, and record different evaluation indicators such as classification accuracy, recall rate, F1 value, etc. of the model on the test set after each improvement to form comparative analysis data.

[0054] Specifically, this operation is to conduct a complete training and testing process for the model after each layer of improvement operation is completed. During each training and testing process, different evaluation indicators such as classification accuracy, recall rate, F1 value, etc. of the model on the test set are recorded. By comparing the various evaluation indicators of the model after each improvement, detailed comparative analysis data is formed. The significance of doing this is that it can clearly understand the specific impact of each improvement measure on the performance of the model and judge the effectiveness and rationality of different improvement methods. Through step-by-step progressive experiments, the performance change trend of the model in the process of gradual improvement can be systematically analyzed, providing accurate data support and scientific basis for further optimization of the model, helping R&D personnel to improve and perfect the model in a targeted manner, and promoting the continuous development of UAV image vegetation classification technology.

[0055] Preferably, in step S2, the seven specific indices are selected from the visible light vegetation index set by calculating the mutual information value between each index and the vegetation category label, and selecting the top seven indices with the mutual information value, to ensure that the selected indices have a strong ability to distinguish vegetation classification.

[0056] Specifically, the screening is performed by calculating the mutual information value between each index and the vegetation category label. The mutual information value reflects the degree of correlation between the index and the vegetation category. The top 7 indices with the highest mutual information values ​​are selected to ensure that the selected index has a strong ability to distinguish vegetation classification. The significance of this screening method is to accurately select the index that can most effectively distinguish different vegetation types from a large number of visible light vegetation indices, remove redundant indices with low correlation with vegetation classification, reduce the input dimension of the model, and reduce the computational complexity of the model. At the same time, it ensures that the vegetation index input to the model can accurately reflect the characteristics of the vegetation, provide the model with high-quality feature data, thereby improving the classification accuracy and efficiency of the model, and enabling the model to more accurately classify vegetation in drone images.

[0057] Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various equivalent changes, modifications, substitutions and variations may be made to these embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A UAV image vegetation classification method based on improved DeepLabV3+, characterized by: The following steps are involved: Step S1: Obtain the original UAV image dataset and implement an automated data augmentation strategy using Python programming. Perform sample expansion operations on the dataset by horizontal, vertical, and diagonal mirror transformations. The expanded dataset is divided into training, validation, and test sets in a fixed ratio of 7:2:

1. The training images are cropped to a uniform size of 1024 × 1024 pixels. Step S2: Construct a vegetation feature engineering system. Based on the principle of gray-level co-occurrence matrix, construct mean and variance statistics for the red, green, and blue bands, and extract 24 texture features. Select seven specific indices from the visible light vegetation index set, use the ReliefF algorithm to perform feature evaluation and sorting, select the VDVI index and G_Entropy, and integrate them with the remote sensing data sample set through band superposition and fusion technology. Step S3: Build an optimized vegetation classification model architecture, replacing the Xception backbone feature extraction network in the DeepLabV3+ network with the MobileNetV2 network; For the ASPP module, a 3×3 convolutional layer with dilation rates of 2 and 4 is added before the convolutional layer with dilation rate of 6, and the convolutional layer with dilation rate of 18 is removed. The improved CBAM module ECBAM is introduced in the decoding stage of the DeepLabV3+ network. Step S4: Input the fused feature data into the constructed model. Relying on a single NVIDIA A100 40G computing device, based on the PyTorch deep learning framework, using the SGD optimizer, and using the cosine annealing algorithm to dynamically adjust the learning rate, the network is brought to a convergence state through iterative training. Step S5: Input the fuzzy processed image data into the trained model for inference operation, use the drone image test set to carry out testing, and implement a step-by-step progressive experimental operation of vegetation classification.

2. The method for vegetation classification based on drone images using improved DeepLabV3+ according to claim 1, characterized in that: In step S1, sample expansion operations of performing horizontal mirror transformation, vertical mirror transformation, and diagonal mirror transformation on the dataset are performed by calling OpenCV library functions in a Python environment. By writing an automated script, the corresponding transformation operations are performed on each image in the original dataset to generate an expanded dataset.

3. The method for vegetation classification based on drone images using improved DeepLabV3+ according to claim 1, characterized in that: In step S2, when extracting texture features based on the gray level co-occurrence matrix principle, the texture features are calculated under different parameter combinations by adjusting the gray level quantization level, offset direction and distance parameters, and multiple groups of texture feature data are obtained. The multiple groups of data are then subjected to dimensionality reduction processing using the principal component analysis method to obtain the final 24 texture features.

4. The method for vegetation classification based on drone images using improved DeepLabV3+ according to claim 1, characterized in that: In step S3, when the MobileNetV2 network replaces the Xception backbone feature extraction network, the convolution kernel size and number of channels of the depthwise separable convolutional layer in the MobileNetV2 network are adjusted to match the input and output dimensions of the subsequent structure of the DeepLabV3+ network, thereby ensuring the consistency of feature extraction and transmission.

5. The method for vegetation classification based on drone images using improved DeepLabV3+ according to claim 1, characterized in that: In step S3, after the ASPP module is improved, an adaptive weight adjustment strategy is adopted during the network training process to dynamically adjust the weight ratio of the newly added convolutional layer and the original convolutional layer according to the changes in the loss function at different training stages, thereby optimizing the network's learning ability for features of different scales.

6. The method for vegetation classification based on drone images using improved DeepLabV3+ according to claim 1, characterized in that: In step S3, after performing the Maxpool and Avgpool operations and the dilation coefficient 2 dilation convolution operation, the ECBAM module performs feature splicing on the acquired feature map, performs channel compression through a 1×1 convolution layer, and then inputs it into the subsequent 3×3 depthwise separable convolution layer to further extract spatial and channel features.

7. The method for vegetation classification based on drone images using improved DeepLabV3+ according to claim 1, characterized in that: In step S4, when dynamically adjusting the learning rate using the cosine annealing algorithm, combined with the early stopping strategy, the training is terminated early when the validation set loss no longer decreases for multiple consecutive training cycles to avoid overfitting, and the current optimal model parameters are recorded.

8. The method for vegetation classification based on drone images using improved DeepLabV3+ according to claim 1, characterized in that: In step S5, the image is blurred by using a Gaussian blur algorithm. By setting different Gaussian kernel sizes and standard deviation parameters, multiple groups of images with different blur levels are generated to test the robustness of the model to blurred images.

9. The method for vegetation classification based on drone images using improved DeepLabV3+ according to claim 1, characterized in that: The step S5, the layer-by-layer progressive experimental operation of vegetation classification, is to conduct a complete training and testing process on the model after each layer of improvement operation is completed, and record the classification accuracy, recall rate, F1 value and other evaluation indicators of the model on the test set after each improvement to form comparative analysis data.

10. The method for vegetation classification based on drone images using improved DeepLabV3+ according to claim 1, characterized in that: In step S2, seven specific indices are selected from the set of visible light vegetation indices. The mutual information value between each index and the vegetation category label is calculated, and the top seven indices with the highest mutual information values ​​are selected to ensure that the selected indices have a strong ability to distinguish vegetation classification.

Images