Vegetation classification method and system based on unmanned aerial vehicle remote sensing image

Abstract

The invention discloses a vegetation classification method and system based on an unmanned aerial vehicle remote sensing image, and the method comprises the steps: making an ortho-image and DEM data in a target region based on remote sensing image data in the target region obtained through employing a sensor carried by an unmanned aerial vehicle; constructing a data set by using the obtained orthoimage and DEM data, dividing the data set into a training set, a verification set and a test set according to a preset proportion, and generating a vegetation classification label; constructing a vegetation classification model based on the convolutional neural network, and training, verifying and testing the model by using the obtained training set, verification set and test set; and taking the orthoimage to be predicted and the DEM data as input, and utilizing the vegetation classification model obtained by training to carry out vegetation classification prediction. The method has the advantages that the remote sensing image vegetation classification precision is improved, the calculation efficiency is improved and the like, and an efficient and high-precision vegetation classification mode can be provided for high-resolution unmanned aerial vehicle remote sensing image data.

Description

Technical Field

[0001] This invention relates to the field of vegetation classification technology, specifically to a vegetation classification method and system based on UAV remote sensing imagery. Background Technology

[0002] With the rapid development of drone technology, drones have become increasingly capable of endurance and carrying capacity, and are now widely used in the surveying and mapping field. Compared with satellite remote sensing, drones can carry different types of sensor equipment to efficiently acquire high-precision images of designated areas. Furthermore, because drone observation positions are often below clouds, drones can observe ground features at closer distances, facilitating the acquisition of remote sensing data within the target area. The acquired data is less affected by weather conditions, reducing post-processing steps.

[0003] Digital Elevation Models (DEMs) describe the spatial distribution of topography within a region. Vegetation growth and distribution are influenced by topography; therefore, DEM data can be incorporated into vegetation classification using remote sensing imagery as a reference. There are three main methods for vegetation classification using imagery: visual interpretation, pixel-based classification, and object-oriented classification. Visual interpretation requires highly skilled interpreters and is inefficient and time-consuming. Although it can achieve high accuracy, it is not suitable for large-scale data acquisition of vegetation classification information. Early vegetation classification primarily used pixel-based methods. To address the low accuracy of pixel-based methods, object-oriented classification was developed. Object-oriented classification divides the image into multiple light spots, classifying objects not as individual pixels but as objects with a certain degree of homogeneity. It then extracts feature information from these basic classification units. Object-oriented classification considers features such as object shape, texture, and contextual relationships, which can improve classification accuracy.

[0004] Machine learning can be categorized into supervised learning, unsupervised learning, and semi-supervised learning, primarily differing in whether the training dataset contains pre-defined labels. In high-precision UAV remote sensing imagery, supervised learning can learn precise mapping relationships from a large amount of labeled data, capturing complex texture and edge information. Deep learning can learn features from massive training data for UAV imagery and perform vegetation classification. Convolutional neural networks (CNNs) can extract information from spatial data, learning various abstract information such as spatial features from large datasets to identify the category and quantity of targets. Combining UAV technology with CNN technology and applying it to vegetation classification provides an efficient and accurate method for vegetation classification. Summary of the Invention

[0005] This invention provides a vegetation classification method and system based on UAV remote sensing imagery to solve the problem of vegetation classification and prediction based on UAV remote sensing imagery and DEM data.

[0006] According to the first aspect, one embodiment provides a vegetation classification method based on UAV remote sensing imagery, the method comprising:

[0007] Based on remote sensing image data of the target area obtained by using sensors on a drone, orthophotos and DEM data of the target area are produced.

[0008] A dataset was constructed using the obtained orthophotos and DEM data. The dataset was divided into training set, validation set and test set according to a preset ratio, and vegetation classification labels were generated.

[0009] A vegetation classification model based on a convolutional neural network was constructed, and the model was trained, validated, and tested using the obtained training, validation, and test sets.

[0010] Using orthophotos and DEM data to be predicted as input, vegetation classification prediction is performed using a trained vegetation classification model.

[0011] Furthermore, based on remote sensing image data of the target area acquired using sensors mounted on a drone, orthophotos and DEM data of the target area are created, specifically including:

[0012] The remote sensing image data includes three bands of visible light: red, green, and blue.

[0013] Furthermore, based on remote sensing image data of the target area acquired using sensors mounted on a drone, orthophotos and DEM data of the target area are created, specifically including:

[0014] Based on the acquired point cloud data and image data of the target area, POS (Position and Orientation System) calculations were performed using Inertial Explorer software to obtain EO (Exterior Orientation) data corresponding to the image data. Based on the obtained image data and corresponding EO data, Agisoft Metashape software was used to generate DEM data and orthophotos of the target area through steps such as aligning photos, optimizing alignment methods, and generating dense point clouds. The DEM data and orthophotos were then standardized.

[0015] Furthermore, a dataset is constructed using the obtained orthophotos and DEM data, specifically including:

[0016] The orthophoto and DEM data are cropped to the same size as the model channels using the sliding window method; the edge prediction neglect method is used to solve the stitching marks that occur when the image is cropped in a regular manner; and the edge repetition rate with a preset percentage value is used to solve the problem of image edge feature loss during cropping.

[0017] Furthermore, vegetation classification labels are generated, specifically including:

[0018] Create binary and multi-class vector labels. The binary vector labels include two categories: vegetation and non-vegetation. The multi-class vector labels include multiple vegetation types present in the target area.

[0019] Furthermore, vegetation classification labels are generated, specifically including:

[0020] The training set is labeled at the pixel level using ArcGIS software, meaning each pixel has a label. The label data is then paired with the training data and used as input data for model training.

[0021] Furthermore, vegetation classification models based on convolutional neural networks specifically include:

[0022] The vegetation classification model adopts an improved DeepLabV3+ network model; the DeepLabV3+ feature extraction backbone network Xception is replaced with the lightweight MobileNetV3; the ASPP module is modified by adding two 3×3 dilated convolutions with dilation rates of 2 and 4 respectively; in the feature fusion stage of the model decoder, the DEM is stitched with the low-level image feature map to obtain an enhanced feature map; and the attention mechanism CBAM is introduced in the decoding stage of the model to allow the model to focus on the task region in a targeted manner.

[0023] Furthermore, the processing steps of the vegetation classification model based on convolutional neural networks include:

[0024] First, input multi-channel data consisting of orthophotos and DEM data, which then enters the backbone network MobileNetV3 in the encoder stage;

[0025] The feature maps extracted by the backbone network MobileNetV3 enter two different branches. One branch enters the ASPP module to extract high-level features of the image. In the ASPP module, dilated convolutions with dilation rates of 2, 4, 6, 12, and 18 are applied to the input feature maps to extract contextual information at different scales, and then the feature maps enter the decoder stage.

[0026] The low-level features containing edge information output by another branch of the backbone network MobileNetV3 are concatenated with the input DEM elevation data to obtain an enhanced feature map. The enhanced feature map is then passed through the attention mechanism CBAM and then fused with the high-level features output by the ASPP module through the concat function. Finally, the final segmentation result is output through a convolutional layer.

[0027] Furthermore, vegetation classification models based on convolutional neural networks specifically include:

[0028] The processed training set data is input into a vegetation classification model based on a convolutional neural network, and the label set is used as the output to train the network. The backpropagation algorithm is used to obtain the model parameters, resulting in a trained vegetation classification model.

[0029] According to a second aspect, one embodiment provides a vegetation classification system based on UAV remote sensing imagery, the system comprising:

[0030] The data acquisition module is used to create orthophotos and DEM data of the target area based on remote sensing image data acquired by sensors on the UAV within the target area.

[0031] The dataset construction module is used to construct a dataset using the obtained orthophotos and DEM data, divide the dataset into training set, validation set and test set according to a preset ratio, and generate vegetation classification labels.

[0032] The model training module is used to build a vegetation classification model based on a convolutional neural network, and to train, validate and test the model using the obtained training set, validation set and test set.

[0033] The vegetation classification module is used to predict vegetation classification by taking the orthophoto and DEM data to be predicted as input and using the trained vegetation classification model.

[0034] According to a third aspect, one embodiment provides an electronic device, the device comprising: a processor and a memory;

[0035] The memory is used to store one or more program instructions;

[0036] The processor is configured to run one or more program instructions to perform the steps of a vegetation classification method based on UAV remote sensing imagery as described in any of the preceding claims.

[0037] This invention provides a vegetation classification method and system based on UAV remote sensing imagery. Based on remote sensing imagery data of a target area acquired using sensors mounted on a UAV, orthophotos and DEM data of the target area are generated. A dataset is constructed using the obtained orthophotos and DEM data, and the dataset is divided into training, validation, and test sets according to a preset ratio, generating vegetation classification labels. A vegetation classification model based on a convolutional neural network is constructed, and the model is trained, validated, and tested using the obtained training, validation, and test sets. The orthophotos and DEM data to be predicted are used as input, and the trained vegetation classification model is used to predict vegetation classification. This invention can achieve vegetation classification based on UAV remote sensing imagery and DEM data. After obtaining the parameters of the convolutional neural network model in the training set using the backpropagation algorithm, the method outputs vegetation classification results through deep learning, using orthophotos and DEM data as input. This method achieves good vegetation classification results. Attached Figure Description

[0038] Figure 1 A flowchart illustrating a vegetation classification method based on UAV remote sensing imagery, provided as an embodiment of the present invention;

[0039] Figure 2 The basic process of vegetation classification using a convolutional neural network-based vegetation classification model is provided in a vegetation classification method based on UAV remote sensing imagery, which is an embodiment of the present invention.

[0040] Figure 3 An improved architecture diagram of the ASPP module in a vegetation classification method based on UAV remote sensing imagery provided in one embodiment of the present invention;

[0041] Figure 4 This is a diagram illustrating the architecture of a vegetation classification model based on a convolutional neural network in a vegetation classification method based on UAV remote sensing imagery, provided in one embodiment of the present invention.

[0042] Figure 5 This is a logical structure diagram of a vegetation classification system based on UAV remote sensing imagery, provided as an embodiment of the present invention. Detailed Implementation

[0043] The present invention will now be described in further detail with reference to specific embodiments and accompanying drawings. Similar elements in different embodiments are referred to by associated similar element reference numerals. In the following embodiments, many details are described to facilitate a better understanding of the invention. However, those skilled in the art will readily recognize that some features may be omitted in different situations, or may be replaced by other elements, materials, or methods. In some cases, certain operations related to the present invention are not shown or described in the specification. This is to avoid obscuring the core parts of the invention with excessive description. For those skilled in the art, detailed description of these related operations is not necessary; they can fully understand the related operations based on the description in the specification and general technical knowledge in the art.

[0044] Furthermore, the features, operations, or characteristics described in the specification can be combined in any suitable manner to form various embodiments. At the same time, the steps or actions in the method description can be rearranged or adjusted in a manner obvious to those skilled in the art. Therefore, the various orders in the specification and drawings are only for the clear description of a particular embodiment and do not imply a necessary order, unless otherwise stated that a particular order must be followed.

[0045] The first embodiment of this invention provides a vegetation classification method based on UAV remote sensing imagery, which will be discussed below in conjunction with... Figure 1 Please provide a detailed explanation.

[0046] like Figure 1 As shown, in step S100, orthophotos and DEM data of the target area are generated based on remote sensing image data of the target area obtained by the sensors on the UAV.

[0047] The above steps specifically include:

[0048] The S110 uses drones and sensors to acquire remote sensing image data of a region in western China. The data includes three bands of visible light: red, green, and blue.

[0049] The S120 uses sensors onboard the UAV to acquire point cloud data and image data of the target area. It uses Inertial Explorer software to perform POS (Position and Orientation System) calculations to obtain EO (Exterior Orientation) data corresponding to the image data. Based on the obtained image data and corresponding EO data, it uses Agisoft Metashape software to generate DEM data and orthophotos of the target area through steps such as aligning photos, optimizing alignment methods, and generating dense point clouds. The DEM data and orthophotos are then standardized.

[0050] like Figure 1 As shown, in step S200, a dataset is constructed using the obtained orthophotos and DEM data. The dataset is divided into a training set, a validation set, and a test set according to a preset ratio, and vegetation classification labels are generated.

[0051] The above steps specifically include:

[0052] S210 uses the sliding window method to crop the orthophoto and DEM data to a size consistent with the model channels, set to 256*256. It adopts the method of ignoring edge prediction to solve the splicing traces that occur when performing regular cropping of the image. During cropping, the edge repetition rate of 10% is used to solve the problem of image edge feature loss.

[0053] S220 dataset contains 3000 pairs of image and DEM data, divided into training, validation, and test sets in an 8:1:1 ratio to ensure representativeness of each set. The training set contains 2400 pairs, the validation set contains 300 pairs, and the test set contains 300 pairs. Data augmentation techniques such as rotation, translation, and cropping are applied to improve the model's robustness and generalization ability.

[0054] like Figure 1 As shown, in step S300, a vegetation classification model based on a convolutional neural network is constructed, and the model is trained, validated, and tested using the obtained training set, validation set, and test set.

[0055] The above steps specifically include:

[0056] S310, create binary and multi-class vector tags. The binary vector tags contain two categories: vegetation and non-vegetation, while the multi-class vector tags contain multiple vegetation types present in this western region.

[0057] S320 uses ArcGIS software to perform pixel-level annotation on the training set data, meaning each pixel has a label, thus obtaining label data. The training data and labels are then paired and used as input data for model training.

[0058] like Figure 1 As shown, in step S400, the orthophoto image to be predicted and the DEM data are used as inputs, and the vegetation classification model obtained by training is used to perform vegetation classification prediction.

[0059] The above steps specifically include:

[0060] S410, Constructing a convolutional neural network for vegetation classification from UAV remote sensing images:

[0061] This embodiment designs a convolutional neural network model suitable for vegetation classification in UAV remote sensing imagery. Based on the DeepLabV3+ architecture, the model modifies its backbone and Atrous Spatial Pyramid Pooling (ASPP) module, and introduces a Convolutional Block Attention Module (CBAM) during the decoding stage. This model can extract fine-grained classification and detailed information from high-resolution UAV imagery in complex backgrounds, improving the accuracy and efficiency of vegetation classification in UAV imagery. Figure 2 This outlines the basic process for using models to classify vegetation.

[0062] Specifically, this invention replaces the DeepLabV3+ feature extraction backbone network Xception with the lightweight MobileNetV3. Compared to MobileNetV2, MobileNetV3 optimizes the network structure and further improves performance, better balancing classification accuracy and computational efficiency.

[0063] This invention also modifies the ASPP module, adjusting the multi-scale receptive field and adding two dilated convolutions with lower dilation rates. Due to the high accuracy of UAV remote sensing images, and the irregular, complex, and varying sizes of vegetation boundaries, 3×3 convolutions with dilation rates of 2 and 4 are added, such as... Figure 3 As shown, this reduces the loss of detailed information, allowing the model to better understand semantic structure and adapt to different tasks;

[0064] In the feature fusion stage of the model decoder, this invention stitches the DEM with the low-level image feature map to obtain an enhanced feature map; and in the model decoding stage, it introduces the attention mechanism CBAM, which allows the model to focus on the task region in a targeted manner, reducing the processing of the entire image and improving processing efficiency and accuracy.

[0065] S420, Model Training:

[0066] After constructing a convolutional neural network suitable for vegetation classification in UAV remote sensing images, the prepared training set data is input into the convolutional neural network model, and the prepared label data is used as the output to train the model. The backpropagation algorithm is used to obtain the model parameters, and the learning rate is set to 0.0001 to obtain the trained convolutional neural network model, which is then saved.

[0067] like Figure 4 As shown, the processing procedure of the convolutional neural network model is as follows: In the convolutional neural network model for vegetation classification of UAV remote sensing images, the orthophoto is first input and enters the backbone network MobileNetV3 in the encoder stage. In order to combine multi-scale information for vegetation classification, the feature map extracted by the backbone network enters two different branches. One branch enters the ASPP module to extract high-level features of the image. In this module, dilated convolutions with dilation rates of 2, 4, 6, 12, and 18 are applied to the input feature map to extract contextual information of different scales, i.e., more abstract information, and then enters the decoder stage. The other branch of MobileNetV3 outputs low-level features including edge information and concatenates them with the input DEM elevation data to obtain an enhanced feature map. The enhanced feature map is then fused with the high-level features output by the ASPP module through the concat function after passing through the attention mechanism CBAM. The model combines information of different levels and scales to improve segmentation accuracy and outputs the final segmentation result through the convolutional layer.

[0068] like Figure 1 As shown, in step S500, the orthophoto image to be predicted and the DEM data are used as inputs, and the vegetation classification model obtained by training is used to perform vegetation classification prediction.

[0069] Corresponding to the above-disclosed vegetation classification method based on UAV remote sensing imagery, this invention also discloses a vegetation classification system based on UAV remote sensing imagery, such as... Figure 5 As shown, it specifically includes:

[0070] The data acquisition module is used to create orthophotos and DEM data of the target area based on remote sensing image data acquired by sensors on the UAV within the target area.

[0071] The dataset construction module is used to construct a dataset using the obtained orthophotos and DEM data, divide the dataset into training set, validation set and test set according to a preset ratio, and generate vegetation classification labels.

[0072] The model training module is used to build a vegetation classification model based on a convolutional neural network, and to train, validate and test the model using the obtained training set, validation set and test set.

[0073] The vegetation classification module is used to predict vegetation classification by taking the orthophoto and DEM data to be predicted as input and using the trained vegetation classification model.

[0074] It should be noted that for a detailed description of the vegetation classification system based on UAV remote sensing imagery provided in the embodiments of the present invention, please refer to the relevant description of the vegetation classification method based on UAV remote sensing imagery provided in the embodiments of the present invention, which will not be repeated here.

[0075] Those skilled in the art will understand that all or part of the functions of the various methods in the above embodiments can be implemented by hardware or by computer programs. When all or part of the functions in the above embodiments are implemented by computer programs, the program can be stored in a computer-readable storage medium, which may include: read-only memory, random access memory, disk, optical disk, hard disk, etc., and the program is executed by a computer to achieve the above functions. For example, the program can be stored in the memory of a device, and when the program in the memory is executed by the processor, all or part of the above functions can be achieved. In addition, when all or part of the functions in the above embodiments are implemented by computer programs, the program can also be stored in a server, another computer, disk, optical disk, flash drive, or external hard drive, etc., and can be downloaded or copied to the memory of a local device, or the system of the local device can be updated. When the program in the memory is executed by the processor, all or part of the functions in the above embodiments can be achieved.

[0076] The above examples illustrate the present invention only to aid in understanding it and are not intended to limit the scope of the invention. Those skilled in the art can make various simple deductions, modifications, or substitutions based on the principles of this invention.

Claims

1. A vegetation classification method based on UAV remote sensing imagery, characterized in that, The method includes: Based on remote sensing image data of the target area obtained by using sensors on a drone, orthophotos and DEM data of the target area are produced. A dataset was constructed using the obtained orthophotos and DEM data. The dataset was divided into training set, validation set and test set according to a preset ratio, and vegetation classification labels were generated. A vegetation classification model based on a convolutional neural network was constructed, and the model was trained, validated, and tested using the obtained training set, validation set, and test set. Using orthophotos and DEM data to be predicted as input, vegetation classification prediction is performed using a trained vegetation classification model.

2. The vegetation classification method based on UAV remote sensing imagery as described in claim 1, characterized in that, Based on remote sensing imagery data of the target area acquired using sensors mounted on a drone, orthophotos and DEM data of the target area are created, specifically including: The remote sensing image data includes three bands of visible light: red, green, and blue. Based on the point cloud data and image data of the target area, POS calculation is performed using IE software to obtain the EO data corresponding to the image data. Based on the obtained image data and corresponding EO data, Agisoft Metashape software is used to generate DEM data and orthophotos of the target area through the steps of aligning photos, optimizing the alignment method, and generating dense point clouds. The DEM data and orthophotos are then standardized.

3. The vegetation classification method based on UAV remote sensing imagery as described in claim 1, characterized in that, A dataset was constructed using the obtained orthophotos and DEM data, specifically including: The orthophoto and DEM data are cropped to the same size as the model channels using the sliding window method; the edge prediction neglect method is used to solve the stitching marks that occur when the image is cropped in a regular manner; and the edge repetition rate with a preset percentage value is used to solve the problem of image edge feature loss during cropping.

4. The vegetation classification method based on UAV remote sensing imagery as described in claim 1, characterized in that, Generate vegetation classification labels, specifically including: Create binary and multi-class vector labels. The binary vector labels include two categories: vegetation and non-vegetation. The multi-class vector labels include multiple vegetation types present in the target area.

5. The vegetation classification method based on UAV remote sensing imagery as described in claim 4, characterized in that, Generate vegetation classification labels, specifically including: The training set is labeled at the pixel level using ArcGIS software, meaning each pixel has a label. The label data is then paired with the training data and used as input data for model training.

6. The vegetation classification method based on UAV remote sensing imagery as described in claim 1, characterized in that, Vegetation classification models based on convolutional neural networks specifically include: The vegetation classification model adopts an improved DeepLabV3+ network model; the DeepLabV3+ feature extraction backbone network Xception is replaced with the lightweight MobileNetV3; the ASPP module is modified by adding two 3×3 dilated convolutions with dilation rates of 2 and 4 respectively; in the feature fusion stage of the model decoder, the DEM is stitched with the low-level image feature map to obtain an enhanced feature map; and the attention mechanism CBAM is introduced in the decoding stage of the model to allow the model to focus on the task region in a targeted manner.

7. The vegetation classification method based on UAV remote sensing imagery as described in claim 6, characterized in that, The processing steps of a vegetation classification model based on convolutional neural networks include: First, input orthophoto data, then enter the backbone network MobileNetV3 in the encoder stage; The feature maps extracted by the backbone network MobileNetV3 enter two different branches. One branch enters the ASPP module to extract high-level features of the image. In the ASPP module, dilated convolutions with dilation rates of 2, 4, 6, 12, and 18 are applied to the input feature maps to extract contextual information at different scales, and then the feature maps enter the decoder stage. The low-level features containing edge information output by another branch of the backbone network MobileNetV3 are concatenated with the input DEM elevation data to obtain an enhanced feature map. The enhanced feature map is then passed through the attention mechanism CBAM and then fused with the high-level features output by the ASPP module through the concat function. Finally, the final segmentation result is output through a convolutional layer.

8. The vegetation classification method based on UAV remote sensing imagery as described in claim 1, characterized in that, Vegetation classification models based on convolutional neural networks specifically include: The processed training set data is input into a vegetation classification model based on a convolutional neural network, and the label set is used as the output to train the network. The backpropagation algorithm is used to obtain the model parameters, resulting in a trained vegetation classification model.

9. A vegetation classification system based on UAV remote sensing imagery, characterized in that, The system includes: The data acquisition module is used to create orthophotos and DEM data of the target area based on remote sensing image data acquired by sensors on the UAV within the target area. The dataset construction module is used to construct a dataset using the obtained orthophotos and DEM data, divide the dataset into training set, validation set and test set according to a preset ratio, and generate vegetation classification labels. The model training module is used to build a vegetation classification model based on a convolutional neural network, and to train, validate and test the model using the obtained training set, validation set and test set. The vegetation classification module is used to predict vegetation classification by taking the orthophoto and DEM data to be predicted as input and using the trained vegetation classification model.

10. An electronic device, characterized in that, The device includes: a processor and a memory; The memory is used to store one or more program instructions; The processor is configured to run one or more program instructions to perform the steps of a vegetation classification method based on UAV remote sensing imagery as described in any one of claims 1 to 8.

Images