A forest scene tree species classification method based on an improved deeplabv3+

Abstract

The application discloses a forest scene tree species classification method based on an improved Deeplabv3+, relates to the field of computer vision and deep learning semantic segmentation, and has the following technical solution: S1, data collection; S2, image segmentation; S3, deep learning sample set construction; S4, network model improvement and training; and S5, tree species classification in a research area.The forest scene tree species classification method based on the improved Deeplabv3+ can be used for low-cost unmanned aerial vehicle forest image that is easy to obtain, can adapt to image information of various wave bands, and reduces manual operation; the network can be self-adaptive to multispectral images, so that the characteristics of forest in an infrared wave band are mined, and compared with visible light images, the network can greatly improve the fine identification performance of forest.

Description

Technical Field

[0001] The present invention relates to the fields of computer vision and deep learning semantic segmentation, and more specifically, to a method for classifying tree species in forest scenes based on an improved DeepLabV3+. Background Art

[0002] Forest surveys are an important part of power engineering surveys. Manual field surveys are time-consuming and labor-intensive, and the accuracy of the forest information obtained is difficult to guarantee. To improve the automation and intelligence of forest surveys, this patent aims to perform refined image classification of trees based on multispectral drone image data for automatic tree identification, achieving a win-win situation for power engineering construction, achieving ecological protection and cost savings, and ensuring safer and more reliable project operation. The main research contents of the project are:

[0003] Based on drone image data of different spectral resolutions, adaptive data extraction is performed. According to the characteristics of various common tree species and special tree species, tree species characteristics are extracted, and appropriate classification algorithms are used to automatically classify and identify tree species.

[0004] Patent application number CN202010434177.8, titled "A Real-time Multi-Category Segmentation Method for Orchard Scenes Based on Improved DeepLab," proposes a multi-category segmentation method for orchard scenes based on an improved DeepLab semantic segmentation model. To enable deployment on an orchard spraying robot, a lightweight MobileNet V3 network replaces the original Xception network to reduce network parameters. The ReLU6 activation function is used in the Atrous Spatial Pyramid Pooling (ASPP) module to reduce accuracy loss on mobile devices. Furthermore, a hybrid dilated convolution (HDC) is used to replace the dilated convolution in the original network. The dilation rates in the ASPP are set to be coprime to reduce the grid effect of the dilated convolution.

[0005] The problem with this patent is that it does not adapt to the data from sensors with different spectral resolutions and requires the input data to be fixed three-band RGB images. Using only RGB images to classify tree species will result in a low recognition rate.

[0006] The patent application number is CN202210390641.7, and its title is: A method and device for tree species classification based on hyperspectral images based on generative adversarial networks; the method provided by the patent includes: obtaining hyperspectral image data; performing image segmentation on the hyperspectral image; inputting the segmented hyperspectral image into a pre-trained Vgg network model for image reconstruction to obtain an optimized hyperspectral image; extracting texture features from the optimized hyperspectral image through the gray-level co-occurrence matrix; performing independent principal component analysis on the optimized hyperspectral image to obtain spectral features of different bands, and selecting the spectral features of the first m bands with more spectral features as the extracted spectral features; inputting the extracted texture features and spectral features into the attention network model to obtain the input The image features with attention are obtained; the image features with attention are combined with the set random noise z that obeys the standard normal distribution as the input of the generator in the generative adversarial network, and the unknown class image features are generated by the generative network; the image features with attention and the unknown class image features are used as the input of the discriminator to obtain the output features of the discriminator after maximization optimization; the output features of the discriminator are returned to the input generator to optimize the generation distribution of the generator, and then the output features of the generator are input into the discriminator, and this step is repeated N times for iterative optimization until the iteration stopping condition is met, and the final discriminator outputs the maximized optimized weighted image features; the weighted image features are used as the input of the classifier to obtain the output hyperspectral image tree species classification results.

[0007] Problems with this patent: Although hyperspectral images can be used to finely classify tree species, the cost of acquiring data is extremely expensive and it is not able to adapt to all data; at the same time, the form based on generative adversarial networks will increase the instability of the network. The form of generative adversarial networks has more stringent requirements on the availability and precision of data, which is more unfavorable for data acquisition. Summary of the Invention

[0008] The purpose of the present invention is to solve the above problems and provide a forest scene tree species classification method based on improved DeepLabV3+.

[0009] 1. The above technical objectives of the present invention are achieved through the following technical solutions: A forest scene tree species classification method based on improved DeepLab v3+, comprising the following steps:

[0010] S1. Data collection: using drones to collect images, obtaining multi-band drone image data, and obtaining forest information maps through field mapping;

[0011] S2. Image segmentation: After processing the multi-band UAV image data in step S1, a UAV orthophoto is obtained. A multi-scale segmentation algorithm is used for the UAV orthophoto, so that the ground objects of different sizes and multi-level spatial information in the UAV orthophoto are fully expressed and described in detail at multiple scales. After the segmentation is completed, VI' is used to separate vegetation objects from non-vegetation objects.

[0012] VI′=(2g-rb)-(1.4rg) (1)

[0013] in,

[0014]

[0015] S3. Constructing a deep learning sample set: labeling samples based on the tree information mapping results after image registration, assigning more detailed tree species categories to vegetation objects, and obtaining a training dataset.

[0016] S4, network model improvement and training;

[0017] S5. Classification of tree species in the study area.

[0018] The present invention is further configured as follows: in step S4, the network model improvement and training includes an adaptive band module, which realizes adaptation to image data of different spectral resolutions by adding a variable convolution kernel, so that it can adapt to spectral band changes, and then perform batch normalization to obtain standard input data.

[0019] The present invention is further configured as follows: in step S4, the network model improvement and training include model fitting, and the model fitting adopts SGD optimizer and cross entropy loss back propagation algorithm to train the sample set in step S3 with Deeplabv3+ semantic segmentation model.

[0020] In summary, the present invention has the following beneficial effects:

[0021] 1. For easily accessible and low-cost drone forest imagery, including RGB three-band visible light imagery and multispectral imagery containing more bands, a data fusion module is proposed that can adapt to image information of various band quantities and reduce manual operations.

[0022] 2. The network can adapt to multispectral images to explore the characteristics of trees in the infrared band. Compared with those based only on visible light images, it can greatly improve the network's fine-grained tree recognition performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is the overall technical roadmap of the embodiment of the present invention;

[0024] Figure 2 This is a comparison of segmentation results at different scales in an embodiment of the present invention;

[0025] Figure 3 Schematic diagram of the physical basis of vegetation remote sensing monitoring in an embodiment of the present invention;

[0026] Figure 4 1. Figure 1 is a diagram of the test area data before and after clipping in an embodiment of the present invention;

[0027] Figure 5 This is a sample area and pixel-level annotation display diagram in an embodiment of the present invention;

[0028] Figure 6 This is a diagram of a multi-source spectral data fusion module in an embodiment of the present invention;

[0029] Figure 7 Schematic diagram of dilated convolution in an embodiment of the present invention;

[0030] Figure 8 It is the ASPP module in the embodiment of the present invention;

[0031] Figure 9 This is a network in which an ASPP module is added in an embodiment of the present invention;

[0032] Figure 10 It is the improved DeepLabv3+ encoding-decoding network structure in the embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to enable those skilled in the art to better understand the present invention, the technical solution of the present invention will be further described in detail below in conjunction with the embodiments of the present invention and the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0034] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments of the present invention can be combined with each other. The present invention will be described in detail below with reference to the embodiments.

[0035] Example:

[0036] like Figures 1-10 As shown, a forest scene tree species classification method based on improved DeepLabv3+ includes the following steps:

[0037] S1. Data collection: using drones to collect images, obtaining multi-band drone image data, and obtaining forest information maps through field mapping;

[0038] S2. Image segmentation: After processing the multi-band UAV image data in step S1, a UAV orthophoto is obtained. A multi-scale segmentation algorithm is used for the UAV orthophoto, so that the ground objects of different sizes and multi-level spatial information in the UAV orthophoto are fully expressed and described in detail at multiple scales. After the segmentation is completed, VI' is used to separate vegetation objects from non-vegetation objects.

[0039] S3. Constructing a deep learning sample set: labeling samples based on the tree information mapping results after image registration, assigning more detailed tree species categories to vegetation objects, and obtaining a training dataset.

[0040] S4, network model improvement and training;

[0041] S5. Classification of tree species in the study area.

[0042] In this embodiment:

[0043] 1 Data preprocessing and sample set construction

[0044] 1.1 Object-oriented image segmentation

[0045] 1.1.1 Pixel-based and object-oriented

[0046] Traditional pixel-based information extraction methods analyze individual pixels, classifying each pixel individually based on the average radiation within each pixel. This method is suitable for low-spatial-resolution images (where the image sampling interval is greater than the object itself) that are rich in spectral information and exhibit significant spectral differences between different objects. Pixel-based information extraction methods focus solely on the spectral characteristics of images. However, as image resolution continues to increase, pixels in images shift from pure pixels to mixed pixels. For high-resolution remote sensing images, a single object often consists of multiple pure pixels. Pixel-based extraction methods are prone to salt-and-pepper noise, hindering the effective utilization of spatial information and making it difficult to meet classification requirements. Furthermore, due to factors such as ambient lighting, the phenomenon of identical objects having different spectra and different objects having the same spectra is common. Consequently, pixel-based information extraction methods inevitably suffer from numerous misclassifications and low classification accuracy.

[0047] Object-oriented image analysis uses image objects, which are collections of pixels containing semantic information, as analysis units. This approach fully considers the connection between objects and their surroundings. Object-oriented image analysis is a region-based image segmentation method. The segmentation principle can be roughly described as a two-step process. The first step is to divide the image into a number of meaningful polygonal regions with uniform internal properties and distinct differences between them. This process is called "image segmentation." The second step is to predict the category of the objects in the region based on specific judgment rules. This process is called "image classification." Object-oriented image analysis fully utilizes object information such as spectrum, texture, shape, shadow, spatial topology, as well as inter-class information related to neighboring objects, parent objects, and child objects. Its accuracy is generally higher than that of pixel-based information extraction methods. The object-oriented concept has greatly improved the interpretation accuracy of high-spatial-resolution image data, effectively meeting the needs of scientific research and engineering applications.

[0048] The image data used in this patent are high-spatial-resolution visible light and multispectral drone images. The structure, shape, and texture features of the ground objects are obvious. For example, in the experimental images of woodland and grassland, the grassland texture is finer than that of the woodland. Obviously, for images whose spectral information is far less rich than spatial feature information, adding features such as texture to assist classification can effectively improve the classification effect. In addition, the vegetation information in the image is complex, and the spectral differences between different types of vegetation are not very obvious. Extracting information based on pixel-based spectral characteristics is obviously unable to meet the needs of tree species information extraction and classification. Therefore, the experiment adopts an object-oriented image analysis method.

[0049] 1.2 Multi-scale Segmentation

[0050] eCogntion's object-oriented segmentation algorithms can be divided into two types. Top-down segmentation, such as chessboard segmentation and quadtree segmentation, divides a parent object into more child objects. Bottom-up segmentation, on the other hand, is the opposite: it continuously merges multiple child objects to form a parent object, such as multi-scale segmentation and spectral difference segmentation. Taking practical considerations into account, the high-resolution drone remote sensing images used in this paper contain a wide variety of tree species and complex hierarchies. However, the segmented objects obtained using chessboard segmentation or quadtree segmentation are square, which does not meet the practical requirements for tree species classification. Segmentation at a single scale also struggles to meet the requirements for extracting image objects at different scales. Therefore, this paper selects a multi-scale segmentation algorithm to fully express and describe the different-sized ground objects and multi-level spatial information in the image at multiple scales.

[0051] Common region-based segmentation methods can be categorized as follows (Table 1). Multiscale segmentation is a region merging algorithm. The specific principle is to use a single pixel as the initial region and minimize local heterogeneity as the criterion for determining similarity. A single pixel (or set of pixels) is merged with adjacent similar pixels (or sets of pixels), increasing the image area. The relationship between the heterogeneity of the merged large image region and a preset threshold is determined. When the heterogeneity exceeds the threshold, the merge is terminated, and multiscale segmentation is completed. The heterogeneity threshold is determined by the scale parameter in the multiscale segmentation parameter settings, which has the greatest impact on the segmentation effect. Furthermore, the segmentation effect is influenced by the combined weights of each band, shape and color homogeneity, and the smoothness and compactness coefficients.

[0052] Table 1 Region-based segmentation methods

[0053]

[0054]

[0055] Among the parameters involved in multi-scale segmentation, segmentation scale is an abstract term. Two image objects are considered similar, or "homogeneous," when their "distance" in a given feature space is close. The segmentation scale defines the maximum allowable heterogeneity of the resulting image objects. A smaller segmentation scale means fewer merges are performed on small regions, resulting in more fragmented image objects. Conversely, a larger segmentation scale means more merges are performed, resulting in larger image objects.

[0056] Heterogeneity is composed of color and shape, with the weights representing the extent to which the shape and color of a feature reflect its characteristics. Shape heterogeneity is further represented by smoothness and compactness, representing the smoothness of the segmented object edges and the overall compactness of the object, respectively. This is generally determined by the overall feature structure of the image.

[0057] Image segmentation is an important step in creating image objects in object-oriented image processing. The quality of its segmentation is closely related to the extraction of ground objects. In this experiment, a scale parameter of 20 was selected with a range of 50 to 110. Other relevant parameters remained unchanged: the weight of the three bands was 1, the shape heterogeneity weight was 0.3, and the compactness index was 0.5. The segmentation results of the comparative experiment are shown in the figure. After comparing the images of the segmentation results at different scales, a segmentation scale of 90 was finally selected. At this time, the image objects were of reasonable size, and independent trees were segmented. At the same time, vegetation with uniform internal properties was divided into blocks of reasonably sized areas. Strip roads (bare land) were segmented completely, and the overall segmentation effect was good, which is suitable for the extraction of ground objects in this experiment.

[0058] 1.3 Vegetation Index

[0059] Vegetation indices are linear or nonlinear combinations of different spectral data from remote sensing images. Chlorophyll in healthy green plant leaves strongly absorbs red light (R), reflects green light (G), and reflects strongly in the near-infrared spectrum (NIR), forming a steep reflectance slope. Based on this physical foundation for vegetation remote sensing monitoring, vegetation indices are generally constructed using the visible-NIR spectrum. Vegetation indices contain over 90% of vegetation information and, under certain conditions, can quantitatively reflect plant growth. They are also closely related to various factors, including atmospheric conditions, biomass, and soil background. Currently, vegetation indices are widely used in vegetation cover assessment, fine-grained vegetation classification, crop yield estimation, and regional biomass inversion. Effective use of vegetation indices in remote sensing image analysis helps enhance the interpretation of remote sensing images, extract rich and detailed vegetation information, facilitate microscopic analysis, and improve the accuracy of vegetation monitoring.

[0060] Common vegetation indices can be divided into three types based on their research and development stages: ① Band combinations based on prior knowledge, such as RVI and GVI, which do not consider the influence of factors such as the atmosphere and soil. ② Indices based on physical knowledge, such as NDVI, SAVI, and PVI, which take into account uncertain influencing factors such as the atmosphere and soil. ③ Indices developed based on hyperspectral remote sensing and thermal infrared remote sensing, such as DVI.

[0061] The vegetation index is affected by multiple factors and has distinct regional characteristics. Different indices have different scopes of application, so comprehensive considerations should be taken into account when selecting a vegetation index. George E. Meyer proposed a vegetation index VI′ (Vegetation index) based on the α-value of the vegetation index. The calculation formula is as follows:

[0062] VI′=(2g-rb)-(1.4rg) (1)

[0063] in,

[0064]

[0065] In this formula, the first term, called the super-green vegetation index (EXG) (see Equation 3), provides an approximately binary intensity image highlighting the plant region of interest. A threshold is then selected for each image set to binarize the vegetation region of interest. Using the super-green vegetation index to create a near-binary image facilitates accurate differentiation between vegetation regions and background (soil and withered plant debris). Furthermore, identifying green plants using a single metric is more difficult when digital images are reddish. This redness can be related to the operation of the digital camera, background lighting, or the inherent redness of the soil and debris. These uncertainties are difficult to uniformly eliminate. To address this issue, Meyer et al. proposed a super-red vegetation index (EXR) (Equation 4). A comparison of the modified vegetation index (VI′ = ExG - ExR) with three parameters, EXG and NDI (see Equation 5), revealed that plant targets extracted using the VI′ quality factor matched manually extracted plant regions best, with an accuracy of 0.88±0.12.

[0066] EXG=2g-rb (3)

[0067] EXR=1.4rg (4)

[0068]

[0069] The image vegetation index is used as a custom feature to participate in the nearest neighbor classification. As a measure of proximity, the vegetation index can effectively integrate the relevant spectral information, better extract vegetation information, reduce noise interference, and improve the accuracy of vegetation classification tasks.

[0070] In sample-based supervised classification, the software will establish a discriminant function based on the spectrum, texture, shape, spatial topology and other information of known classified samples to determine the category of unclassified objects. The accuracy of the classification algorithm is largely related to the number and quality of classified samples. Therefore, when selecting samples, attention should be paid to typicality and representativeness, sufficient number, and uniform distribution throughout the image.

[0071] 1.4 Construction of Deep Learning Sample Set

[0072] Taking the data of a wetland park experimental area as an example, the pixel size of the original orthophoto data of the experimental area is 11008x16640, of which the effective area excluding the black edges is about 60%. The bands are red, green, blue and infrared six bands. They are cropped into 512x512 non-overlapping samples, and 726 basic images are obtained.

[0073] Sample selection should meet the following requirements:

[0074] 1. The sample should meet the requirements of single category classification, and the same sample area should only correspond to one category.

[0075] 2. The boundaries of the sample should be clear and should not contain miscellaneous items (miscellaneous trees).

[0076] 3. The number of sample pixels and the number of plants should be sufficient, and the ratio of the number of samples of different categories should not be greater than 10, that is, scattered tree species cannot be used as samples.

[0077] Based on the above conditions, eight representative vegetation samples were selected, including camphor, ginkgo, bayberry, pine, privet, osmanthus, elaeocarpus, and Heshu. The image area corresponding to the vector was cropped using ArcGIS, and pixel-level annotation was performed after segmentation using ecognition. A total of 26 usable samples were obtained, as shown in the following figure. Figure 5 shown.

[0078] 2. Improve the Deeplabv3+ model

[0079] Deep learning semantic segmentation is an end-to-end image segmentation method in the field of image processing, and "semantics" refers to the understanding of the content contained in the image. Deep learning semantic segmentation extracts image features and semantic information from image data with a sufficient number of annotations, and then recognizes each pixel at the pixel level, divides the pixels into several non-overlapping pixel sets according to semantic units, and assigns category labels to the divided intervals. In this way, each pixel of the output image is labeled with a category, realizing semantic segmentation by category. Deeplab is a typical semantic segmentation algorithm in deep learning. It continues to use the full convolution operation of the traditional fully convolutional neural network, and has made innovative improvements such as dilated convolution and pyramid pooling. In recent years, research on image processing based on the Deeplab network model has been continuously carried out, and the scope of application of the model has been continuously expanded. The Deeplabv3+ model is the result of continuous optimization and expansion of the Deeplab series in recent years. The main features of the improved deeplabv3+ model are as follows:

[0080] 2.1 Multi-source spectral data fusion module

[0081] In order to process multi-spectral data sources, it is necessary to normalize the input data feature information. By adding a variable convolution kernel, it can adapt to image data with different spectral resolutions, so that it can adapt to the changes in spectral bands, perform feature normalization, and retain the information features of different data as much as possible. Its structure is as follows Figure 6 .

[0082] 2.2 Dilated convolution

[0083] A standard convolutional neural network typically performs a two-step operation on an input image: convolution followed by pooling. The pooling operation aims to reduce the input image size while expanding the pixel receptive field. Upsampling is then used to restore the feature map to its original input size for prediction. This process of image size reduction and expansion inevitably leads to information loss, a problem addressed by the dilated convolution algorithm.

[0084] like Figure 7 As shown, dilated convolution expands the receptive field by adding holes. It introduces a hyperparameter: the dilation rate, which represents the number of gaps between convolution kernels. Each pixel in the figure undergoes a 3×3 convolution. (a) is a 3×3-1 dilated convolution. A dilated convolution with a dilation rate of 1 is the same as a standard convolution, with a receptive field of 3×3 for each pixel. (b) corresponds to a 3×3-2 dilated convolution, and (c) is a 3×3-4 dilated convolution. The latter two figures build on (a) by adding holes around each convolution kernel, padding them with zeros, and then performing convolution. In reality, only the nine red points undergo the 3×3 convolution; the remaining points have weights of 0.

[0085] The relationship between the dilated convolution and the actual convolution kernel (kernel size) is shown in formula (6), and the receptive field calculation is shown in formula (7). According to the listed formulas, the parameters of each graph are calculated as shown in Table 2.

[0086] k′=k+(k-1)(d-1) (6)

[0087] in,

[0088] k- original convolution kernel size;

[0089] d - dilation rate of the atrous convolution.

[0090] RF i+1 =RF i +(k′-1)×S i ) (7)

[0091] in,

[0092] RF i+1 -The receptive field of the current layer;

[0093] RF i -The receptive field of the previous layer;

[0094] k′-convolution kernel size;

[0095] S i -The stride product of all previous layers

[0096]

[0097] Table 2 Parameter calculation of dilated convolution with different expansion rates

[0098] Expansion rate (Dilated) convolution kernel Receptive field (a) 3×3 standard convolution 1 3 3×3 (b) 3×3-2 dilated convolution 2 5 7×7 (c) 3×3-4 dilated convolution 4 9 15×15

[0099] As shown in Table 2, applying dilated convolution yields the same 3×3 convolution operation as 5×5 and 9×9 convolutions. During convolution, the number of parameters is equal to the sum of the kernel size and the bias term. Clearly, dilated convolution can exponentially increase the receptive field without adding additional parameters. When a network requires a large receptive field to extract global, higher-level semantic features, but computational resources are limited and it's impossible to increase the kernel size or number of kernels, dilated convolution is a preferred approach. Furthermore, while the kernel size of each layer remains the same (3×3), varying the dilation rate can yield different receptive fields, capturing multi-scale information.

[0100] Dilated convolution expands the receptive field without pooling, allowing each convolution output to contain a larger receptive field. Applying dilated convolution to convolutional neural networks expands the receptive field without losing image information, reducing the spatial dimensionality of the convolutional layer, or adding additional parameters, achieving a balance between accuracy and speed.

[0101] 2.3 Atrous spatial pyramid pooling (ASPP)

[0102] The ASPP module, originally proposed in Deeplabv2, uses dilated convolutions with different sampling rates to parallelize the feature map for a given input. Deeplabv3 adds a Batch Normalization (BN) layer to ASPP to implement batch normalization, preventing vanishing and exploding gradients, speeding up training, and reducing the need for initialization.

[0103] Figure 8 This is a network with the final improved ASPP module added, with an output stride of 16. It includes a 1×1 standard convolution and three 3×3 dilated convolutions with dilation rates of [6, 12, 18] (the number of convolution kernels is 256, including the BN layer). In particular, it also includes a global average pooling layer, such as Figure 8This approach aims to address the convolution degradation problem caused by increasing the dilation rate (in extreme cases, when the dilation rate equals the feature map size), increase image-level features, and capture global image information. Finally, all five modules are merged on the model's final feature map. The resulting image-level features are fed into a 1×1 convolution operation (with 256 kernels, including a batch normalization layer). Bilinear upsampling is then performed to convert the features to the desired spatial dimensions.

[0104] ASPP processes a given input feature map using parallel sampling with dilated convolutions at multiple sampling rates, multi-receptive field convolution, and pooling. This module employs the concept of feature layering, layering and stacking convolution outputs with different dilation rates. This not only mitigates the grid effect caused by dilation, but also enhances the model's ability to segment objects at different scales, obtaining multi-scale information gain and improving the accuracy of convolutional neural networks.

[0105] 2.4 Encoder-Decoder Architecture

[0106] The encoder-decoder architecture utilizes a symmetric network structure for image semantic parsing. The encoder, consisting of operations such as convolution and pooling, encodes pixel image features and semantic information. Downsampling is performed during the encoding process to reduce the resolution of the feature map. Correspondingly, decoding is the inverse of encoding. The decoder, consisting of operations such as deconvolution and up-pooling, parses and restores the output feature map to the original input image resolution.

[0107] The Deeplabv3+ semantic segmentation network model proposes Encoder-Decoder based on Deeplabv3, uses Deeplabv3 as the encoding module, and adds a simple and efficient deep separable convolutional neural network as the decoder module to refine the segmentation results, especially for the segmentation results along the edges of objects, to achieve improvements to DeepLabv3. The encoding-decoding structure is shown in Figure 10 As an encoder, Deeplabv3 uses dilated convolution to extract features calculated by Deep Convolution Neural Networks (DCNN) at any resolution. The encoding process can be described as follows: an image is input, and features are extracted using a deep convolutional neural network with a dilated convolution, resulting in a high-level semantic feature and a low-level semantic feature. The high-level semantic feature is processed by the ASPP module, and the low-level semantic feature is obtained. Figure 10The final feature map is used as the encoder output. During decoding, the encoder output features are bilinearly upsampled by a factor of 4 and concatenated with the corresponding low-level features from the DCNN layer at the same spatial resolution. Before concatenation, a 1×1 convolution is applied to reduce the large number of channels contained in the low-level features. The concatenated result is then subjected to a 3×3 convolution to refine the features. Finally, a simple bilinear upsampling by a factor of 4 is performed to produce the prediction result.

[0108] The output stride of image classification tasks is generally 32 (before global pooling or fully connected layers, "output stride" means the ratio of the spatial resolution of the input and output images). In semantic segmentation, the stride in the last one or two blocks of the encoding module of Deeplabv3+ can be removed, and the dilated convolution can be appropriately applied to reduce the output stride to achieve denser feature extraction. The decoding module takes advantage of the small receptive field of low-level semantic features and is sensitive to target position information. It utilizes low-level features to improve the accuracy of target extraction. The Encoder-Decoder structure of Deeplabv3+ can control the resolution of the encoded features by setting the expansion rate of the dilated convolution operation to balance accuracy and running time. In addition, Deeplabv3+ also applies depthwise separable convolution to the ASPP and decoding modules to build a faster and stronger encoding-decoding network.

[0109] Deeplabv3+ combines the ASPP method with the Encoder-Decoder architecture, using ASPP to capture multi-scale contextual information and a decoding module to reconstruct image-accurate object boundaries. The ASPP network encodes multi-scale contextual information on input features through convolution and pooling operations at multiple void rates and multiple perspectives. Simultaneously, the decoding module gradually recovers spatial information to capture clearer and more detailed object boundaries. In summary, Deeplabv3+ can extract dense image features, significantly increase the receptive field, and achieve a balance between accuracy and speed, making it one of the most mature semantic segmentation networks currently available.

[0110] Based on the above, this patent realizes automatic classification of tree species in drone images based on the improved Deeplabv3+ architecture, achieving high-precision, high-speed and highly automated vegetation classification.

[0111] This specific embodiment is merely an explanation of the present invention and is not intended to limit the present invention. After reading this specification, those skilled in the art may make non-creative modifications to this embodiment as needed. However, as long as such modifications are within the scope of the claims of the present invention, they are protected by patent law.

Claims

1. A forest scene tree species classification method based on improved Deeplabv3+, characterized by: The following steps are involved: S1. Data collection: using drones to collect images, obtaining multi-band drone image data, and obtaining forest information maps through field mapping; S2. Image segmentation: After processing the multi-band UAV image data in step S1, a UAV orthophoto is obtained. A multi-scale segmentation algorithm is used for the UAV orthophoto, so that the ground objects of different sizes and multi-level spatial information in the UAV orthophoto are fully expressed and described in detail at multiple scales. After the segmentation is completed, VI' is used to separate vegetation objects from non-vegetation objects. VI′=(2g-rb)-(1.4rg) (1) in, S3. Constructing a deep learning sample set: labeling samples based on the tree information mapping results after image registration, assigning more detailed tree species categories to vegetation objects, and obtaining a training dataset. S4, network model improvement and training; S5. Classification of tree species in the study area; In step S4, the network model improvement and training includes an adaptive band module, which adapts to image data of different spectral resolutions by adding a variable convolution kernel, so that it can adapt to spectral band changes, and then perform batch normalization to obtain standard input data.

2. The method for classifying tree species in forest scenes based on the improved DeepLab v3+ according to claim 1, characterized in that: In step S4, the network model improvement and training includes model fitting, and the model fitting adopts SGD optimizer and cross entropy loss back propagation algorithm to train the Deeplabv3+ semantic segmentation model on the sample set in step S3.

Images