Unmanned aerial vehicle visible light image individual tree detection method suitable for complex forest environment
Through the improved YOLO-DCAM model, combined with drone visible light imaging and deep learning technology, the problem of insufficient single wood detection performance in complex forest environments is solved, and efficient and accurate single wood detection is achieved, improving detection accuracy and adaptability .
Patent Information
- Application Number
- CN202411052424.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-01
- Publication Date
- 2024-11-08
- Estimated Expiration
- Not applicable · inactive patent
Smart Images

Figure CN118918479A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of forest resource monitoring in forestry science research, and relates to a single tree detection method using visible light images of unmanned aerial vehicles (UAVs) suitable for complex forest environments. Background Art
[0002] Planted forests are an important component of global forest resources, accounting for 7% of the global forest area (about 294 million hectares), carrying multiple ecological regulation service functions such as energy supply, climate regulation, windbreak and sand fixation, and carbon sequestration. Individual trees are the basic units of forest ecosystems, which profoundly affect the system structure, pattern, quality, and function. Accurate perception of individual tree elements (such as individual tree position, density, spatial pattern, etc.) can provide important technical support for forest resource monitoring and management, precise improvement of forest quality, forest resource utilization, protection, and sustainable development. The existing individual tree survey methods mainly include field, satellite, and drone observation platforms. Through active and passive sensors (visible light, multi-spectral, hyperspectral, laser point cloud, etc.), based on the fusion of data texture, spectrum, space and other features and classifiers, individual trees are segmented and identified. Field survey methods can provide detailed data, but they are time-consuming, labor-intensive, and costly, and are limited to observations at the sample plot and forest stand scale. Satellite observation platforms can achieve large-scale observations, but data resolution and accuracy face challenges. They are suitable for scenes such as cities and nurseries where individual trees are sparsely distributed. UAV visible light imaging can efficiently, low-cost, and obtain high-resolution data over a large area, achieving high-precision single tree detection. At the same time, efficient, low-cost, detailed, and accurate large-scale survey methods have become a trend.
[0003] In terms of image-based single tree detection technology, there are mainly two types of methods: rule-driven and data-driven. Rule-driven single tree detection methods, such as local maximum method and watershed segmentation, detect single trees according to certain rules based on data texture, spectrum and other features, but there are limitations such as limited use of deep representation information and the need to readjust parameters according to different scenarios. Data-driven methods can automatically learn to extract complex abstract features, have strong adaptability to different data distributions and scene changes, and can achieve accurate and intelligent detection of single trees, including Faster R-CNN and YOL0 series. Single-stage models represented by the YOL0 model can achieve efficient and accurate detection and become the mainstream solution. At present, the deep learning YOL0v5 target detection model performs well in low-density, uniformly distributed artificial forest scenes, but still faces challenges in high-density, canopy occlusion, and complex background scenes; previous single tree detection research has mainly focused on regular, homogeneous artificial forest environments, lacking consideration of multiple complex artificial forest scenes at the same time. Summary of the invention
[0004] In response to the problems of the prior art, the present invention provides a single tree detection method using UAV visible light images suitable for complex forest environments. Based on UAV visible light images and an improved YOL0v5 model, a single tree precision detection technology for complex forest scenes is created, which achieves efficient, precise and intelligent detection of single trees in a variety of complex scenes including dense, overlapping tree crowns, and confusing backgrounds.
[0005] A single tree detection method using visible light images of unmanned aerial vehicles (UAVs) suitable for complex forest environments comprises the following steps: based on an UAV observation platform and a visible light sensor, collecting visible light images of the upper canopy of a forest in a single tree detection area, performing image processing including image alignment and dense point cloud generation steps to generate a digital orthophoto of the forest, slicing the digital orthophoto data of the forest to generate image blocks of N×M pixels, and using a target detection instance annotation tool to annotate instances of single tree crowns. After all crowns are annotated, a single tree detection data set is constructed. The data set comprises three components: training, validation and testing. Pre-trained weights obtained by training with a public data set are used to transfer pre-trained knowledge of a YOL0-DCAM single tree detection model, and localized correction of the model weights is performed according to the training set data. Verification feedback of the model training performance is performed based on the validation set data, hyperparameters are fine-tuned, and optimal model weights are obtained by training. Image data are input for single tree detection reasoning.
[0006] The advantages of the present invention are: the proposed method of single tree detection of the improved YOLO-DCAM model for complex forest scenes is reasonable and effective. The YOLO-DCAM model integrates deformable convolution in the backbone network, integrates learnable offset variables into the convolution kernel sampling points, realizes the adaptability of crowns of different target shapes and sizes, and enhances the backbone network's ability to extract fine-grained and high-level semantic features; in the neck network, it integrates contextual information of different scales and parallelized convolution kernels learned across space, efficiently captures short-term and long-term dependencies, effectively improves attention to crown target information, and suppresses interference from redundant background information. The present invention effectively enhances the feature extraction capability of the single tree detection model, improves attention to target information, achieves a good balance between model accuracy, size, and detection rate, and effectively improves single tree detection performance in a complex forest environment, with Precision (96.1%), Recall (93.0%), F1-score (94.5%), and AP@0.5 (97.3%). Compared with the most advanced detection models such as YOLOv7, YOLOv8, and RT-DERT-I, the model of the present invention achieves the best detection performance, while effectively balancing the model size and detection rate. Compared with the baseline model YOLOv5, Precision is improved by 2.6%, Recall is improved by 1.6%, F1-score is improved by 2.1%, AP@0.5 is improved by 1.4%, the model size is 14.8MB, and the FPS value is 53. It can achieve lightweight real-time and accurate detection, and all performances meet the needs of grassroots forestry surveys. The present invention can be used as an efficient, accurate, and highly adaptable single tree detection solution for complex forest environments, serving forestry monitoring, management, protection and other businesses, and providing technical support for the high-quality development of modern forestry. BRIEF DESCRIPTION OF THE DRAWINGS
[0007] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. As shown in the figure:
[0008] Figure 1 It is a technical flow chart of the present invention.
[0009] Figure 2 Schematic diagram of the YOLO-DCAM model structure of the present invention.
[0010] Figure 3 Schematic diagram of deformable convolution feature extraction of the present invention.
[0011] Figure 4This is a schematic diagram of the structure of the cross-stage partial module CSP_DCN of the fused deformable convolution of the present invention.
[0012] Figure 5 It is a schematic diagram of the EMA structure of the present invention.
[0013] FIG. 6( a ) is a diagram showing single tree detection results in different scenarios of the present invention.
[0014] FIG6( b ) is a diagram showing the single tree detection results in different scenarios of the present invention.
[0015] FIG6( c ) is a diagram showing the single tree detection results in different scenarios of the present invention.
[0016] FIG6( d ) is a diagram showing the single tree detection results in different scenarios of the present invention.
[0017] FIG7( a ) is one of the comparison histograms of the detection results of the YOLO-DCAM model of the present invention and the benchmark model in three 100 m×100 m cross-regional and cross-time sample plots.
[0018] FIG7( b ) is the second comparison histogram of the detection results of the YOLO-DCAM model of the present invention and the benchmark model in three 100 m×100 m cross-regional and cross-time sample plots.
[0019] Figure 8 This is a diagram of single tree detection results of YOLO-DCAM of the present invention in a 100m×100m high-density mixed forest plot.
[0020] Fig. 9 This is a diagram of single tree detection results of YOLO-DCAM of the present invention in a 100m×100m high-density pure forest plot.
[0021] Fig.10 The YOLO-DCAM of the present invention is in a 100m×100m cross-region high-density pure forest plot (Huangfengqiao Forest Farm). DETAILED DESCRIPTION
[0022] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0023] Example 1: Figure 1 , Figure 2 , Figure 3 , Figure 4 , Figure 5, Figure 6(a), Figure 6(b), Figure 6(c), Figure 6(d), Figure 7(a), Figure 7(b), Figure 8 , Fig. 9 and Fig.10 As shown in the figure, a single tree detection method using visible light images of unmanned aerial vehicles (UAVs) suitable for complex forest environments is presented, which mainly involves forest resource monitoring technology in forest management and intelligent vision technology in computer science.
[0024] A single tree detection method using visible light images from drones, suitable for complex forest environments, can make up for the defects of missed and wrong detection in single tree detection in various complex scenes such as dense, overlapping tree crowns, and confusing backgrounds, and achieve efficient, accurate, and intelligent detection of single trees, providing important support for resource assessment and management, accurate improvement of forest quality, and sustainable forestry. It mainly includes the construction of a deep learning dataset for single tree detection and an improved YOL0-DCAM model for complex forest scenes.
[0025] like Figure 1 As shown in the figure, firstly, based on the UAV observation platform and the visible light sensor, visible light images of the upper canopy of the forest are collected in the single tree detection area, and image processing including image alignment and dense point cloud generation is performed to generate a digital orthophoto of the forest. Secondly, the digital orthophoto data of the forest is sliced to generate image blocks of N×M pixels, and the instance annotation tool of the single tree crown is used to annotate the single tree crown. After all the crowns are annotated, the single tree detection dataset is constructed. The dataset includes three components: training, validation, and testing. Finally, the pre-trained weights obtained by training with the public dataset are used to transfer the pre-trained knowledge of the YOL0-DCAM single tree detection model, and the model weights are locally corrected according to the training set data. Based on the validation set data, the model training performance is verified and feedback is given, and the hyperparameters are fine-tuned to train the optimal model weights, which can be passed into the image data for single tree detection reasoning.
[0026] A single tree detection method using visible light images from unmanned aerial vehicles (UAVs) suitable for complex forest environments includes the following steps:
[0027] Contains the following steps:
[0028] Step 1. Construction of single tree detection deep learning dataset, including the following steps:
[0029] Step 1.1) Data acquisition and processing.
[0030] Step 1.2) Preparation of deep learning dataset.
[0031] Step 2. Single tree detection based on the improved YOLO-DCAM model includes the following steps:
[0032] Step 2.1) Model overview.
[0033] Step 2.2) Enhance the backbone network based on deformable convolution for efficient feature extraction.
[0034] Step 2.3) Single tree detection based on the YOLO-DCAM model.
[0035] in:
[0036] Step 1.1) Data acquisition and processing:
[0037] The visible light image data of drones containing a variety of complex forest scenes were obtained by using visible light sensors carried by drones. The images were processed by Pix4D software, including image alignment, dense point cloud generation, and digital elevation model generation, to finally generate digital orthophotos.
[0038] Step 1.2) Deep learning dataset preparation:
[0039] The digital orthophoto data is sliced according to the M×N pixel size (for example, 640×640) to generate fixed-size slice image blocks to adapt to the hardware memory size and prevent system overload. Based on the sliced image blocks, combined with visual interpretation rules and target detection image annotation software, single tree crowns are annotated. After the tree crowns in all slice images are annotated, the preparation of the single tree detection deep learning dataset is completed.
[0040] Step 2. Improved YOLO-DCAM model for complex forest scenes:
[0041] Step 2.1) Model Overview:
[0042] The underlying framework of the YOLO-DCAM model consists of three parts: the backbone, the neck, and the head. The backbone network is responsible for extracting features from the original data, the neck further refines and enhances the spatial details and semantic information extracted from the backbone, and the head is responsible for making the final prediction of the location and category of the object. In the backbone network, an efficient feature extraction module integrating deformable convolution is proposed; in the neck network, an efficient multi-scale attention module is proposed. The overall YOLO-DCAM model structure is as follows: Figure 2As shown in the figure, the original image data is first passed into the backbone network and then goes through Conv, Conv, CSP, Conv, CSP_DCN, Conv, CSP_DCN, Conv, CSP_DCN, and SPPF modules for feature extraction, and then converted into an abstract high-dimensional feature space. In this process, multi-layer convolution and downsampling operations are used to generate feature maps of different fine-grained levels. At the same time, the CSP_DCN module with deformable convolution is integrated to realize feature extraction modeling for irregular deformed objects. The feature data generated by the backbone network flows into the neck network and then goes through Conv, Up_Sample upsampling, Concat splicing, CSP, MEA efficient multi-scale attention, Conv, Up_Sample upsampling, Concat splicing, CSP, MEA efficient multi-scale attention, Conv, Concat splicing, CSP, MEA efficient multi-scale attention, Conv, Concat splicing, CSP, MEA efficient multi-scale attention, Conv, Concat splicing, CSP, MEA efficient multi-scale attention, Conv, Concat splicing, and CSP modules. The neck network uses the "top-down, bottom-up" feature pyramid technology to fuse feature information of different granularities, that is, the feature maps of different scales generated during the downsampling process of the backbone network are concat-fused with the feature maps of different scales generated during the upsampling process of the neck network to achieve the refined fusion of feature information of multiple scales, so as to improve the detection capability of crown targets of different sizes. The head network contains three output heads, which extract information from convolution operations based on multi-scale feature maps of sizes 19*19, 38*38, and 76*76, respectively, and are used to output the bounding box positions and category probability information of single tree crowns of different sizes.
[0043] Step 2.2) Enhance the backbone network based on deformable convolution for efficient feature extraction:
[0044] Step 2.2.1 Deformable convolution network (DCN):
[0045] Traditional feature extraction methods use convolution kernels of regular size and shape to perform feature extraction operations, so they have strong feature modeling capabilities for regular geometric objects. However, when faced with objects of irregular shapes and sizes, such as tree crowns of different shapes and sizes, their feature extraction potential may be limited. Deformable convolution is an advanced convolution operation that integrates a learnable offset variable into each sampling point within the standard convolution kernel, so that it can adaptively change the current position of the convolution operation, breaking through the limitations of regular grid point sampling in traditional convolution. The feature extractor based on deformable convolution enhances the adaptability to different tree crown target shapes and sizes, and enhances the ability to extract fine-grained and high-level semantic features.
[0046] For a standard convolution operation, the operation is as follows: Assume that the sampling points of the convolution kernel are N = 9, W i is the weight of the i-th sampling point of the convolution kernel on the input feature X mapping, P i is the predefined offset of the i-th sampling point position. The process of performing a standard convolution operation on the input feature X to obtain the output feature Y can be defined as follows:
[0047]
[0048] For a standard deformable convolution, the operation is as follows:
[0049]
[0050] Among them, ΔP i is the learnable position offset of the i-th sample point, which is used to adjust the range of the receptive field, ΔO i is a learnable modulation scalar used to modulate the perceived amplitude of the input feature map. The deformable convolution feature extraction operation, such as Figure 3 As shown in the figure, the deformable convolution network (DCN) has an adaptive receptive field size when performing feature extraction, and can realize the feature modeling capability of irregular complex boundaries or deformable objects. Where N represents the number of sampling points of the deformable convolution. First, by performing a convolution operation on the input feature map, three channels of feature information are generated for each sampling point, of which two channels are used to generate offsets in the horizontal and vertical directions, and the third channel is used to generate amplitude information, and its range is (0, 1). In this way, the offset and amplitude information of each sampling point can be obtained, and then integrated into the specific sampling implementation process, so that the conventional regular sampling operation can be broken through during feature extraction.
[0051] Step 2.2.2 Enhanced backbone network with fused deformable convolution:
[0052] The original backbone network consists of a series of cross-stage partial (CSP) networks, convolution (Conv) blocks and spatial pyramid pool fusion (SPPF) modules, among which the CSP module is the basic component of the backbone network and mainly plays the role of feature extraction. After considering factors such as model efficiency and computational complexity, the present invention chooses to apply DCN to the bottleneck part of the CSP module (referred to as bottleneck_DCN). The specific structure of the CSP module embedded in DCN is as follows Figure 4As shown, first, for the bottleneck module Bottleneck_DCN fused with deformable convolution, the input features are processed by the convolution module Conv operation for dimensionality reduction to reduce the amount of model calculation, and then the deformable convolution DCN operation is performed to learn irregular features. The features processed by the deformable convolution are added to the initial input features through a residual structure to obtain the output features. For the cross-stage partial module CSP_DCN fused with deformable convolution, the input features are sequentially passed to the convolution module Conv, the bottleneck modules Bottleneck_DCN fused with N deformable convolutions, and then based on the idea of residual connection, the input features are spliced with the features processed by BottIeneck_DCN, and the spliced features are output after passing through the convolution module Conv.
[0053] Step 2.2.3: Enhanced neck network integrating efficient multi-scale attention:
[0054] On the feature map, the importance of features at different positions and channels is different. The attention mechanism can realize the adaptive weight allocation of features at different positions and channels, which is conducive to strengthening the focus on more discriminative features while suppressing the interference of background information. Efficient multi-scale attention (EMA) is an efficient attention method. It uses a feature grouping strategy to process input feature data in parallel, thereby accelerating model training and reducing memory consumption and computational complexity. At the same time, it integrates multi-scale parallel subnetworks and cross-space learning methods to capture short-term and long-term dependencies. Its structure is as follows: Figure 5 As shown in the figure, for the input features, they are split into g group features along the channel direction to achieve parallel operation. The group features are processed by average pooling in the X direction, average pooling in the Y direction, and convolution (3×3). The features obtained after average pooling in the X direction and average pooling in the Y direction are spliced, and then convolution and Sigmoid activation are performed in sequence to obtain weight information, which is multiplied with the initial input group features to obtain weighted features Re-weight, and then passed to the GroupNorm, average pooling, Softmax normalization, and Matmul modules for further processing. The third branch performs convolution on the features, and then passes them to the average pooling, Softmax normalization, and Matmul modules for further processing. At the same time, the left and right branches also have cross-feature fusion, such as the features processed by the GroupNorm module are Matmul fused with the features of the right branch, and the features processed by the right convolution are Matmul fused with the features of the left branch. Finally, the features of the left and right branches are added, and the weight information is obtained again after sigmoid activation, and multiplied with the initial input group features to obtain the output result.
[0055] First, for the input feature F∈R C×H×W , divide F into G sub-feature groups F along the channel dimension i =[F1, F2, F3, ..., F G ],F i ∈R C / / G×H×W To achieve parallel processing operations, the grouped sub-features are processed by three parallel branches, two of which use 1D horizontal and vertical global pooling to decompose the channel information into two separate one-dimensional feature encoding processes, and then merge the encoded features along the height direction and further process them through 1×1 convolution. Then, the sigmoid function is applied to limit the range of the feature vector value to (0, 1), and the channel attention maps from two directions in each group are aggregated to adjust the weight of the input group feature map. The third branch uses 3×3 convolution to capture local cross-channel information and integrates rich features of different spatial scales through cross-space learning methods. Finally, the feature maps processed by different branches are connected in series and input into the sigmoid function to generate the final attention weight.
[0056] The EMA module is integrated into the neck to fuse contextual information of different scales, efficiently capture short-term and long-term dependencies, effectively improve the focus on target information, and suppress the interference of redundant background information.
[0057] Example 2: Figure 1 , Figure 2 , Figure 3 , Figure 4 , Figure 5 , Figure 6(a), Figure 6(b), Figure 6(c), Figure 6(d), Figure 7(a), Figure 7(b), Figure 8 , Fig. 9 and Fig.10 As shown in the figure, a single tree detection method using UAV visible light images suitable for complex forest environments is presented.
[0058] Example of single-tree detection of Chinese fir based on visible light imaging of drones at Shanxia Experimental Forest Farm in Jiangxi Province:
[0059] Step 1. Data acquisition:
[0060] Using DJI M210 drone equipped with Zenmuse X7 visible light sensor, under clear and cloudless weather, the flight altitude was set to about 110m, and the forward and side overlap rate was 80%. Visible light image data of the study area was collected, including spectral images of various forest scenes such as low-density pure forest, high-density pure forest, medium-density mixed forest, and high-density mixed forest. All collected images were placed in DJI professional image processing software and pre-processed according to the standardized photogrammetry processing process to generate orthophotos. Then, the drone orthophotos were sliced to adapt to the hardware memory size and prevent system overload. Based on the sliced images, the crown of individual trees was annotated to construct a single tree detection deep learning dataset, and the annotated crowns of all scenes were divided into training, validation, and test data according to 6:2:2. The characteristics of the dataset for single tree detection are shown in Table 1.
[0061] Table 1 Feature analysis of the comprehensive dataset for single tree detection
[0062]
[0063] Step 2. Improve YOLO single tree detection model training:
[0064] The training set data is used to improve the training of the YOLO single tree detection model. During training, the weights obtained from the C0C0 dataset are used as the pre-trained weights of this model to initialize the model. Then the Adam gradient optimizer is used to perform gradient descent calculations during training. The momentum, learning rate, and weight decay rate are set to 0.937, 0.01, and 0.0005, respectively. The batch size is 16, and the epochs are set to 300. The entire deep learning model is built based on the Pytorch framework. The model weight value of the epoch with the best performance on the validation set is used as the final weight for subsequent model testing.
[0065] Step 3. Single tree detection results:
[0066] Step 3.1) Ablation experiment test:
[0067] In order to evaluate the performance of the proposed single tree detection model YOLO-DCAM, the standard YOLOv5s network was selected as the evaluation benchmark, and the performance of the improved model was tested. The experimental results are shown in Table 2.
[0068] The evaluation results of the YOLO-DCAM model are Precision (96.1%), Recall (93.0%), F1-score (94.5%) and AP@0.5 (97.3%). Compared with YOLOv5, Precision is improved by 2.6%, Recall is improved by 1.6%, F1-score is improved by 2.1%, and AP@0.5 is improved by 1.4%. Compared with the baseline model, the model of the present invention pays more attention to the crown area of the Chinese fir tree, and the confidence of the detected crown target is higher, which effectively avoids missed detection and false detection.
[0069] Table 2 Ablation comparison experimental results
[0070]
[0071] Step 3.2) Multi-model comparison test:
[0072] In order to better test the performance of this method, it is compared with the most advanced YOL0v8 and RT-DERT-I in experimental tests. The training and testing of all models are set to the same configuration. Multiple indicators such as precision, recall, F1-score, AP@0.5, model size, and fps are selected as the evaluation criteria for single tree detection performance. The comparison results of multiple detection models are shown in Table 3. The model of the present invention achieves the best detection performance and effectively balances the model size and detection rate.
[0073] Table 3. Multi-model comparison experimental results
[0074]
[0075] Step 3.3) Single tree detection results in complex forest environment:
[0076] The image data of four complex scenes are passed into the YOLO-DCAM model for inference, and the results are shown in Figure 6(a), Figure 6(b), Figure 6(c), and Figure 6(d).
[0077] Figure 6(a), Figure 6(b), Figure 6(c), and Figure 6(d) are the single tree detection results obtained by the YOLO-DCAM model inference in low-density pure forest, high-density pure forest, high-density mixed forest, and medium-density mixed forest, respectively.
[0078] It can be seen from the detection results in the above figure that the single tree detection method proposed in this invention can achieve accurate detection of single trees in various scenarios such as high density, overlapping crowns, and background confusion.
[0079] Step 3.4) Model robustness test:
[0080] Deep learning object detection models sometimes have problems with poor generalization performance and low accuracy when trained in one area and used for object detection in another area. To this end, three 100×100m high-density pure forest and mixed forest test plots across regions and time were selected for robustness experiments. The results are shown in Figures 7(a), 7(b), Figure 8 , Fig. 9 and Fig.10 As shown, test plot 1 is a high-density mixed forest plot in the forest farm at the foot of the mountain. Test plot 2 is a high-density pure forest plot in the forest farm at the foot of the mountain. Test plot 3 is a high-density pure forest plot in the state-owned forest farm of Huangfengqiao, Hunan. Figure 7(a) shows the calculation results of the quantitative evaluation index of the reasoning results of the YOL0-DCAM model in test plot 1, test plot 2, and test plot 3. Figure 7(b) shows the calculation results of the quantitative evaluation index of the reasoning results of the Y0L0v5 benchmark model in test plot 1, test plot 2, and test plot 3. Figure 8 This is the single tree detection result in test plot 1 using the method of the present invention. Fig. 9 This is the single tree detection result of the method of the present invention in test plot 2. Fig.10 This is the single tree detection result of the method of the present invention in test plot 3.
[0081] The results show that this method is highly robust and can achieve accurate detection of single trees under different environmental conditions.
[0082] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A single tree detection method using visible light images from unmanned aerial vehicles (UAVs) suitable for complex forest environments, characterized in that: The method comprises the following steps: based on an unmanned aerial vehicle observation platform and a visible light sensor, visible light images of the upper canopy of the forest are collected in the single tree detection area, and image processing including image alignment and dense point cloud generation steps is performed to generate a digital orthophoto of the forest; the digital orthophoto data of the forest is sliced to generate image blocks of N×M pixels; the instance annotation tool of the single tree crown is used to annotate the instances; after all the crowns are annotated, a single tree detection data set is constructed. The data set comprises three components: training, validation, and testing; the pre-trained weights obtained by training with a public data set are used to transfer the pre-trained knowledge of the YOLO-DCAM single tree detection model; the model weights are locally modified according to the training set data; the model training performance is verified and fed back based on the validation set data; the hyperparameters are fine-tuned; the optimal model weights are obtained by training; and the image data are passed in for single tree detection reasoning.
2. The single tree detection method using visible light images of unmanned aerial vehicles applicable to complex forest environments according to claim 1, characterized in that: Contains the following steps: Step 1. Construction of single tree detection deep learning dataset, including the following steps: Step 1.1) Data acquisition and processing, Step 1.2) Deep learning dataset preparation, Step 2. Single tree detection based on the improved YOLO-DCAM model includes the following steps: Step 2.1) Model overview, Step 2.2) Enhance the backbone network based on deformable convolution for efficient feature extraction. Step 2.3) Single tree detection based on the YOLO-DCAM model.
3. The single tree detection method using visible light images of unmanned aerial vehicles applicable to complex forest environments according to claim 2 is characterized in that: Step 1.1) Data acquisition and processing, including the following steps: The visible light image data of the drone containing various complex forest scenes was obtained by using the visible light sensor carried by the drone. The images were aligned, dense point cloud was generated, and digital elevation model was generated based on the Pix4D software to finally generate digital orthophotos. Step 1.2) deep learning dataset preparation includes the following steps: sliding slice the digital orthophoto data according to the M×N pixel size to generate fixed-size slice image blocks to adapt to the hardware memory size and prevent system overload; based on the sliced image blocks, combined with visual interpretation rules and target detection image annotation software, single tree crowns are annotated; after the crowns in all slice images are annotated, the preparation of the single tree detection deep learning dataset is completed.
4. The single tree detection method using visible light images of unmanned aerial vehicles applicable to complex forest environments according to claim 2, characterized in that: Step 2.1) Model overview, including the following steps: The underlying framework of the YOLO-DCAM model consists of three parts: backbone, neck, and head. The entire model includes a convolutional module (Conv), a cross-stage partial module (CSP), a cross-stage partial module fused with deformable convolution (CSP_DCN), upsampling (Up_Sample), concatenation (Concat), an efficient multi-scale attention module (MEA), a detection head (Detect), and a fast spatial pyramid pooling (SPPF). The backbone network is responsible for extracting features from the original data, the neck further refines and enhances the spatial details and semantic information extracted from the backbone, and the head is responsible for the final prediction of the position and category of the object. In the backbone network, an efficient feature extraction module integrating deformable convolution is proposed; in the neck network, an efficient multi-scale attention module is proposed. In the overall YOLO-DCAM model structure, the original image data is first passed into the backbone network and then passes through Conv, Conv, CSP, Conv, OSP_DCN, Conv, CSP_DCN, Conv, CSP_DCN, SPPF modules for feature extraction, and converts it into an abstract high-dimensional feature space. In this process, different fine-grained feature maps are generated through multi-layer convolution and downsampling operations, and the CSP of deformable convolution is integrated at the same time. _ The DCN module realizes the feature extraction modeling of irregularly deformed objects. The feature data generated by the backbone network flows into the neck network and passes through the Conv, Up_Sample, Concat, CSP, MEA, Conv, Up_Sample, Concat, CSP, MEA, Conv, Concat, CSP, MEA, Conv, Concat, CSP modules in sequence. The "top-down and bottom-up" feature pyramid technology is adopted in the neck network to fuse feature information of different granularities, that is, the feature maps of different scales generated in the downsampling process of the backbone network are concat-joined with the feature maps of different scales generated in the upsampling process of the neck network to realize the refined fusion of feature information of multiple scales and improve the detection ability of crown targets of different sizes. The Head network contains 3 output heads, which extract information of convolution operation based on multi-scale feature maps of 19*19, 38*38, and 76*76 sizes respectively, and are used to output the bounding box position and category probability information of single tree crowns of different sizes.
5. The single tree detection method using visible light images of unmanned aerial vehicles applicable to complex forest environments according to claim 2, characterized in that: Step 2.2) The feature efficient extraction based on deformable convolution enhances the backbone network, including the following steps: Step 2.2.1 deformable convolutional network (DCN): Deformable convolution is an advanced convolution operation that integrates a learnable offset variable into each sampling point in the standard convolution kernel, enabling it to adaptively change the current position of the convolution operation. For a standard convolution operation, the operation is as follows: Assume that the sampling points of the convolution kernel are N = 9, W i is the weight of the i-th sampling point of the convolution kernel on the input feature X mapping, P i is the predefined offset of the i-th sampling point position. The process of performing a standard convolution operation on the input feature X to obtain the output feature Y is defined as follows: For a standard deformable convolution, the operation is as follows: Where ΔP i is the learnable position offset of the i-th sample point, which is used to adjust the range of the receptive field, ΔO i It is a learnable modulation scalar used to modulate the perceived amplitude of the input feature map. The deformable convolution feature extraction operation contains the following steps: Deformable convolution network (DCN) has an adaptive receptive field size when performing feature extraction, and realizes the feature modeling capability of irregular complex boundaries or deformable objects. N represents the number of sampling points of deformable convolution. First, by performing a convolution operation on the input feature map, three channels of feature information are generated for each sampling point, two of which are used to generate horizontal and vertical offsets, and the third channel is used to generate amplitude information, which ranges from (0, 1). Thus, the offset and amplitude information of each sampling point can be obtained, and then integrated into the specific sampling implementation process, so that the conventional regular sampling operation can be broken through during feature extraction. Step 2.2.2 Enhanced backbone network with fused deformable convolution: The original backbone network consists of a series of cross-stage partial CSP networks, convolution blocks Conv and spatial pyramid pool fusion modules SPPF. The CSP module is the basic component of the backbone network and is mainly responsible for feature extraction. After considering the model efficiency and computational complexity factors, DCN is selected to be applied to the bottleneck part of the CSP module. The specific structure of the CSP module embedded in DON contains the following steps: First, for the bottleneck module Bottleneck_DCN fused with deformable convolution, the input features are processed by the convolution module Conv operation for dimensionality reduction to reduce the calculation amount of the model, and then the deformable convolution DCN operation is performed to learn irregular features. The features processed by the deformable convolution are added to the initial input features through a residual structure to obtain the output features. For the cross-stage partial module CSP_DCN fused with deformable convolution, the input features are passed to the convolution module Conv, the bottleneck module Bottleneck fused with N deformable convolutions in turn. _ DCN, then based on the idea of residual connection, the input features are concatenated with the features processed by Bottleneck_DCN, and the concatenated features are output after passing through the convolution module Conv. Step 2.2.3: Enhanced neck network integrating efficient multi-scale attention: For the input features, they are split into g group features along the channel direction to achieve parallel operation. The group features are processed by average pooling in the X direction, average pooling in the Y direction, and convolution 3×3. The features obtained after average pooling in the X direction and average pooling in the Y direction are spliced, and then convolution and Sigmoid activation are performed in sequence to obtain weight information, and multiply it with the initial input group features to obtain the weighted feature Re-weight, and then pass it into the GroupNorm, average pooling, Softmax normalization, and Matmul modules for further processing. The third branch convolves the features and then passes them into the average pooling, Softmax normalization, and Matmul modules for further processing. At the same time, the left and right branches also have cross-feature fusion. The features processed by the GroupNorm module are Matmul fused with the features of the right branch, and the features processed by the right convolution are Matmul fused with the features of the left branch. Finally, the features of the left and right branches are added, and the weight information is obtained again after sigmoid activation, and multiplied with the initial input group features to obtain the output result. First, for the input feature F∈R C×H×W , divide F into G sub-feature groups F along the channel dimension i =[F1, F2, F3, ..., F G ],F i ∈R C / / G×H×W , to achieve parallel processing operations, the grouped sub-features are processed by three parallel branches, two of which use 1D horizontal and vertical global pooling to decompose the channel information into two separate one-dimensional feature encoding processes, and then merge the encoded features along the height direction, and further process them through 1×1 convolution, and then apply the sigmoid function to limit the range of feature vector values to (0, 1), and aggregate the channel attention maps from two directions in each group to adjust the weight of the input group feature map. The third branch uses 3×3 convolution to capture local cross-channel information, and integrates rich features of different spatial scales through cross-space learning methods. Finally, the feature maps processed by different branches are connected in series and input into the sigmoid function to generate the final attention weight.
6. The single tree detection method using visible light images of unmanned aerial vehicles applicable to complex forest environments according to claim 2, characterized in that: Step 2.3) Single tree detection based on the YOLO-DCAM model contains the following steps: The image data to be detected is read, and the size of the input image is adjusted to a specific size as needed, and then passed into the constructed YOLO-DCAM for model inference to obtain the target bounding box, category and confidence information. There are a large number of repeated and low-confidence detection boxes in the inferred single tree detection results. The non-maximum suppression (NMS) algorithm is used to eliminate overlapping, redundant and low-confidence detection boxes.
Citation Information
Patent Citations
Forest tree type and quantity identification method and system based on deep learning
CN115761545A
High body seriola quinqueradiata detection method based on YOLOv8 network structure
CN118279935A
Cited By
Object-oriented change target detection method and system based on improved YOLO
CN119832293A
An object-oriented changing target detection method and system based on improved YOLO
CN119832293B
Mangrove forest seedling survival rate evaluation method and system based on deep learning
CN119919838A
Logistics sorting algorithm based on RCNN
CN120198734A
Photovoltaic station capital construction progress detection method and device for unmanned aerial vehicle inspection and computer readable storage medium
CN120278962A
