Under-forest standing tree instance segmentation method based on improved YOLOv8 model
By improving the YOLOv8 model, replacing key modules and adding DAT modules, the problems of high cost and low accuracy of the vertical wood segmentation method are solved, and efficient under-forest vertical wood detection and segmentation in complex environments are achieved, improving segmentation accuracy and efficiency.
Patent Information
- Application Number
- CN202411137583.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-19
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-08-19
AI Technical Summary
The existing vertical wood instance segmentation method is costly and is not suitable for large-scale segmentation tasks, especially in complex environments, with low segmentation accuracy and efficiency.
Using the improved YOLOv8 model, by replacing the C2f module as ParC module, the SPPF module as SPPF-LSKA module, and adding a DAT module at the end of the backbone network, a down-of-forest wood instance segmentation method based on the improved YOLOv8 model is built to process the forest image Tree trunk feature extraction and segmentation.
It realizes accurate and rapid detection and segmentation of the standing trees under the forest in complex environments, improves the accuracy of detection and segmentation, reduces the calculation amount, and meets the need for low-cost and accurate segmentation of large numbers of tree trunks in forests.
Smart Images

Figure CN119027665A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of image recognition and deep learning, and specifically relates to a method for segmenting forest understory trees based on an improved YOLOv8 model. Background Art
[0002] Forests cover most of the earth's land area and are an important part of the ecosystem. my country has vast land and rich forest resources. As of 2021, the national forest area has reached 220 million hectares, the forest coverage rate has reached 23.04%, and the forest stock has reached 17.56 billion cubic meters. Surveying the growth and density of standing trees in forests is one of the top priorities of forestry management, and forests are subject to many threats, including forest fires, illegal logging, and climate change. Due to the vast forest area and sparse population, there is an urgent need to intelligently and efficiently manage forest resources and improve the accuracy and convenience of forest resource surveys, so as to adapt to the management requirements of digital forestry and precision forestry. Therefore, effective monitoring and management are needed. Standing tree segmentation is one of the key links, which helps to understand the health status and dynamic changes of forests.
[0003] Traditional standing tree segmentation methods generally extract tree features manually, and then use algorithms to further separate the segmentation target from other irrelevant backgrounds. However, the segmentation accuracy of traditional image segmentation methods depends largely on the quality of the manually extracted target features, and only when the target features are more prominent can better results be achieved. In recent years, with the continuous advancement of deep learning, it has been widely used in image processing, target detection, instance segmentation and other fields. In order to improve the image processing and learning capabilities, various image processing models and methods have emerged. Some methods focus on image preprocessing to highlight the characteristics of the trunk itself and thus improve the accuracy of model training. Some methods focus on the performance of the model itself, and improve the accuracy of trunk segmentation more accurately by improving the feature extraction ability of the model. However, some related studies, such as the paper "SHI L, WANG G, MO L, et al. Automatic Segmentation of Standing Trees from Forest Images Based on Deep Learning [J]. Sensors, 2022, 22 (17)", proposed a lightweight network segmentation model SEMD based on neural network to segment the whole tree, but the segmented images have fewer trees and no environmental interference, and the tree features are more prominent. The paper "GRONDIN V, FORTIN JM, POMERLEAU F, et al. Tree detection and diameter estimation based on deep learning [J]. Forestry, 2023, 96 (2): 264-76" proposed a simulated data set containing 43K data, which simulated the real forest environment by computer and then output a large amount of labeled forest data for training. However, the data output by this method did not take into account the complex environmental interference factors, and the trained model did not have good performance for tree species that did not exist in the data set. The paper "LI R, SUN GD, WANG S, et al. Tree trunk detection in urban scenes using a multiscale attention-based deep learning method [J]. Ecological Informatics, 2023, 77." proposed an end-to-end tree trunk detection deep model TrunkNet, which is specifically used to segment complex tree structures. It targets single trees and is not suitable for a large number of tree trunk targets in forest environments.
[0004] In recent years, most of the research that can eliminate most environmental interference and accurately segment tree trunks is based on the three-dimensional point cloud images generated by lidar for tree trunk segmentation. The literature "CHANG L, FAN H, ZHU N, et al. A Two-Stage Approach for Individual Tree Segmentation From TLS Point Clouds [J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2022, 15: 8682-93." uses lidar to construct point cloud images, and then uses semantic segmentation to divide the point cloud into tree points and non-tree points. Then, by combining object detection and post-processing refinement, the instance segmentation module is used to roughly extract trees, filter out understory vegetation, and increase the probability of extracting small trees. The paper "ITAKURA K, HOSOI F. Automatic Tree Detection from Three-Dimensional Images Reconstructed from 360 degrees Spherical Camera Using YOLO v2 [J]. Remote Sensing, 2020, 12 (6)" uses forest images taken with a 360-degree spherical camera and uses the SFM (Structure From Motion) technology to create a 3D point cloud image, which greatly reduces the cost of constructing a 3D point cloud image. The YOLOv8 model is then used to segment the trees in the 3D image and extract information such as the tree's diameter at breast height and tree height. However, these methods are still expensive and are not suitable for tasks such as large-scale forest inventory. Summary of the invention
[0005] The purpose of the present invention is to solve the problem that the existing standing tree instance segmentation method is high in cost and not suitable for large-scale segmentation tasks, and proposes an understory standing tree instance segmentation method based on an improved YOLOv8 model.
[0006] The technical solution adopted by the present invention to solve the above technical problems is: a method for segmenting understory trees based on an improved YOLOv8 model, the method specifically comprising the following steps:
[0007] Step 1: Create a training dataset for segmentation of standing trees under the forest;
[0008] Step 2: Build an improved YOLOv8 model. The improved YOLOv8 model is obtained by improving the original YOLOv8 model. The improvement method is as follows:
[0009] (1) Replace the C2f module of the original YOLOv8 model with the ParC module;
[0010] (2) Replace the SPPF module in the original YOLOv8 model with the SPPF-LSKA module;
[0011] (3) Add the DAT module at the end of the backbone network of the original YOLOv8 model;
[0012] Step 3: Use the training data set to train the improved YOLOv8 model;
[0013] Step 4: Use the trained improved YOLOv8 model to segment the understory trees in the image to be segmented.
[0014] Furthermore, the specific process of step one is:
[0015] Step 11: Obtain an image dataset containing standing trees under the forest;
[0016] Step 1 and 2: Use annotation tools to annotate the standing trees in the acquired image and generate labels for the image data set;
[0017] Step 13: Use the image dataset and corresponding labels to form a training dataset.
[0018] Furthermore, the SPPF-LSKA module includes a first Conv layer, a first MaxPool2d layer, a second MaxPool2d layer, a third MaxPool2d layer, a second Conv layer, a first DW-Conv layer, a second DW-Conv layer, a first DW-D-Conv layer, a second DW-D-Conv layer and a third Conv layer.
[0019] Furthermore, the working process of the SPPF-LSKA module is as follows:
[0020] The input F of the SPPF module in the original YOLOv8 model C It also serves as the input of the first Conv layer and the first DW-Conv layer;
[0021] The output of the first Conv layer then passes through the first MaxPool2d layer, the second MaxPool2d layer, and the third MaxPool2d layer in sequence;
[0022] The output of the first DW-Conv layer then passes through the second DW-Conv layer:
[0023]
[0024] in, is the convolution kernel of the first DW-Conv layer, is the convolution kernel of the second DW-Conv layer, d is the dilation rate, the subscript is the size of the convolution kernel, and * indicates the convolution operation. is the output of the second DW-Conv layer;
[0025] Then pass through the first DW-D-Conv layer and the second DW-D-Conv layer in turn:
[0026]
[0027] in, is the convolution kernel of the first DW-D-Conv layer, is the convolution kernel of the second DW-D-Conv layer. The subscript is the size of the convolution kernel. * indicates the convolution operation. Z C is the output of the second DW-D-Conv layer;
[0028] Z C After the third Conv layer:
[0029] A C =W 1×1 *Z C
[0030] Among them, W 1×1 is the convolution kernel of the third Conv layer. The subscript is the size of the convolution kernel. * indicates the convolution operation. A C is the output of the third Conv layer;
[0031] Calculate A again C and F C The Hadamard product:
[0032]
[0033] in, represents the Hadamard product operation, Indicates A C and F C Hadamard product;
[0034] Will The output of the first Conv layer, the output of the first MaxPool2d layer, the output of the second MaxPool2d layer, and the output of the third MaxPool2d layer are concatenated, and then the concatenated result passes through the second Conv layer, and the output of the second Conv layer is used as the output of the SPPF-LSKA module.
[0035] Furthermore, the working process of the ParC module is:
[0036] Step 1: Embed the vertical basis Generate instance position embedding pe through bilinear interpolation function F(·) V :
[0037]
[0038] in, They are instance position embedding pe V The 0th, 1st, ..., H-1th elements in , the superscript T represents transpose;
[0039] For pe V Copy and pe V Expand to the same dimension as x, and record the expanded matrix as x represents the input feature map of the C2f module in the original YOLOv8 model, the height of x is H, the width of x is W, and the number of channels of x is C;
[0040]
[0041] Among them, W is pe V The number of replications, EV(·) is the horizontal expansion function;
[0042] Will Concatenate with input x: Among them, x p It is the result of splicing;
[0043] For the splicing result x p Perform convolution and get the convolution result y:
[0044]
[0045] Among them, y i,j is the element in the i-th row and j-th column of the convolution result y, is the splicing result x p The element in the (i+t)-th row and the j-th column in , (i+t)modH means calculating the remainder when i+t is divided by H. represent The tth row in ;
[0046] Step 2: Embed the horizontal basis Generate instance position embedding pe through bilinear interpolation function F(·) U :
[0047]
[0048] in, They are instance position embedding peU The 0th, 1st, …, W-1th elements in ;
[0049] For pe U Copy and pe U Expand to the same dimension as the input x, and record the expanded matrix as
[0050]
[0051] Among them, H is the value of pe U is the number of replications, EU(·) is the vertical expansion function;
[0052] Will Concatenate with input x:
[0053] For the splicing result x′ p Perform convolution and get the convolution result y′:
[0054]
[0055] Among them, y i ' ,j is the element in the i-th row and j-th column of the convolution result y′, is the concatenation result x′ p The element in the i-th row and the (j+t)modW column in , (j+t)modW means calculating the remainder of j+t divided by W, represent The t-1th column in ;
[0056] Step 3: Concatenate the convolution result y and the convolution result y′, and use the concatenation result as the output of the ParC-Net module.
[0057] Furthermore, the working process of the DAT module is:
[0058] Step ①: record the feature image input by the DAT module as x′∈R H×W×C , downsample the feature image to generate a reference grid, and record the grid point in the reference grid as p. The coordinate value of the grid point p is (0,0),…,(H G -1,W G -1), and then normalize the coordinate value of each grid point to between [-1, +1];
[0059] Among them, (-1,-1) represents the coordinate value of the lower left corner grid point after normalization, and (+1,+1) represents the coordinate value of the upper right corner grid point after normalization;
[0060] Step ②: Linearly project the input feature image x′ to obtain the coordinates q of the feature points in the image:
[0061] q=x′W q
[0062] Among them, W q is the projection matrix;
[0063] Step 3: Input the coordinate q into the offset network and generate the offset Δp through the offset network;
[0064] Step ④, apply the obtained offset to the grid point to obtain the position of the deformation point;
[0065] Step 5: Sample the deformation point to obtain the feature feature After multi-head attention calculation, the query, key and value output by each attention head are obtained respectively;
[0066]
[0067]
[0068] in, is the key of the mth attention output, is the value output by the mth attention head, and is the parameter matrix of the m-th attention head;
[0069] Then, according to the query, key and value output by each attention head, the features of each attention head output are obtained:
[0070]
[0071] Where σ(·) represents the softmax function, d = C / M is the size of each attention head, M is the total number of attention heads, and z (m) represents the embedding output of the mth attention head, q (m) represents the query output by the mth attention head, Indicated in the continuous relative offset table Interpolation in ;
[0072] Step 6: The output of the DAT module is expressed as Z = [z (1) ,z (2) ,…,z (M) ].
[0073] Furthermore, the downsampling coefficient is r, H G=H / r,W G =W / r.
[0074] Further, the offset satisfies Δp←stanh(Δp), where s is a preset factor.
[0075] Furthermore, the improved YOLOv8 model replaces the FPN unit in the original YOLOv8 model with a feature fusion unit based on a collection and distribution mechanism, that is, the improved YOLOv8 model includes a Backbone unit, a feature fusion unit based on a collection and distribution mechanism, and a Segment unit;
[0076] The Backbone unit includes a first ConvModule module, a second ConvModule module, a first CSPLayer2Conv-Parc module, a third ConvModule module, a second CSPLayer2Conv-Parc module, a fourth ConvModule module, a third CSPLayer2Conv-Parc module, a fifth ConvModule module, a fourth CSPLayer2Conv-Parc module, an SPPF-LSKA module and a DAT module;
[0077] The feature fusion unit based on the collection and distribution mechanism includes a first SimFusion_4in module, a first FIM module, a sixth ConvModule module, a second SimFusion_4in module, a first InjectionMultiSum module, a first CSPLayer2Conv module, a seventh ConvModule module, a third SimFusion_4in module, a second InjectionMultiSum module, a second CSPLayer2Conv module, a first PyramidPoolAgg module, a first TopBasicLayer module, a first AdvPoolFusion module, a third InjectionMultiSum module, a third CSPLayer2Conv module, a second AdvPoolFusion module, a fourth InjectionMultiSum module and a fourth CSPLayer2Conv module;
[0078] And the working process of the improved YOLOv8 model is:
[0079] The input of the improved YOLOv8 model is used as the input of the first ConvModule module, and the output of the first ConvModule module then passes through the second ConvModule module, the first CSPLayer2Conv-Parc module, the third ConvModule module, the second CSPLayer2Conv-Parc module, the fourth ConvModule module, the third CSPLayer2Conv-Parc module, the fifth ConvModule module, the fourth CSPLayer2Conv-Parc module, the SPPF-LSKA module and the DAT module in sequence;
[0080] Use the outputs of the first CSPLayer2Conv-Parc module, the second CSPLayer2Conv-Parc module, the third CSPLayer2Conv-Parc module and the DAT module as the input of the first SimFusion_4in module;
[0081] Use the output of the first SimFusion_4in module as the input of the first FIM module;
[0082] Use the output of the DAT module as the input of the sixth ConvModule module;
[0083] Use the output of the sixth ConvModule module, the output of the second CSPLayer2Conv-Parc module, and the output of the third CSPLayer2Conv-Parc module as the input of the second SimFusion_4in module;
[0084] Use the output of the second SimFusion_4in module and the output of the first FIM module as the input of the first InjectionMultiSum module;
[0085] Use the output of the first InjectionMultiSum module as the input of the first CSPLayer2Conv module;
[0086] Use the output of the third CSPLayer2Conv-Parc module as the input of the seventh ConvModule module;
[0087] Use the output of the seventh ConvModule module, the first CSPLayer2Conv-Parc module, and the second CSPLayer2Conv-Parc module as the input of the third SimFusion_4in module;
[0088] Use the output of the third SimFusion_4in module and the output of the first FIM module as the input of the second InjectionMultiSum module;
[0089] Use the output of the second InjectionMultiSum module as the input of the second CSPLayer2Conv module;
[0090] Use the output of the DAT module, the output of the first CSPLayer2Conv module, and the output of the second CSPLayer2Conv module as the input of the first PyramidPoolAgg module;
[0091] Use the output of the first PyramidPoolAgg module as the input of the first TopBasicLayer module;
[0092] The output of the seventh ConvModule module and the output of the second CSPLayer2Conv module are used as the input of the first AdvPoolFusion module;
[0093] Use the output of the first TopBasicLayer module and the output of the first AdvPoolFusion module as the input of the third InjectionMultiSum module;
[0094] Use the output of the third InjectionMultiSum module as the input of the third CSPLayer2Conv module;
[0095] The output of the third CSPLayer2Conv module and the output of the sixth ConvModule module are used as the input of the second AdvPoolFusion module;
[0096] Use the output of the first TopBasicLayer module and the output of the second AdvPoolFusion module as the input of the fourth InjectionMultiSum module;
[0097] Use the output of the fourth InjectionMultiSum module as the input of the fourth CSPLayer2Conv module;
[0098] The output of the second CSPLayer2Conv module, the output of the third CSPLayer2Conv module, and the output of the fourth CSPLayer2Conv module are used as the input of the Segment unit, and the output of the Segment unit is used as the output of the improved YOLOv8 model.
[0099] Furthermore, the first InjectionMultiSum module includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a Sigmoid activation function layer, a first avgpool layer, a second avgpool layer and a RepConv-blocks layer;
[0100] And the working process of the first InjectionMultiSum module is:
[0101] The input of the first InjectionMultiSum module is used as the input of the first convolutional layer, the second convolutional layer, and the third convolutional layer respectively;
[0102] The output of the second convolutional layer is used as the input of the Sigmoid activation function layer. The output of the Sigmoid activation function layer passes through the first avgpool layer, and then the Hadamard product of the output of the first convolutional layer and the first avgpool layer is calculated to obtain a;
[0103] The output of the third convolutional layer is used as the input of the second avgpool layer, and the Hadamard product of the output of the second avgpool layer and a is calculated to obtain b;
[0104] Finally, b is passed through the RepConv-blocks layer, and the output of the RepConv-blocks layer is used as the output of the first InjectionMultiSum module.
[0105] The beneficial effects of the present invention are:
[0106] The present invention combines improved YOLOv8 deep learning and digital image processing technology to extract the trunk features of standing trees under the forest in forest images, and realizes accurate and rapid detection and segmentation of standing tree trunks in forest images with complex understory environments, providing technical means for the study of trunk instance segmentation in forest environments. By adopting digital image processing technology, it is possible to accurately segment multiple trunk targets simultaneously under complex environmental interference. The segmented objects include trunks blocked by branches and bushes, trunks that are far away in the image, and trunk targets of different sizes. It effectively solves the problem of low segmentation accuracy and low efficiency caused by forest environmental interference, and at the same time meets the actual needs of low-cost and accurate segmentation of a large number of trunks in the forest.
[0107] The detection and segmentation accuracy of the improved YOLOv8 model of the present invention reached 88.6% and 87.7%, respectively, which are 9.8% and 8.8% higher than the original YOLOv8 model. Moreover, the computational complexity of the method of the present invention is 14GFLOPs, which is much lower than the existing method, and the frame rate can reach 86.8 frames / s, which is much higher than the comparison model. At the same time, the average accuracy of detection and segmentation is 1.8% and 3.6% higher than the best result in the traditional two-stage model. The improved YOLOv8 model is less affected by the occlusion between trees and between trees and bushes, and can identify and segment tree trunks of different distances and sizes. It can accurately detect and segment dense tree trunk targets in complex forest environments. BRIEF DESCRIPTION OF THE DRAWINGS
[0108] Figure 1 It is a flow chart of a method for segmenting standing trees under forests based on an improved YOLOv8 model of the present invention;
[0109] Figure 2 This is a comparison chart of the effects of the present invention and the other six algorithms;
[0110] Figure 3 It is a structural diagram of the SPPF module in the YOLOv8 model improved by the SPPF-LSKA module in the present invention;
[0111] Figure 4 This is the workflow diagram of the ParC-Net module;
[0112] Figure 5 This is the flow chart of the variability attention mechanism module;
[0113] Figure 6 This is the structural diagram of the improved YOLOv8 model;
[0114] In the figure, CSPLayer2Conv-Parc represents the addition of Parc after CSPLayer2Conv, that is, the C2f module is replaced by Parc, and then the module composed of CSPLayer2Conv and Parc is used;
[0115] Figure 7 This is the structure diagram of the InjectionMultiSum module. DETAILED DESCRIPTION
[0116] Specific implementation method 1: Combination Figure 1 The present embodiment describes a method for segmenting standing trees under the forest based on an improved YOLOv8 model, and the method specifically includes the following steps:
[0117] Step 1: Create a training dataset for segmentation of standing trees under the forest;
[0118] Step 2: Build an improved YOLOv8 model. The improved YOLOv8 model is obtained by improving the original YOLOv8 model. The improvement method is as follows:
[0119] (1) Replace the C2f module of the original YOLOv8 model with the ParC (Position aware circular convolution) module;
[0120] (2) Replace the SPPF module in the original YOLOv8 model with the SPPF-LSKA module;
[0121] (3) Add a DAT module (Vision Transformer with Deformable Attention) at the end of the backbone network of the original YOLOv8 model;
[0122] The other parts that have not been improved are the same as the original YOLOv8 model;
[0123] Step 3: Use the training data set to train the improved YOLOv8 model;
[0124] Step 4: Use the trained improved YOLOv8 model to segment the understory trees in the image to be segmented.
[0125] Specific implementation method 2: This implementation method is different from the specific implementation method 1 in that the specific process of step 1 is as follows:
[0126] Step 11: Obtain an image dataset containing standing trees under the forest;
[0127] Step 1 and 2: Use annotation tools to annotate the standing trees in the acquired image and generate labels for the image data set;
[0128] Step 13: Use the image dataset and corresponding labels to form a training dataset.
[0129] The other steps and parameters are the same as those in the first embodiment.
[0130] The labeling tools that can be used in the present invention include but are not limited to the labelme tool. Similarly, the verification data set can be obtained by using the method for obtaining the training data set, and the verification data set can be used to verify the trained model.
[0131] Specific implementation method three: Combination Figure 3This embodiment is different from the first or second embodiment in that the SPPF-LSKA module includes a first Conv layer, a first MaxPool2d layer, a second MaxPool2d layer, a third MaxPool2d layer, a second Conv layer, a first DW-Conv layer, a second DW-Conv layer, a first DW-D-Conv layer, a second DW-D-Conv layer and a third Conv layer.
[0132] The other steps and parameters are the same as those in the first or second embodiment.
[0133] Specific implementation method 4: This implementation method is different from any one of the specific implementation methods 1 to 3 in that the working process of the SPPF-LSKA module is as follows:
[0134] The input F of the SPPF module in the original YOLOv8 model C It also serves as the input of the first Conv layer and the first DW-Conv layer;
[0135] The output of the first Conv layer then passes through the first MaxPool2d layer, the second MaxPool2d layer, and the third MaxPool2d layer in sequence;
[0136] The output of the first DW-Conv layer then passes through the second DW-Conv layer:
[0137]
[0138] in, is the convolution kernel of the first DW-Conv layer, is the convolution kernel of the second DW-Conv layer, d is the dilation rate, the subscript is the size of the convolution kernel, and * indicates the convolution operation. is the output of the second DW-Conv layer;
[0139] It captures local spatial information and compensates for the kernel size of subsequent depth-expanded convolutions. The grid effect;
[0140] Then pass through the first DW-D-Conv layer and the second DW-D-Conv layer in turn:
[0141]
[0142] in, is the convolution kernel of the first DW-D-Conv layer, is the convolution kernel of the second DW-D-Conv layer. The subscript is the size of the convolution kernel. * indicates the convolution operation. Z Cis the output of the second DW-D-Conv layer;
[0143] Z C After the third Conv layer:
[0144] A C =W 1×1 *Z C
[0145] Among them, W 1×1 is the convolution kernel of the third Conv layer. The subscript is the size of the convolution kernel. * indicates the convolution operation. A C is the output of the third Conv layer;
[0146] Calculate A again C and F C The Hadamard product:
[0147]
[0148] in, represents the Hadamard product operation, Indicates A C and F C Hadamard product;
[0149] Will The output of the first Conv layer, the output of the first MaxPool2d layer, the output of the second MaxPool2d layer, and the output of the third MaxPool2d layer are concatenated, and then the concatenated result passes through the second Conv layer, and the output of the second Conv layer is used as the output of the SPPF-LSKA module.
[0150] The other steps and parameters are the same as those in Specific Embodiments 1 to 3.
[0151] like Figure 3 As shown, the LSKA in this embodiment decomposes the two-dimensional convolution kernel of the deep convolution into cascaded horizontal and vertical one-dimensional convolution kernels, so that the deep convolution layer does not cause a quadratic increase in calculation and memory usage as the size of the convolution kernel increases, thereby improving the feature extraction capability of the model.
[0152] Specific implementation method five: Combination Figure 4 This embodiment is different from the first to fourth embodiments in that the working process of the ParC module is as follows:
[0153] Step 1: Embed the vertical basis Generate instance position embedding pe through bilinear interpolation function F(·) V :
[0154]
[0155] in, They are instance position embedding pe V The 0th, 1st, ..., H-1th elements in , the superscript T represents transpose; The dimension is C×B×1;
[0156] For pe V Make a copy (that is, the copied vectors are sequentially used as the 2nd column, the 3rd column, ..., the Wth column), and set pe V Expand to the same dimension as x, and record the expanded matrix as x represents the input feature map of the C2f module in the original YOLOv8 model, the height of x is H, the width of x is W, and the number of channels of x is C;
[0157]
[0158] Among them, W is pe V The number of replications, EV(·) is the horizontal expansion function;
[0159] Will Concatenate with input x (the concatenation method is vertical stacking, that is, spliced to the bottom of the input x): Among them, x p It is the result of splicing;
[0160] For the splicing result x p Perform convolution and get the convolution result y:
[0161]
[0162] Among them, y i,j is the element in the i-th row and j-th column of the convolution result y, is the splicing result x p The element in the (i+t)-th row and the j-th column in , (i+t)modH means calculating the remainder when i+t is divided by H. represent The tth row in ;
[0163] Step 2: Embed the horizontal basis Generate instance position embedding pe through bilinear interpolation function F(·) U :
[0164]
[0165] in, They are instance position embedding pe U The 0th, 1st, …, W-1th elements in ; The dimensions are C×1×B;
[0166] For pe U Make a copy (that is, the copied vectors are sequentially used as the 2nd row, the 3rd row, ..., the Hth row), and set pe U Expand to the same dimension as the input x, and record the expanded matrix as
[0167]
[0168] Among them, H is the value of pe U is the number of replications, EU(·) is the vertical expansion function;
[0169] Will Concatenate with input x:
[0170] For the splicing result x′ p Perform convolution and get the convolution result y′:
[0171]
[0172] Among them, y i ' ,j is the element in the i-th row and j-th column of the convolution result y′, is the concatenation result x′ p The element in the i-th row and the (j+t) mod W column, (j+t) mod W means calculating the remainder of j+t divided by W. represent The t-1th column in ;
[0173] Step 3: Concatenate the convolution result y and the convolution result y′, and use the concatenation result as the output of the ParC-Net module.
[0174] The other steps and parameters are the same as those in Specific Embodiments 1 to 4.
[0175] For input with dimension CHW, the position embedding with dimension CB1 is first adjusted to the dimension CH1 suitable for the input through the bilinear interpolation function F to adapt to the input of features of different sizes, and pe VThe horizontal copy is expanded to the CHW dimension, and the expansion result is added to the input feature. In addition, the convolution kernel of the CH1 size that adapts to the input dimension is obtained by interpolation, and the convolution operation is performed on the feature addition result, that is, step 1 is equivalent to completing a global information exchange in the vertical direction. For the input of dimension CHW, the PositionEmbedding of dimension CB1 is first adjusted to the dimension CH1 suitable for the input through the bilinear interpolation function F to adapt to the input of features of different sizes, and the pe is copied H Expand to CHW dimension, and add the expanded result to the input feature. Also, the convolution kernel of CH1 size that adapts to the input dimension is obtained by interpolation, and the convolution operation is performed on the feature addition result, that is, step 2 is equivalent to completing a global information exchange in the horizontal direction. The ParC-Net module integrates the advantages of visual transformer into the convolutional neural network, and has faster reasoning speed and better performance than the model based on visual transformer.
[0176] Specific implementation method six: Combination Figure 5 This embodiment is different from the first to fifth embodiments in that the working process of the DAT module is as follows:
[0177] Step ①: record the feature image input by the DAT module as x′∈R H×W×C , downsample the feature image to generate a reference grid, and record the grid point in the reference grid as p. The coordinate value of the grid point p is (0,0),…,(H G -1,W G -1), and then normalize the coordinate value of each grid point to between [-1, +1];
[0178] Among them, (-1,-1) represents the coordinate value of the lower left corner grid point after normalization, and (+1,+1) represents the coordinate value of the upper right corner grid point after normalization;
[0179] Step ②: Linearly project the input feature image x′ to obtain the coordinates q of the feature points in the image:
[0180] q=x′W q
[0181] Among them, W q is the projection matrix;
[0182] Step 3: Input the coordinate q into the offset network (lightweight subnetwork), and generate the offset Δp = θ through the offset network offset(q) ;
[0183] Step ④, apply the obtained offset to the grid point to obtain the position of the deformation point;
[0184] Step 5: Perform feature sampling (bilinear interpolation) on the deformation point position to obtain the feature feature After multi-head attention calculation, the query, key and value output by each attention head are obtained respectively;
[0185]
[0186]
[0187] in, is the key of the mth attention output, is the value output by the mth attention head, and is the parameter matrix of the m-th attention head;
[0188] Then, according to the query, key and value output by each attention head, the features of each attention head output are obtained:
[0189]
[0190] Where σ(·) represents the softmax function, d = C / M is the size of each attention head, M is the total number of attention heads, and z (m) represents the embedding output of the mth attention head, q (m) represents the query output by the mth attention head, Indicated in the continuous relative offset table Interpolation in ;
[0191] Step 6: The output of the DAT module is expressed as Z = [z (1) ,z (2) ,…,z (M) ].
[0192] The other steps and parameters are the same as those in Specific Implementations 1 to 5.
[0193] By adding a deformable mechanism and narrowing the range of the key corresponding to the query, the interference of irrelevant information can be reduced and the capture of information can be enhanced. This can effectively reduce the interference of irrelevant factors in the environment and improve the ability to detect dense targets.
[0194] Specific implementation method 7: This implementation method is different from any one of the specific implementation methods 1 to 6 in that the downsampling coefficient is r, H G =H / r,W G =W / r.
[0195] The other steps and parameters are the same as those in Specific Embodiments 1 to 6.
[0196] Specific implementation eight: This implementation differs from any one of specific implementations one to seven in that the offset satisfies Δp←stanh(Δp), where “←” indicates that stanh(Δp) approaches Δp, and s is a preset factor (the specific value can be set according to actual conditions).
[0197] The other steps and parameters are the same as those in Specific Embodiments 1 to 7.
[0198] In order to stabilize the training process, this implementation uses a predefined factor to measure the amplitude of Δp to prevent an offset that is too large.
[0199] Specific implementation nine: This implementation is different from any one of specific implementations one to eight in that the improved YOLOv8 model replaces the FPN unit in the original YOLOv8 model with a feature fusion unit based on a collection and distribution mechanism, that is, the improved YOLOv8 model includes a Backbone unit, a feature fusion unit based on a collection and distribution mechanism (i.e., a Neck part), and a Segment unit;
[0200] The Backbone unit includes a first ConvModule module (i.e., the module numbered 0 in the figure), a second ConvModule module (i.e., the module numbered 1 in the figure), a first CSPLayer2Conv-Parc module (i.e., the module numbered 2 in the figure), a third ConvModule module (i.e., the module numbered 3 in the figure), a second CSPLayer2Conv-Parc module (i.e., the module numbered 4 in the figure), a fourth ConvModule module (i.e., the module numbered 5 in the figure), a third CSPLayer2Conv-Parc module (i.e., the module numbered 6 in the figure), a fifth ConvModule module (i.e., the module numbered 7 in the figure), a fourth CSPLayer2Conv-Parc module (i.e., the module numbered 8 in the figure), an SPPF-LSKA module (i.e., the module numbered 9 in the figure) and a DAT module (i.e., the module numbered 10 in the figure);
[0201] The feature fusion unit based on the collection and distribution mechanism includes the first SimFusion_4in module (i.e., the module numbered 11 in the figure), the first FIM module (i.e., the module numbered 12 in the figure), the sixth ConvModule module (i.e., the module numbered 13 in the figure), the second SimFusion_4in module (i.e., the module numbered 14 in the figure), the first InjectionMultiSum module (i.e., the module numbered 15 in the figure), the first CSPLayer2Conv module (i.e., the module numbered 16 in the figure), the seventh ConvModule module (i.e., the module numbered 17 in the figure), the third SimFusion_4in module (i.e., the module numbered 18 in the figure), and the second InjectionMultiSum module (i.e., the module numbered 19 in the figure) , the second CSPLayer2Conv module (i.e., the module numbered 20 in the figure), the first PyramidPoolAgg module (i.e., the module numbered 21 in the figure), the first TopBasicLayer module (i.e., the module numbered 22 in the figure), the first AdvPoolFusion module (i.e., the module numbered 23 in the figure), the third InjectionMultiSum module (i.e., the module numbered 24 in the figure), the third CSPLayer2Conv module (i.e., the module numbered 25 in the figure), the second AdvPoolFusion module (i.e., the module numbered 26 in the figure), the fourth InjectionMultiSum module (i.e., the module numbered 27 in the figure) and the fourth CSPLayer2Conv module (i.e., the module numbered 28 in the figure);
[0202] And the working process of the improved YOLOv8 model is:
[0203] The input of the improved YOLOv8 model is used as the input of the first ConvModule module, and the output of the first ConvModule module then passes through the second ConvModule module, the first CSPLayer2Conv-Parc module, the third ConvModule module, the second CSPLayer2Conv-Parc module, the fourth ConvModule module, the third CSPLayer2Conv-Parc module, the fifth ConvModule module, the fourth CSPLayer2Conv-Parc module, the SPPF-LSKA module and the DAT module in sequence;
[0204] The outputs of the first CSPLayer2Conv-Parc module, the second CSPLayer2Conv-Parc module, the third CSPLayer2Conv-Parc module and the DAT module are used as the inputs of the first SimFusion_4in module;
[0205] Use the output of the first SimFusion_4in module as the input of the first FIM module;
[0206] Use the output of the DAT module as the input of the sixth ConvModule module;
[0207] Use the output of the sixth ConvModule module, the output of the second CSPLayer2Conv-Parc module, and the output of the third CSPLayer2Conv-Parc module as the input of the second SimFusion_4in module;
[0208] Use the output of the second SimFusion_4in module and the output of the first FIM module as the input of the first InjectionMultiSum module;
[0209] Use the output of the first InjectionMultiSum module as the input of the first CSPLayer2Conv module;
[0210] Use the output of the third CSPLayer2Conv-Parc module as the input of the seventh ConvModule module;
[0211] Use the output of the seventh ConvModule module, the first CSPLayer2Conv-Parc module, and the second CSPLayer2Conv-Parc module as the input of the third SimFusion_4in module;
[0212] Use the output of the third SimFusion_4in module and the output of the first FIM module as the input of the second InjectionMultiSum module;
[0213] Use the output of the second InjectionMultiSum module as the input of the second CSPLayer2Conv module;
[0214] Use the output of the DAT module, the output of the first CSPLayer2Conv module, and the output of the second CSPLayer2Conv module as the input of the first PyramidPoolAgg module;
[0215] Use the output of the first PyramidPoolAgg module as the input of the first TopBasicLayer module;
[0216] The output of the seventh ConvModule module and the output of the second CSPLayer2Conv module are used as the input of the first AdvPoolFusion module;
[0217] Use the output of the first TopBasicLayer module and the output of the first AdvPoolFusion module as the input of the third InjectionMultiSum module;
[0218] Use the output of the third InjectionMultiSum module as the input of the third CSPLayer2Conv module;
[0219] The output of the third CSPLayer2Conv module and the output of the sixth ConvModule module are used as the input of the second AdvPoolFusion module;
[0220] Use the output of the first TopBasicLayer module and the output of the second AdvPoolFusion module as the input of the fourth InjectionMultiSum module;
[0221] Use the output of the fourth InjectionMultiSum module as the input of the fourth CSPLayer2Conv module;
[0222] The output of the second CSPLayer2Conv module, the output of the third CSPLayer2Conv module, and the output of the fourth CSPLayer2Conv module are used as the input of the Segment unit, and the output of the Segment unit is used as the output of the improved YOLOv8 model.
[0223] The other steps and parameters are the same as those in Specific Embodiments 1 to 8.
[0224] By globally fusing multi-level features and injecting global information into higher levels to achieve effective information exchange in YOLO, this significantly enhances the information fusion capability of the neck without significantly increasing latency, thereby improving the performance of the model on different object sizes. Figure 5 and Figure 6As shown, the collection and distribution of the process corresponds to three modules. The collection process consists of two steps: first, FAM collects and aligns features from each level; second, IFM fuses the aligned features to generate global information. After the injection module (Inject) obtains the fused global information from the collection process, it distributes the information to each level and injects it using a simple attention operation, thereby enhancing the detection ability of the branch.
[0225] Specific implementation method ten: Combination Figure 7 This embodiment is described. The difference between this embodiment and any one of the specific embodiments 1 to 9 is that the first InjectionMultiSum module includes a first convolution layer, a second convolution layer, a third convolution layer, a Sigmoid activation function layer, a first avgpool layer, a second avgpool layer and a RepConv-blocks layer;
[0226] And the working process of the first InjectionMultiSum module is:
[0227] The input of the first InjectionMultiSum module is used as the input of the first convolutional layer, the second convolutional layer, and the third convolutional layer respectively;
[0228] The output of the second convolutional layer is used as the input of the Sigmoid activation function layer. The output of the Sigmoid activation function layer passes through the first avgpool layer, and then the Hadamard product of the output of the first convolutional layer and the first avgpool layer is calculated to obtain a;
[0229] The output of the third convolutional layer is used as the input of the second avgpool layer, and the Hadamard product of the output of the second avgpool layer and a is calculated to obtain b;
[0230] Finally, b is passed through the RepConv-blocks layer, and the output of the RepConv-blocks layer is used as the output of the first InjectionMultiSum module.
[0231] The other steps and parameters are the same as those in Specific Embodiments 1 to 9.
[0232] The working processes of the first InjectionMultiSum module, the second InjectionMultiSum module, and the third InjectionMultiSum module are the same as the working process of the first InjectionMultiSum module.
[0233] It should be noted that the convolutional layers in the SPPF-LSKA module and the InjectionMultiSum module are numbered separately, that is, the first convolutional layer in the SPPF-LSKA module is not the first convolutional layer in the first InjectionMultiSum module, but only represents the first convolutional layer in the module itself.
[0234] The present invention selected six models for comparative experiments, including two classic segmentation models Mask-R-CNN and Cascade-R-CNN, and four models that were improved on the basis of Mask-R-CNN and Cascade-R-CNN and are specifically for standing tree trunk segmentation: Mask-Swin-T, Mask-DB-Swin-T, Cascade-Swin-S, and Cascade-DB-Swin-S. DB refers to the dual backbone (DB) feature extraction method using CBNetV2, and Swin refers to the use of the self-attention moving window strategy of the Swin transformer to improve the feature extraction efficiency for different objects. In order to ensure the fairness of the experiment, these models were trained and tested using the default hyperparameters in the same environment.
[0235] Figure 2 The sample results of the model improved by the present invention and other CNN models are shown in Figure 1. It can be seen that the model of the present invention is superior to other models in the segmentation accuracy of dense tree trunks, distant tree trunks, severely occluded tree trunks, and tree trunks of different sizes.
[0236] The above calculation examples of the present invention are only used to explain the calculation model and calculation process of the present invention in detail, and are not intended to limit the implementation methods of the present invention. For ordinary technicians in the relevant field, other different forms of changes or modifications can be made based on the above description. It is impossible to list all the implementation methods here. All obvious changes or modifications derived from the technical solution of the present invention are still within the scope of protection of the present invention.
Claims
1. A method for segmenting standing trees under forest based on an improved YOLOv8 model, characterized in that: The method specifically comprises the following steps: Step 1: Create a training dataset for segmentation of standing trees under the forest; Step 2: Build an improved YOLOv8 model. The improved YOLOv8 model is obtained by improving the original YOLOv8 model. The improvement method is as follows: (1) Replace the C2f module of the original YOLOv8 model with the ParC module; (2) Replace the SPPF module in the original YOLOv8 model with the SPPF-LSKA module; (3) Add the DAT module at the end of the backbone network of the original YOLOv8 model; Step 3: Use the training data set to train the improved YOLOv8 model; Step 4: Use the trained improved YOLOv8 model to segment the understory trees in the image to be segmented.
2. The method for segmenting standing trees under forests based on an improved YOLOv8 model according to claim 1, characterized in that: The specific process of step one is: Step 11: Obtain an image dataset containing standing trees under the forest; Step 1 and 2: Use annotation tools to annotate the standing trees in the acquired image and generate labels for the image data set; Step 13: Use the image dataset and corresponding labels to form a training dataset.
3. The method for segmenting standing trees under forest based on the improved YOLOv8 model according to claim 2, characterized in that: The SPPF-LSKA module includes a first Conv layer, a first MaxPool2d layer, a second MaxPool2d layer, a third MaxPool2d layer, a second Conv layer, a first DW-Conv layer, a second DW-Conv layer, a first DW-D-Conv layer, a second DW-D-Conv layer and a third Conv layer.
4. The method for segmenting standing trees under forest based on the improved YOLOv8 model according to claim 3, characterized in that: The working process of the SPPF-LSKA module is: The input F of the SPPF module in the original YOLOv8 model C It also serves as the input of the first Conv layer and the first DW-Conv layer; The output of the first Conv layer then passes through the first MaxPool2d layer, the second MaxPool2d layer, and the third MaxPool2d layer in sequence; The output of the first DW-Conv layer then passes through the second DW-Conv layer: in, is the convolution kernel of the first DW-Conv layer, is the convolution kernel of the second DW-Conv layer, d is the dilation rate, the subscript is the size of the convolution kernel, and * indicates the convolution operation. is the output of the second DW-Conv layer; Then pass through the first DW-D-Conv layer and the second DW-D-Conv layer in turn: in, is the convolution kernel of the first DW-D-Conv layer, is the convolution kernel of the second DW-D-Conv layer. The subscript is the size of the convolution kernel. * indicates the convolution operation. Z C is the output of the second DW-D-Conv layer; Z C After the third Conv layer: AND C =In 1×1 *WITH C Among them, W 1×1 is the convolution kernel of the third Conv layer. The subscript is the size of the convolution kernel. * indicates the convolution operation. A C is the output of the third Conv layer; Calculate A again C and F C The Hadamard product: in, represents the Hadamard product operation, Indicates A C and F C Hadamard product; Will The output of the first Conv layer, the output of the first MaxPool2d layer, the output of the second MaxPool2d layer, and the output of the third MaxPool2d layer are concatenated, and then the concatenated result passes through the second Conv layer, and the output of the second Conv layer is used as the output of the SPPF-LSKA module.
5. The method for segmenting standing trees under forest based on the improved YOLOv8 model according to claim 4, characterized in that: The working process of the ParC module is: Step 1: Embed the vertical basis Generate instance position embedding pe through bilinear interpolation function F(·) V : in, They are instance position embedding pe V The 0th, 1st, ..., H-1th elements in , the superscript T represents transpose; For pe V Copy and pe V Expand to the same dimension as x, and record the expanded matrix as x represents the input feature map of the C2f module in the original YOLOv8 model, the height of x is H, the width of x is W, and the number of channels of x is C; Among them, W is pe V The number of replications, EV(·) is the horizontal expansion function; Will Concatenate with input x: Among them, x p It is the result of splicing; For the splicing result x p Perform convolution and get the convolution result y: Among them, y i,j is the element in the i-th row and j-th column of the convolution result y, is the splicing result x p The element in the (i+t)-th row and the j-th column in , (i+t)modH means calculating the remainder when i+t is divided by H. represent The tth row in ; Step 2: Embed the horizontal basis Generate instance position embedding pe through bilinear interpolation function F(·) U : in, They are instance position embedding pe U The 0th, 1st, …, W-1th elements in ; For pe U Copy and pe U Expand to the same dimension as the input x, and record the expanded matrix as Among them, H is the value of pe U is the number of replications, EU(·) is the vertical expansion function; Will Concatenate with input x: For the splicing result x′ p Perform convolution and get the convolution result y′: Among them, y i ' ,j is the element in the i-th row and j-th column of the convolution result y′, is the concatenation result x′ p The element in the i-th row and the (j+t)modW column in , (j+t)modW means calculating the remainder of j+t divided by W, represent The t-1th column in ; Step 3: Concatenate the convolution result y and the convolution result y′, and use the concatenation result as the output of the ParC-Net module.
6. The method for segmenting standing trees under forests based on an improved YOLOv8 model according to claim 5, characterized in that: The working process of the DAT module is: Step ①: record the feature image input by the DAT module as x′∈R H×W×C , downsample the feature image to generate a reference grid, and record the grid point in the reference grid as p. The coordinate value of the grid point p is (0,0),…,(H G -1,W G -1), and then normalize the coordinate value of each grid point to between [-1, +1]; Among them, (-1,-1) represents the coordinate value of the lower left corner grid point after normalization, and (+1,+1) represents the coordinate value of the upper right corner grid point after normalization; Step ②: Linearly project the input feature image x′ to obtain the coordinates q of the feature points in the image: q=x′W q Among them, W q is the projection matrix; Step 3: Input the coordinate q into the offset network and generate the offset Δp through the offset network; Step ④, apply the obtained offset to the grid point to obtain the position of the deformation point; Step 5: Sample the deformation point to obtain the feature feature After multi-head attention calculation, the query, key and value output by each attention head are obtained respectively; in, is the key of the mth attention output, is the value output by the mth attention head, and is the parameter matrix of the m-th attention head; Then, according to the query, key and value output by each attention head, the features of each attention head output are obtained: Where σ(·) represents the softmax function, d = C / M is the size of each attention head, M is the total number of attention heads, and z ( m ) represents the embedding output of the mth attention head, q ( m ) represents the query output by the mth attention head, Indicated in the continuous relative offset table Interpolation in ; Step 6: The output of the DAT module is represented as Z = [z (1) ,z (2) ,…,z (M) ].
7. The method for segmenting standing trees under forest based on the improved YOLOv8 model according to claim 6, characterized in that: The downsampling coefficient is r, H G =H / r,W G =W / r.
8. The method for segmenting standing trees under forest based on the improved YOLOv8 model according to claim 7, characterized in that: The offset satisfies Δp←stanh(Δp), where s is a preset factor.
9. The method for segmenting standing trees under forests based on an improved YOLOv8 model according to claim 8, characterized in that: The improved YOLOv8 model replaces the FPN unit in the original YOLOv8 model with a feature fusion unit based on a collection and distribution mechanism, that is, the improved YOLOv8 model includes a Backbone unit, a feature fusion unit based on a collection and distribution mechanism, and a Segment unit; The Backbone unit includes a first ConvModule module, a second ConvModule module, a first CSPLayer2Conv-Parc module, a third ConvModule module, a second CSPLayer2Conv-Parc module, a fourth ConvModule module, a third CSPLayer2Conv-Parc module, a fifth ConvModule module, a fourth CSPLayer2Conv-Parc module, an SPPF-LSKA module and a DAT module; The feature fusion unit based on the collection and distribution mechanism includes a first SimFusion_4in module, a first FIM module, a sixth ConvModule module, a second SimFusion_4in module, a first InjectionMultiSum module, a first CSPLayer2Conv module, a seventh ConvModule module, a third SimFusion_4in module, a second InjectionMultiSum module, a second CSPLayer2Conv module, a first PyramidPoolAgg module, a first TopBasicLayer module, a first AdvPoolFusion module, a third InjectionMultiSum module, a third CSPLayer2Conv module, a second AdvPoolFusion module, a fourth InjectionMultiSum module and a fourth CSPLayer2Conv module; And the working process of the improved YOLOv8 model is: The input of the improved YOLOv8 model is used as the input of the first ConvModule module, and the output of the first ConvModule module then passes through the second ConvModule module, the first CSPLayer2Conv-Parc module, the third ConvModule module, the second CSPLayer2Conv-Parc module, the fourth ConvModule module, the third CSPLayer2Conv-Parc module, the fifth ConvModule module, the fourth CSPLayer2Conv-Parc module, the SPPF-LSKA module and the DAT module in sequence; The outputs of the first CSPLayer2Conv-Parc module, the second CSPLayer2Conv-Parc module, the third CSPLayer2Conv-Parc module and the DAT module are used as the inputs of the first SimFusion_4in module; Use the output of the first SimFusion_4in module as the input of the first FIM module; Use the output of the DAT module as the input of the sixth ConvModule module; Use the output of the sixth ConvModule module, the output of the second CSPLayer2Conv-Parc module, and the output of the third CSPLayer2Conv-Parc module as the input of the second SimFusion_4in module; Use the output of the second SimFusion_4in module and the output of the first FIM module as the input of the first InjectionMultiSum module; Use the output of the first InjectionMultiSum module as the input of the first CSPLayer2Conv module; Use the output of the third CSPLayer2Conv-Parc module as the input of the seventh ConvModule module; Use the output of the seventh ConvModule module, the first CSPLayer2Conv-Parc module, and the second CSPLayer2Conv-Parc module as the input of the third SimFusion_4in module; Use the output of the third SimFusion_4in module and the output of the first FIM module as the input of the second InjectionMultiSum module; Use the output of the second InjectionMultiSum module as the input of the second CSPLayer2Conv module; Use the output of the DAT module, the output of the first CSPLayer2Conv module, and the output of the second CSPLayer2Conv module as the input of the first PyramidPoolAgg module; Use the output of the first PyramidPoolAgg module as the input of the first TopBasicLayer module; The output of the seventh ConvModule module and the output of the second CSPLayer2Conv module are used as the input of the first AdvPoolFusion module; Use the output of the first TopBasicLayer module and the output of the first AdvPoolFusion module as the input of the third InjectionMultiSum module; Use the output of the third InjectionMultiSum module as the input of the third CSPLayer2Conv module; The output of the third CSPLayer2Conv module and the output of the sixth ConvModule module are used as the input of the second AdvPoolFusion module; Use the output of the first TopBasicLayer module and the output of the second AdvPoolFusion module as the input of the fourth InjectionMultiSum module; Use the output of the fourth InjectionMultiSum module as the input of the fourth CSPLayer2Conv module; The output of the second CSPLayer2Conv module, the output of the third CSPLayer2Conv module, and the output of the fourth CSPLayer2Conv module are used as the input of the Segment unit, and the output of the Segment unit is used as the output of the improved YOLOv8 model.
10. The method for segmenting standing trees under forests based on an improved YOLOv8 model according to claim 9, characterized in that: The first InjectionMultiSum module includes a first convolution layer, a second convolution layer, a third convolution layer, a Sigmoid activation function layer, a first avgpool layer, a second avgpool layer and a RepConv-blocks layer; And the working process of the first InjectionMultiSum module is: The input of the first InjectionMultiSum module is used as the input of the first convolutional layer, the second convolutional layer, and the third convolutional layer respectively; The output of the second convolutional layer is used as the input of the Sigmoid activation function layer. The output of the Sigmoid activation function layer passes through the first avgpool layer, and then the Hadamard product of the output of the first convolutional layer and the first avgpool layer is calculated to obtain a; The output of the third convolutional layer is used as the input of the second avgpool layer, and the Hadamard product of the output of the second avgpool layer and a is calculated to obtain b; Finally, b is passed through the RepConv-blocks layer, and the output of the RepConv-blocks layer is used as the output of the first InjectionMultiSum module.
Citation Information
Patent Citations
Stumpage image segmentation method based on improved ResNet-UNet network
CN111091580A
Mask R-CNN-based street tree image instance segmentation method
CN112116612A
Mangrove forest individual tree target detection method based on improved YOLOv5
CN113705478A
Image instance segmentation method based on deep learning
CN115131556A
Method for constructing three-dimensional measurement and calculation model of forest stand sample plot by using images
CN116844034A
