Single tree segmentation method based on infrared thermal image and visible light image

Abstract

The invention belongs to the technical field of individual tree segmentation, and provides an individual tree segmentation method based on an infrared thermal image and a visible light image, which comprises the steps of image acquisition, image splicing, feature fusion and model segmentation. According to the method, feature fusion is performed on the visible light image and the infrared thermal image, so that input information is enriched, point cloud data acquisition equipment is successfully replaced, and the cost is reduced; through the single-tree segmentation model pre-trained based on the main curve rate and the watershed algorithm, accurate segmentation of the feature fusion image is realized, and the problems of poor data quality and low large-range segmentation precision of traditional point cloud data segmentation are avoided.

Description

Technical Field

[0001] The invention relates to the technical field of single tree segmentation, and in particular to a single tree segmentation method based on infrared thermal images and visible light images. Background Art

[0002] Accurate tree segmentation can help scientists and forestry managers obtain detailed information about each tree, such as health status, growth rate, and environmental adaptability. These data are essential for pest and disease monitoring, resource management, and forest planning. Through efficient and accurate tree segmentation technology, sustainable management of forest resources can be achieved, forestry productivity can be improved, and the health and stability of the ecosystem can be ensured. In the past, tree segmentation tasks were basically manual sampling and marking after demarcating the sample plot. This method is time-consuming and labor-intensive, inefficient, and cannot fully cover the area in charge. In recent years, due to the vigorous development of the drone industry, many methods have been proposed in the field to use drone-borne radar to achieve tree segmentation tasks. At present, the algorithms for tree extraction can be roughly divided into two ideas: detection methods based on canopy height models and detection methods based on point clouds. The mainstream algorithms are region growing methods and watershed methods. In existing technologies, data acquisition is generally divided into two categories. One is to directly use lidar to extract point cloud data in the target area, and the other is to use high-resolution visible light images to calculate point cloud data.

[0003] Existing technologies are extremely dependent on point cloud data, which results in: the collection of point cloud data generally requires large-scale drone equipment and supporting lidar, which is costly; the point cloud data calculated by using oblique photography technology to obtain visible light images and subsequent point cloud reconstruction has quality problems. Summary of the invention

[0004] In order to overcome the shortcomings of the prior art, the purpose of the present invention is to provide a single tree segmentation method based on infrared thermal images and visible light images. By fusing the features of visible light images and infrared thermal images, the input information is enriched, the point cloud data acquisition equipment is replaced, and the cost is reduced. Through the single tree segmentation model pre-trained based on the principal curve rate and watershed algorithm, the accurate segmentation of the feature fused image is achieved, avoiding the problems of poor data quality and low large-scale segmentation accuracy of traditional point cloud data segmentation.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] A single tree segmentation method based on infrared thermal images and visible light images, comprising:

[0007] Using a drone equipped with both a visible light camera and a multispectral camera to collect images of the target identification area, obtaining an original visible light image set and an original thermal imaging image set;

[0008] Separately stitching the original visible light image set and the original thermal imaging image set to obtain an overall visible light image and an overall thermal imaging image;

[0009] Extracting and fusing features of the overall visible light image and the overall thermal imaging image to obtain a feature fused image;

[0010] The feature fusion image is input into a single tree segmentation model pre-trained based on principal curve rate and watershed algorithm to obtain a target segmentation image.

[0011] Preferably, extracting and fusing the features of the overall visible light image and the overall thermal imaging image to obtain a feature fused image comprises:

[0012] Extracting the entire visible light image using a preset visible light feature extraction network to obtain a visible light feature matrix;

[0013] Extracting the overall thermal imaging image using a preset thermal imaging feature extraction network to obtain a thermal imaging feature matrix;

[0014] Splicing the visible light feature matrix and the thermal imaging feature matrix to obtain a multi-modal splicing feature;

[0015] Performing a first global average pooling and a first maximum pooling in a channel domain on the multimodal splicing features respectively to obtain a global channel description and a maximum channel description;

[0016] Inputting the global channel description and the maximum channel description into a basic neural network calculation to obtain a channel weight coefficient; the basic neural network includes: a Relu activation function, a Sigmoid activation function, and at least two neural network layers;

[0017] Multiplying the channel weight coefficient by the multimodal concatenation feature to obtain a channel-corrected multimodal feature;

[0018] Performing a second global average pooling and a second maximum pooling in the spatial domain on the multimodal features of the channel correction respectively to obtain a global spatial description and a maximum spatial description;

[0019] Inputting the global spatial description and the maximum spatial description into a basic neural network calculation to obtain a spatial weight coefficient;

[0020] Multiplying the channel-corrected multimodal feature by the spatial weight coefficient to obtain a spatially corrected multimodal feature;

[0021] After performing a third global average pooling and Sigmoid calculation on the spatially corrected multimodal features, a comprehensive image feature is obtained;

[0022] The overall visible light image is corrected according to the comprehensive image characteristics to obtain the feature fusion image.

[0023] Preferably, the construction process of the single tree segmentation model includes:

[0024] Convert the pre-collected forest image set into grayscale format to obtain a grayscale atlas;

[0025] Extracting the grayscale value of each pixel in the grayscale atlas, and defining the grayscale value as the height, to obtain a three-dimensional point distribution set;

[0026] Using a radial basis function difference method to fit the three-dimensional point distribution set into a surface, to obtain a grayscale surface set;

[0027] Calculating the principal curve ratio of the grayscale surface set, and obtaining the grayscale surface edge image set according to the principal curve ratio;

[0028] Performing a primary segmentation on the grayscale atlas using a watershed algorithm to obtain a primary segmentation image set;

[0029] The forest image set, the initial segmentation image set and the grayscale surface edge image set are used as inputs of a preset convolutional neural network, and the manually segmented forest single tree segmentation image set is used as the output of the convolutional neural network. The convolutional neural network is trained to obtain the trained single tree segmentation model.

[0030] Preferably, the tool used for stitching the original visible light image set and the original thermal imaging image set is: Stitcher library file.

[0031] Preferably, the acquisition parameter settings of the UAV include: an altitude range of 60 to 80 meters, a heading overlap of 60%, and a lateral overlap of 60%.

[0032] Preferably, calculating the principal curve ratio of the grayscale surface set and obtaining the grayscale surface edge image set according to the principal curve ratio comprises:

[0033] Randomly select a grayscale surface image according to the grayscale surface set;

[0034] Constructing a Hessian matrix according to the grayscale surface image;

[0035] Calculating the maximum principal curvature and the minimum principal curvature of each pixel in the grayscale surface image according to the Hessian matrix;

[0036] The maximum principal curvature and the minimum principal curvature are screened according to a preset edge point determination formula to obtain an original edge point set; the edge point determination formula is: EP is the original edge point set; k1 is the maximum principal curvature; k2 is the minimum principal curvature; σ1 is the adaptive first threshold; σ2 is the adaptive second threshold;

[0037] The original edge point set is denoised using the noise reduction formula of Yusa to obtain the main edge point set; the noise reduction formula is: EP′ is the main edge point set; p i is the number of original edge points within the 8 nearest pixels of the ith original edge point in the original edge point set; i is the length of the curve formed by the edge points where the ith original edge point in the original edge point set is located; S i is the area of ​​the region formed by the edge points where the ith original edge point in the original edge point set is located; η is the adaptive length threshold; δ is the adaptive area threshold;

[0038] Matching the main edge point set to the grayscale curved surface image to obtain a curved surface image with fused edge points;

[0039] All images in the grayscale surface set are traversed to obtain surface images of several fused edge points, and the surface images of all fused edge points are integrated to obtain the grayscale surface edge image set.

[0040] Preferably, the grayscale atlas is initially segmented using a watershed algorithm to obtain an initial segmented image set, including:

[0041] Denoising any grayscale image in the grayscale atlas to obtain a denoised image;

[0042] According to the preset single image contour number, the findContours function is used to search the contours of the denoised image to obtain a plurality of contour distributions;

[0043] Using the contour distribution, the images in the forest image set corresponding to the grayscale image are segmented to obtain a plurality of contour segmented images;

[0044] All images in the grayscale atlas are traversed and integrated to obtain the initial segmentation image set.

[0045] Preferably, the forest image set, the initial segmentation image set and the gray surface edge image set are used as inputs of a preset convolutional neural network, and the manually segmented forest single tree segmentation image set is used as the output of the convolutional neural network. The convolutional neural network is trained to obtain the trained single tree segmentation model, including:

[0046] Input the forest image set and the grayscale surface edge image set into a 1×1 convolution respectively to extract local detail features, so as to obtain a first feature representation and a second feature representation;

[0047] Inputting the first feature representation and the second feature representation into a BN layer for feature normalization to obtain a first normalized feature representation and a second normalized feature representation;

[0048] fusing the first normalized feature representation and the second normalized feature representation to obtain a fused feature representation;

[0049] Inputting the fused feature representation into a preset deep learning network for prediction to obtain a predicted segmented image;

[0050] According to the forest farm single tree segmentation image set, a preset loss function is used to calculate the loss value of the predicted segmentation image; the calculation formula of the loss function is: Loss b is the loss value of the b-th iteration; is the predicted area of ​​the a-th single tree segmentation region in the b-th iteration; S ab is the overlap area between the predicted area and the actual area of ​​the a-th single tree segmentation area in the b-th iteration; m is the total number of actual single tree segmentation areas;

[0051] Based on the minimum gradient descent method, the parameters of the deep learning network, the edge point determination formula and the noise reduction formula are optimized according to the loss value;

[0052] The deep learning network, the edge point determination formula and the noise reduction formula are iteratively trained to obtain the trained single tree segmentation model.

[0053] The present invention discloses the following technical effects:

[0054] The present invention provides a single tree segmentation method based on infrared thermal images and visible light images. By fusing the features of visible light images and infrared thermal images, the problem of high cost of traditional point cloud data acquisition equipment is solved, and the enrichment of input information and the replacement of point cloud data acquisition equipment are achieved. By using a single tree segmentation model pre-trained based on the principal curve rate and watershed algorithm, the problems of poor data quality and low large-range segmentation accuracy of traditional point cloud data segmentation are solved, and accurate segmentation of feature fused images is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0056] Figure 1 A schematic diagram of a single tree segmentation process based on infrared thermal images and visible light images provided in an embodiment of the present invention;

[0057] Figure 2 A schematic diagram of a feature fusion image acquisition process provided by an embodiment of the present invention;

[0058] Figure 3 A schematic diagram of the process of constructing a single wood segmentation model provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] The purpose of the present invention is to provide a single tree segmentation method based on infrared thermal images and visible light images. By fusing the features of visible light images and infrared thermal images, the input information is enriched, the point cloud data acquisition equipment is replaced, and the cost is reduced. Through the single tree segmentation model pre-trained based on the principal curve rate and watershed algorithm, the accurate segmentation of the feature fused image is achieved, avoiding the problems of poor data quality and low large-scale segmentation accuracy of traditional point cloud data segmentation.

[0061] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0062] Figure 1 A schematic diagram of a single tree segmentation process based on infrared thermal images and visible light images provided in an embodiment of the present invention is shown in FIG. Figure 1 As shown, the present invention provides a single tree segmentation method based on infrared thermal images and visible light images, comprising:

[0063] Step 100: using a drone equipped with both a visible light camera and a multispectral camera to collect images of the target recognition area, and obtaining an original visible light image set and an original thermal imaging image set;

[0064] Step 200: respectively stitching the original visible light image set and the original thermal imaging image set to obtain an overall visible light image and an overall thermal imaging image;

[0065] Step 300: extracting and fusing features of the overall visible light image and the overall thermal imaging image to obtain a feature fused image;

[0066] Step 400: input the feature fusion image into a single tree segmentation model pre-trained based on principal curve rate and watershed algorithm to obtain a target segmentation image.

[0067] Specifically, Figure 2 As shown, the step 300 specifically includes:

[0068] Step 301: extracting the entire visible light image using a preset visible light feature extraction network to obtain a visible light feature matrix;

[0069] Step 302: extracting the overall thermal imaging image using a preset thermal imaging feature extraction network to obtain a thermal imaging feature matrix;

[0070] Step 303: splicing the visible light feature matrix and the thermal imaging feature matrix to obtain a multi-modal splicing feature;

[0071] Step 304: performing a first global average pooling and a first maximum pooling in a channel domain on the multimodal splicing features respectively to obtain a global channel description and a maximum channel description;

[0072] Step 305: input the global channel description and the maximum channel description into a basic neural network calculation to obtain a channel weight coefficient; the basic neural network includes: a Relu activation function, a Sigmoid activation function, and at least two neural network layers;

[0073] Step 306: multiplying the channel weight coefficient by the multimodal splicing feature to obtain a channel-corrected multimodal feature;

[0074] Step 307: performing a second global average pooling and a second maximum pooling in the spatial domain on the channel-corrected multimodal features respectively to obtain a global spatial description and a maximum spatial description;

[0075] Step 308: Input the global space description and the maximum space description into a basic neural network calculation to obtain a space weight coefficient;

[0076] Step 309: multiplying the channel-corrected multimodal feature by the spatial weight coefficient to obtain a spatially corrected multimodal feature;

[0077] Step 310: performing a third global average pooling and Sigmoid calculation on the spatially corrected multimodal features to obtain a comprehensive image feature;

[0078] Step 311: Correct the overall visible light image according to the comprehensive image characteristics to obtain the feature fusion image.

[0079] Furthermore, if Figure 3 As shown, the construction process of the single tree segmentation model in step 400 includes:

[0080] Step 401: converting the pre-collected forest image set into a grayscale format to obtain a grayscale atlas;

[0081] Step 402: extracting the grayscale value of each pixel in the grayscale atlas, and defining the grayscale value as the height, to obtain a three-dimensional point distribution set;

[0082] Step 403: fitting the three-dimensional point distribution set into a surface using a radial basis function difference method to obtain a grayscale surface set;

[0083] Step 404: Calculate the principal curve ratio of the gray surface set, and obtain the gray surface edge image set according to the principal curve ratio;

[0084] Step 405: using a watershed algorithm to perform a primary segmentation on the grayscale atlas to obtain a primary segmentation image set;

[0085] Step 406: Use the forest image set, the initial segmentation image set and the grayscale surface edge image set as inputs of a preset convolutional neural network, use the manually segmented forest single tree segmentation image set as outputs of the convolutional neural network, train the convolutional neural network, and obtain the trained single tree segmentation model.

[0086] Optionally, a tool used for stitching the original visible light image set and the original thermal imaging image set is: a Stitcher library file.

[0087] Preferably, the acquisition parameter settings of the UAV include: an altitude range of 60 to 80 meters, a heading overlap of 60%, and a lateral overlap of 60%.

[0088] Specifically, calculating the principal curve ratio of the grayscale surface set, and obtaining the grayscale surface edge image set according to the principal curve ratio, includes:

[0089] Randomly select a grayscale surface image according to the grayscale surface set;

[0090] Constructing a Hessian matrix according to the grayscale surface image;

[0091] Calculating the maximum principal curvature and the minimum principal curvature of each pixel in the grayscale surface image according to the Hessian matrix;

[0092] The maximum principal curvature and the minimum principal curvature are screened according to a preset edge point determination formula to obtain an original edge point set; the edge point determination formula is: EP is the original edge point set; k1 is the maximum principal curvature; k2 is the minimum principal curvature; σ1 is the adaptive first threshold; σ2 is the adaptive second threshold;

[0093] The original edge point set is denoised using the noise reduction formula of Yusa to obtain the main edge point set; the noise reduction formula is: EP′ is the main edge point set; p i is the number of original edge points within the 8 nearest pixels of the ith original edge point in the original edge point set; i is the length of the curve formed by the edge points where the ith original edge point in the original edge point set is located; S i is the area of ​​the region formed by the edge points where the ith original edge point in the original edge point set is located; η is the adaptive length threshold; δ is the adaptive area threshold;

[0094] Matching the main edge point set to the grayscale curved surface image to obtain a curved surface image with fused edge points;

[0095] All images in the grayscale surface set are traversed to obtain surface images of several fused edge points, and the surface images of all fused edge points are integrated to obtain the grayscale surface edge image set.

[0096] Preferably, the grayscale atlas is initially segmented using a watershed algorithm to obtain an initial segmented image set, including:

[0097] Denoising any grayscale image in the grayscale atlas to obtain a denoised image;

[0098] According to the preset single image contour number, the findContours function is used to search the contours of the denoised image to obtain a plurality of contour distributions;

[0099] Using the contour distribution, the images in the forest image set corresponding to the grayscale image are segmented to obtain a plurality of contour segmented images;

[0100] All images in the grayscale atlas are traversed and integrated to obtain the initial segmentation image set.

[0101] Furthermore, the forest image set, the initial segmentation image set and the gray surface edge image set are used as inputs of a preset convolutional neural network, and the manually segmented forest single tree segmentation image set is used as output of the convolutional neural network. The convolutional neural network is trained to obtain the trained single tree segmentation model, including:

[0102] Input the forest image set and the grayscale surface edge image set into a 1×1 convolution respectively to extract local detail features, so as to obtain a first feature representation and a second feature representation;

[0103] Inputting the first feature representation and the second feature representation into a BN layer for feature normalization to obtain a first normalized feature representation and a second normalized feature representation;

[0104] fusing the first normalized feature representation and the second normalized feature representation to obtain a fused feature representation;

[0105] Inputting the fused feature representation into a preset deep learning network for prediction to obtain a predicted segmented image;

[0106] According to the forest farm single tree segmentation image set, a preset loss function is used to calculate the loss value of the predicted segmentation image; the calculation formula of the loss function is: Loss b is the loss value of the b-th iteration; is the predicted area of ​​the a-th single tree segmentation region in the b-th iteration; S ab is the overlap area between the predicted area and the actual area of ​​the a-th single tree segmentation area in the b-th iteration; m is the total number of actual single tree segmentation areas;

[0107] Based on the minimum gradient descent method, the parameters of the deep learning network, the edge point determination formula and the noise reduction formula are optimized according to the loss value;

[0108] The deep learning network, the edge point determination formula and the noise reduction formula are iteratively trained to obtain the trained single tree segmentation model.

[0109] Specifically, a drone equipped with both a visible light camera and a multispectral camera is used to collect data, and infrared thermal imaging and visible light frontal images are obtained; when conditions permit, a drone with a fixed route is used to collect spectral data, and multiple photos with overlapping parts are obtained. The drone settings are: flight altitude 60 meters to 80 meters, heading overlap 60%, and lateral overlap 60%.

[0110] Furthermore, the image data was preprocessed: the flight route and the corresponding photos were combined and stitched using the Stitcher library file to obtain the corresponding front view; the UAV GPS data was combined with the scope of the contracted forest land of the project, and the orthophoto map of the forest land to be used was cut out by expanding the actual contracted area 30m outward, and the noise reduction algorithm was used to improve the quality of the visible light image.

[0111] Optionally, this embodiment provides another neural network design idea, including: designing a neural network with an encoder-decoder structure and a parameter amount of 4M, using different regularization functions and texture differences between the output image and the input image as a guide, and using the Adam optimizer to optimize the network parameters. Use the multi-scale features extracted from the encoder to perform image feature-level fusion. In the model extraction feature, we use multi-scale features of different dimensions and sizes generated at different stages of the encoder to participate in the fusion task. The fusion module fuses the global features and texture features at the same scale of visible light and thermal imaging respectively, and then integrates them. Finally, the integrated multi-scale features enter the decoder for image reconstruction. Use real data in the actual tree crown project, and fine-tune the model with texture information as an indicator to obtain an image fusion neural network suitable for the project scenario.

[0112] Optionally, the segmentation model of this embodiment outputs a json file for each tree crown range. After the segmentation results are visualized according to the json file, data such as the number of green plants in the area, area, and percentage of the area can be obtained based on statistics of the visualized segmented image.

[0113] The beneficial effects of the present invention are as follows:

[0114] The present invention achieves rich input information by fusing the features of visible light images and infrared thermal images, successfully replaces point cloud data acquisition equipment, and reduces costs; through a single tree segmentation model pre-trained based on principal curve rate and watershed algorithm, accurate segmentation of feature fused images is achieved, avoiding the problems of poor data quality and low large-scale segmentation accuracy in traditional point cloud data segmentation.

[0115] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0116] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A single tree segmentation method based on infrared thermal images and visible light images, It is characterized in that include: Using a drone equipped with both a visible light camera and a multispectral camera to collect images of the target identification area, obtaining an original visible light image set and an original thermal imaging image set; Separately stitching the original visible light image set and the original thermal imaging image set to obtain an overall visible light image and an overall thermal imaging image; Extracting and fusing features of the overall visible light image and the overall thermal imaging image to obtain a feature fused image; The feature fusion image is input into a single tree segmentation model pre-trained based on principal curve rate and watershed algorithm to obtain a target segmentation image.

2. A single tree segmentation method based on infrared thermal images and visible light images according to claim 1, It is characterized in that Extracting and fusing the overall visible light image and the overall thermal imaging image to obtain a feature fused image includes: Extracting the entire visible light image using a preset visible light feature extraction network to obtain a visible light feature matrix; Extracting the overall thermal imaging image using a preset thermal imaging feature extraction network to obtain a thermal imaging feature matrix; Splicing the visible light feature matrix and the thermal imaging feature matrix to obtain a multi-modal splicing feature; Performing a first global average pooling and a first maximum pooling in a channel domain on the multimodal splicing features respectively to obtain a global channel description and a maximum channel description; Inputting the global channel description and the maximum channel description into a basic neural network calculation to obtain a channel weight coefficient; the basic neural network includes: a Relu activation function, a Sigmoid activation function, and at least two neural network layers; Multiplying the channel weight coefficient by the multimodal concatenation feature to obtain a channel-corrected multimodal feature; Performing a second global average pooling and a second maximum pooling in the spatial domain on the multimodal features of the channel correction respectively to obtain a global spatial description and a maximum spatial description; Inputting the global spatial description and the maximum spatial description into a basic neural network calculation to obtain a spatial weight coefficient; Multiplying the channel-corrected multimodal feature by the spatial weight coefficient to obtain a spatially corrected multimodal feature; After performing a third global average pooling and Sigmoid calculation on the spatially corrected multimodal features, a comprehensive image feature is obtained; The overall visible light image is corrected according to the comprehensive image characteristics to obtain the feature fusion image.

3. The method for segmenting a single tree based on infrared thermal images and visible light images according to claim 1, It is characterized in that The construction process of the single tree segmentation model includes: Convert the pre-collected forest image set into grayscale format to obtain a grayscale atlas; Extracting the grayscale value of each pixel in the grayscale atlas, and defining the grayscale value as the height, to obtain a three-dimensional point distribution set; Using a radial basis function difference method to fit the three-dimensional point distribution set into a surface, to obtain a grayscale surface set; Calculating the principal curve ratio of the grayscale surface set, and obtaining the grayscale surface edge image set according to the principal curve ratio; Performing a primary segmentation on the grayscale atlas using a watershed algorithm to obtain a primary segmentation image set; The forest image set, the initial segmentation image set and the grayscale surface edge image set are used as inputs of a preset convolutional neural network, and the manually segmented forest single tree segmentation image set is used as the output of the convolutional neural network. The convolutional neural network is trained to obtain the trained single tree segmentation model.

4. The method for segmenting a single tree based on infrared thermal images and visible light images according to claim 1, It is characterized in that The tool used for stitching the original visible light image set and the original thermal imaging image set is: Stitcher library file.

5. The method for segmenting a single tree based on infrared thermal images and visible light images according to claim 1, It is characterized in that The acquisition parameter settings of the UAV include: an altitude range of 60 to 80 meters, a heading overlap of 60%, and a lateral overlap of 60%.

6. The method for segmenting a single tree based on infrared thermal images and visible light images according to claim 3, It is characterized in that Calculating the principal curve ratio of the grayscale surface set, and obtaining the grayscale surface edge image set according to the principal curve ratio, including: Randomly select a grayscale surface image according to the grayscale surface set; Constructing a Hessian matrix according to the grayscale surface image; Calculating the maximum principal curvature and the minimum principal curvature of each pixel in the grayscale surface image according to the Hessian matrix; The maximum principal curvature and the minimum principal curvature are screened according to a preset edge point determination formula to obtain an original edge point set; the edge point determination formula is: EP is the original edge point set; k 1 is the maximum principal curvature; k 2 is the minimum principal curvature; σ 1 is the first adaptive threshold; σ 2 is the adaptive second threshold; The original edge point set is denoised using the noise reduction formula of Yusa to obtain the main edge point set; the noise reduction formula is: EP′ is the main edge point set; p i is the number of original edge points within the 8 nearest pixels of the ith original edge point in the original edge point set; i is the length of the curve formed by the edge points where the ith original edge point in the original edge point set is located; S i is the area of ​​the region formed by the edge points where the ith original edge point in the original edge point set is located; η is the adaptive length threshold; δ is the adaptive area threshold; Matching the main edge point set to the grayscale curved surface image to obtain a curved surface image with fused edge points; All images in the grayscale surface set are traversed to obtain surface images of several fused edge points, and the surface images of all fused edge points are integrated to obtain the grayscale surface edge image set.

7. The method for segmenting a single tree based on infrared thermal images and visible light images according to claim 3, It is characterized in that The grayscale atlas is segmented initially using a watershed algorithm to obtain an initial segmented image set, including: Denoising any grayscale image in the grayscale atlas to obtain a denoised image; According to the preset single image contour number, the findContours function is used to search the contours of the denoised image to obtain a plurality of contour distributions; Using the contour distribution, the images in the forest image set corresponding to the grayscale image are segmented to obtain a plurality of contour segmented images; All images in the grayscale atlas are traversed and integrated to obtain the initial segmentation image set.

8. The method for segmenting a single tree based on infrared thermal images and visible light images according to claim 6, It is characterized in that The forest image set, the initial segmentation image set and the gray surface edge image set are used as inputs of a preset convolutional neural network, the manually segmented forest single tree segmentation image set is used as output of the convolutional neural network, and the convolutional neural network is trained to obtain the trained single tree segmentation model, including: Input the forest image set and the grayscale surface edge image set into a 1×1 convolution respectively to extract local detail features, so as to obtain a first feature representation and a second feature representation; Inputting the first feature representation and the second feature representation into a BN layer for feature normalization to obtain a first normalized feature representation and a second normalized feature representation; fusing the first normalized feature representation and the second normalized feature representation to obtain a fused feature representation; Inputting the fused feature representation into a preset deep learning network for prediction to obtain a predicted segmented image; According to the forest farm single tree segmentation image set, a preset loss function is used to calculate the loss value of the predicted segmentation image; the calculation formula of the loss function is: Loss b is the loss value of the b-th iteration; is the predicted area of ​​the a-th single tree segmentation region of the b-th iteration; S ab is the overlap area between the predicted area and the actual area of ​​the a-th single tree segmentation area in the b-th iteration; m is the total number of actual single tree segmentation areas; Based on the minimum gradient descent method, the parameters of the deep learning network, the edge point determination formula and the noise reduction formula are optimized according to the loss value; The deep learning network, the edge point determination formula and the noise reduction formula are iteratively trained to obtain the trained single tree segmentation model.

Images