Mountain forest aerial image tree species identification and counting method based on multi-model fusion

Abstract

The invention discloses a mountain forest aerial image tree species identification and counting method based on multi-model fusion, and the method comprises the steps: employing an unmanned plane to collect and splice large-scale mountain forest orthoimages with latitude and longitude coordinates under a top view angle in a simulated flight mode, and making a multi-tree species sample data set; selecting a plurality of object detection models based on different technical routes and network structures, and training the object detection models in a multi-tree-species sample data set; performing comprehensive scoring on each model, and screening out the first n models with the highest scores for subsequent identification result fusion; normalizing the confidence of the multi-tree identification results output by different models by using a confidence normalized improved weighted detection box fusion strategy, and then fusing the multiple results; converting the identification result of each tree species into the position of the tree species in the original orthograph according to the position relation of each small graph in the original orthograph, then converting the position information into latitude and longitude coordinates, and counting the final number and distribution of each type of target tree species in the mountain forest area.

Description

Technical Field

[0001] The present invention relates to a field, and in particular to a method for identifying and counting tree species in aerial forest images based on multi-model fusion. Background Art

[0002] A full census of the main tree species in the forest can provide a comprehensive understanding of the number and distribution of various tree species in the forest, provide data support for the protection and management of forest resources, and provide a basis for relevant departments to formulate policies; with the rapid development of drones and artificial intelligence technology, it is now possible to use drones to efficiently obtain high-quality aerial images of large-scale areas, and on this basis, use deep learning technology to accurately identify and locate various target objects. In the field of forest tree species census, since various types of trees have certain differences in appearance features such as leaf color, texture, and shape, drones can also be used to collect images of target mountain forest areas, and based on deep learning technology, the appearance features of various tree species can be extracted to achieve high-precision identification and location of individual trees and statistics on the number and distribution of various tree species in the area.

[0003] The existing technology usually selects several representative object detection models and directly applies all the selected models to the target mountain forest area for multi-tree species identification and positioning. Due to the particularity of the multi-tree species identification scene, the selected object detection model may not be suitable for this scene, and its recognition accuracy may not be high. Directly fusing the recognition results of all models may not necessarily produce the best recognition effect; and if all models are applied separately to the entire target area for inference and recognition, the recognition time will be very long and the efficiency will be low; secondly, when fusing multi-model recognition results, the existing technology often treats the confidence of different model recognition results equally, and different models have large differences in their principles and structures, and the distribution of their confidence values ​​often also has large differences. Directly comparing the numerical values ​​of different model confidences will result in imbalance between models, which can easily cause inaccurate fusion results. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the present invention proposes a method for identifying and counting tree species in aerial forest images based on multi-model fusion.

[0005] An embodiment of the present invention solves the technical problem by adopting a technical solution: a method for identifying and counting tree species in aerial forest images based on multi-model fusion, comprising:

[0006] S1. Use a drone with a camera to collect aerial images from a bird's-eye view in a terrain-mimicking flight mode over the target mountain forest area, and stitch the original aerial photos into a large-scale orthophoto with longitude and latitude coordinates;

[0007] S2. Divide the large-scale orthophoto into small images of equal size, select several small images containing various target tree species in typical areas, annotate the multi-species samples to form a multi-species sample dataset, and divide the multi-species sample dataset into a training set, a validation set, and a test set. When naming the small image files, their position information in the original orthophoto is retained;

[0008] S3. Select several object detection models based on different technical routes and network structures, train them separately in the training set of the multi-tree sample data set, use the validation set to tune the parameters of each object detection model, and test them in the test set to obtain the recognition results of each object detection model;

[0009] S4, comprehensively scoring each object detection model, selecting the top n object detection models with the highest scores and applying them to all small images cut out from the orthophoto of the target mountain forest area for multi-tree species recognition, and obtaining n different multi-tree species recognition results;

[0010] S5, fusing the multi-tree species recognition results output by the n different object detection models in step S4, and outputting a fused final recognition result;

[0011] S6. According to the positional relationship of each small image in the orthophoto, the position of the tree species in each small image is converted into its position in the orthophoto, and the position information is converted into longitude and latitude coordinates to count the final number and distribution of various target tree species in the mountain forest area.

[0012] As one of the preferred embodiments of the present invention, in step S3, the object detection model includes at least a single-stage object detection model, a two-stage object detection model, a deep convolutional network-based object detection model and / or a Transformer-based object detection model.

[0013] As one of the preferred embodiments of the present invention, in step S4, the comprehensive score calculation formula of the object detection model is as follows:

[0014] score i =accuracy i *α+di ver si ty i *β

[0015] +(ti me max -ti me i ) / (ti me max -ti me min )*γ

[0016] Among them, accuracy iThe accuracy of the recognition results for each object detection model; i The difference between each object detection model and other object detection models; time max The average recognition time of the model with the longest recognition time among all object detection models; time min The time taken by the model with the shortest average recognition time among all object detection models; time i is the average recognition time of a single picture of model i; α, β, γ are weights.

[0017] As one of the preferred embodiments of the present invention, the accuracy calculation formula of the object detection model recognition result is as follows:

[0018] accu racy i =N TP_i / (N TP_i +N FP_i +N FN_i )

[0019] Among them, N TP_i is the number of samples correctly identified by model i in the test set; N FP_i is the number of samples misidentified by model i; N FN_i The number of samples missed by model i.

[0020] As one of the preferred embodiments of the present invention, the difference calculation formula between each object detection model and other object detection models is as follows:

[0021] di ver si ty i_al l + = (1-N overlap_i_j / (N prediction_i +N prediction_j -N ove

[0022] rlap_i_j ))

[0023] di ver si ty i =di ver si ty i_al l / (N _al l -1)

[0024] Among them, N over lap_i_j is the number of target objects predicted by both model i and model j; N prediction_i is the number of target objects predicted by model i; N prediction_j is the number of target objects predicted by model j; N _all is the total number of candidate models in the model library; i_al l It is the sum of the differences between the recognition results of model i and other object detection models, and its value is divided by (N _al l -1) is the average difference between the recognition results of model i and other object detection models i .

[0025] As one of the preferred embodiments of the present invention, in step S4, each object detection model recognition result includes at least the file name of the thumbnail, the type of target object recognized in the thumbnail, the position and size of the recognition box, and the confidence information of each recognition prediction result.

[0026] As one of the preferred embodiments of the present invention, in step S5, the steps of fusing the multi-tree species recognition results output by n different object detection models are as follows:

[0027] S5.1. Normalize the confidences corresponding to the recognition results of n object detection models, and use all the recognition outputs of each object detection model as the input of the weighted target box fusion strategy;

[0028] S5.2. Each prediction box in each object detection model is added to a separate list B in turn, and then the prediction boxes in list B are sorted in descending order according to their corresponding confidence C;

[0029] S5.3. Define an empty list L and a list F to represent the prediction box cluster and the fused prediction box respectively;

[0030] S5.4, loop through list B and find a matching prediction box in list F. If no matching prediction box is found, add it to the end of list L and list F. If a matching prediction box is found, add the box to list L, and the added position is the index position index of the matching box of the prediction box in list F. When the intersection and union ratio of two prediction boxes is greater than a certain threshold, the two are considered to match.

[0031] S5.5. After the traversal is completed, the confidence values ​​of the elements in list F are adjusted again, and the improved weighted detection frames are fused after confidence normalization to output a fused final multi-tree species recognition result.

[0032] As one of the preferred embodiments of the present invention, in step S5.4, if there are multiple different prediction boxes at each position in the list L, the value of the corresponding fused box F[index] is updated according to the multiple prediction boxes, and the updating formula is as follows:

[0033]

[0034]

[0035]

[0036] Among them, C is the updated confidence of the corresponding fusion frame, X1,2 is the coordinate value of the two points in the upper left corner and the lower right corner of the fusion frame in the horizontal direction; Y1,2 is the coordinate value of the two points in the upper left corner and the lower right corner of the fusion frame in the vertical direction.

[0037] As one of the preferred embodiments of the present invention, the ratio of the training set, the validation set and the test set is 4:1:1.

[0038] Beneficial effects of the present invention:

[0039] 1. All the selected object detection models are tested in the multi-tree training set, and some models suitable for this scenario are selected for subsequent recognition and fusion, which overcomes the problem of too long recognition time caused by selecting all object detection models for recognition and fusion, and the negative effects of models not suitable for this scenario in the fusion process;

[0040] 2. When selecting object detection models, comprehensively consider multiple factors such as the recognition accuracy of the object detection model in multiple tree species data sets, the difference in recognition results with other object detection models, and the recognition time of the object detection model, so as to select object detection models with a good balance in recognition performance and accuracy and obtain better fusion effects;

[0041] 3. When fusing the recognition results of the first n object detection models screened out, the confidence of the recognition results of each object detection model is normalized before applying the specific fusion strategy to eliminate the defect of incomparability of confidence values ​​caused by differences in the specific implementation and calculation methods of confidence among different object detection models. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0043] Figure 1 This is a flowchart of the tree species identification and counting method for mountain forest aerial images based on multi-model fusion;

[0044] Figure 2 Schematic diagram of some mountain and forest orthophoto areas collected and stitched for the terrain-simulating flight mode;

[0045] Figure 3 Annotate a diagram for a multi-species sample dataset;

[0046] Figure 4Schematic diagram of the recognition output information of a single object detection model;

[0047] Figure 5 This is a schematic diagram of the final multi-tree species recognition results after fusing the multi-model recognition results. DETAILED DESCRIPTION

[0048] This section will describe in detail the specific embodiments of the present invention. The preferred embodiments of the present invention are shown in the accompanying drawings. The purpose of the accompanying drawings is to supplement the description of the text part of the specification with graphics, so that people can intuitively and vividly understand each technical feature and the overall technical solution of the present invention, but it cannot be understood as a limitation on the scope of protection of the present invention.

[0049] In the description of the present invention, the meaning of "above", "below", "exceed", etc. is not inclusive of the number itself, and the meaning of "above", "below", "within", etc. is inclusive of the number itself. If there is a description of "first" or "second", it is only used to distinguish the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0050] In the description of the present invention, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., and orientations or positional relationships indicated are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0051] In the present invention, unless otherwise clearly defined, the words "set", "install", "connect" and the like should be understood in a broad sense, for example, they can be directly connected or indirectly connected through an intermediate medium; they can be fixedly connected or detachably connected or integrally formed; they can be mechanically connected; they can be the internal connection of two elements or the interaction relationship between two elements. Those skilled in the art can reasonably determine the specific meanings of the above words in the present invention in combination with the specific content of the technical solution.

[0052] Reference Figure 1 , a tree species identification and counting method for mountain forest aerial images based on multi-model fusion, including:

[0053] S1. Use a drone with a camera to collect aerial images from a bird's-eye view in a terrain-mimicking flight mode over the target mountain forest area, and stitch the original aerial photos into a large-scale orthophoto with longitude and latitude coordinates;

[0054] S2. Divide the large-scale orthophoto into small images of equal size, select several small images containing various target tree species in typical areas, annotate the multi-species samples to form a multi-species sample dataset, and divide the multi-species sample dataset into a training set, a validation set, and a test set. When naming the small image files, their position information in the original orthophoto is retained;

[0055] S3. Select several object detection models based on different technical routes and network structures, train them separately in the training set of the multi-tree sample data set, use the validation set to tune the parameters of each object detection model, and test them in the test set to obtain the recognition results of each object detection model;

[0056] S4, comprehensively scoring each object detection model, selecting the top n object detection models with the highest scores and applying them to all small images cut out from the orthophoto of the target mountain forest area for multi-tree species recognition, and obtaining n different multi-tree species recognition results;

[0057] S5, fusing the multi-tree species recognition results output by the n different object detection models in step S4, and outputting a fused final recognition result;

[0058] S6. According to the positional relationship of each small image in the orthophoto, the position of the tree species in each small image is converted into its position in the orthophoto, and the position information is converted into longitude and latitude coordinates to count the final number and distribution of various target tree species in the mountain forest area.

[0059] In the present invention:

[0060] 1) Aerial image acquisition and orthophoto stitching of mountains and forests based on terrain-simulating flight mode

[0061] Reference Figure 2 When using drones to collect forest images, use multi-rotor drones equipped with high-resolution cameras to collect images in a terrain-mimicking flight mode, that is, the drone always maintains a fixed distance from the ground directly below during flight. The sizes of tree species in the forest images collected in this way are relatively uniform; the drone's flight altitude can be set to 200-250 meters, the flight speed is about 10 meters per second, the heading overlap rate is 60%-80% when setting the flight route, and the lateral overlap rate is 60%. During flight, an aerial original film is taken at a fixed interval from a bird's-eye view; after the field aerial image collection is completed, all the collected aerial original films are stitched into an orthophoto with precise latitude and longitude coordinates using image stitching algorithms or software.

[0062] 2) Orthophoto segmentation and multi-species sample dataset production

[0063] Reference Figure 3, the orthophoto of the target forest area is evenly divided into 1024*1024 pixel small images of equal size. The name of each small image file contains the row number and column number of the area corresponding to the small image in the orthophoto, so as to facilitate the subsequent conversion of the position of the identified trees in the small image to its position in the orthophoto and calculate the longitude and latitude;

[0064] About 2000 small images containing a large number of target tree species were randomly selected from all the segmented small images as the sample data set. Image annotation tools such as labe ll mg were used to annotate all sample images with multiple tree species. The target tree species included bamboo, fir, pine and broad-leaved trees. During the annotation process, attention was paid to not missing target trees and the size of the annotation box should be able to match the corresponding trees well. After all samples were annotated, they were divided into training set, validation set and test set in a ratio of approximately 4:1:1.

[0065] 3) Construction of candidate model library for multi-tree species identification and training and optimization of each model

[0066] Select multiple object detection models to build a multi-tree recognition candidate model library. In order to make the selected models more representative and the recognition results more diverse, the construction of the candidate model library needs to include typical models based on various technical routes and network architectures, such as: single-stage object detection model, two-stage object detection model, deep convolutional network-based object detection model, and Transformer-based object detection model.

[0067] After completing the model selection of the candidate model library, each object detection model is trained separately in the training set of the multi-tree species sample data set produced in the above 2). During the training process, the validation set is used to tune the parameters of each model. Sample augmentation and other strategies can also be used to improve the model accuracy. After the training is completed, the recognition results of each model are obtained by testing them in the test set.

[0068] 4) Selection of multi-species detection models based on comprehensive factors

[0069] Based on multiple factors such as the accuracy of the object detection model recognition results, the difference between the current object detection model recognition results and other object detection model recognition results, and the average object detection model recognition time, each candidate model in the model library is comprehensively scored, and the top n models with the highest scores are selected for subsequent recognition result fusion;

[0070] The comprehensive score calculation formula of the object detection model is as follows:

[0071] score i =accuracy i *α+di ver si ty i *β

[0072] +(ti me max -ti me i ) / (ti me max -ti me min )*γ

[0073] Among them, accuracy i The accuracy of the recognition results for each object detection model; i The difference between each object detection model and other object detection models; time max The average recognition time of the model with the longest recognition time among all object detection models; time min The time taken by the model with the shortest average recognition time among all object detection models; time i is the average recognition time of a single picture of model i; α, β, γ are weights.

[0074] In one embodiment, the accuracy calculation formula of the object detection model recognition result is as follows:

[0075] accu racy i =N TP_i / (N TP_i +N FP_i +N FN_i )

[0076] Among them, N TP_i is the number of samples correctly identified by model i in the test set; N FP_i is the number of samples misidentified by model i; N FN_i The number of samples missed by model i.

[0077] In one embodiment, the code snippet for calculating the difference between each object detection model and other object detection models is as follows:

[0078]

[0079] Among them, N over lap_i_j is the number of target objects predicted by both model i and model j; N prediction_i is the number of target objects predicted by model i; N prediction_j is the number of target objects predicted by model j; N _al l is the total number of candidate models in the model library; i_al l It is the sum of the differences between the recognition results of model i and other object detection models, and its value is divided by (N _all -1) is the average difference between the recognition results of model i and other object detection models i ;

[0080] The comprehensive scores of all object detection models in the model library are obtained according to the calculation results of the comprehensive score calculation formula of the object detection model, and all object detection models are sorted from high to low according to the score. The first n object detection models are selected for subsequent recognition result fusion. According to experiments, when the value of n is 3 in the multi-tree species recognition scenario, better results can be achieved.

[0081] 5) Referring to FIG. 4, the n object detection models selected in 4) above are respectively applied to all small images cut out from the orthophoto to perform multi-tree species recognition, and n different multi-tree species recognition results are obtained. The recognition result of each object detection model includes the file name of the small image, the type of target object recognized in the small image, the position and size of the recognition box, and the confidence information of each recognition prediction result.

[0082] 6) Improved weighted detection box fusion based on confidence normalization

[0083] Reference Figure 5 , an improved weighted detection box fusion strategy based on confidence normalization is proposed, which fuses the n different recognition results of the n object detection models in 5) above, and uses the diversity of recognition results of different object detection models to correct the erroneous recognition results of a single object detection model, thereby improving the overall recognition accuracy. The specific process is as follows:

[0084] S5.1. Normalize the confidences corresponding to the recognition results of n object detection models to eliminate the differences in the distribution of confidence values ​​of different models. Common normalized data processing methods such as maximum-minimum normalization can be used. All recognition outputs of each object detection model are used as inputs to the weighted target box fusion strategy, including the positions of all prediction boxes and the normalized prediction confidences.

[0085] S5.2. Each prediction box in each object detection model is added to a separate list B in turn, and then the prediction boxes in list B are sorted in descending order according to their corresponding confidence C;

[0086] S5.3. Define an empty list L and a list F to represent the prediction box cluster and the fused prediction box respectively;

[0087] S5.4, loop through list B and find a matching prediction box in list F. If no matching prediction box is found, add it to the end of list L and list F. If a matching prediction box is found, add the box to list L, and the added position is the index position index of the matching box of the prediction box in list F. When the intersection and union ratio of two prediction boxes is greater than a certain threshold, the two are considered to match.

[0088] S5.5. After the traversal is completed, the confidence values ​​of the elements in list F are adjusted again, and the improved weighted detection frames are fused after confidence normalization to output a fused final multi-tree species recognition result.

[0089] In step S5.4, if there are multiple different prediction boxes at each position in the list L, the value of the corresponding fusion box F[index] is updated according to the multiple prediction boxes. The update formula is as follows:

[0090]

[0091]

[0092]

[0093] Among them, C is the confidence of the corresponding fusion frame after update, X1,2 is the coordinate value of the two points in the upper left corner and lower right corner of the fusion frame in the horizontal direction; Y1,2 is the coordinate value of the two points in the upper left corner and lower right corner of the fusion frame in the vertical direction;

[0094] After the traversal is completed, the confidence values ​​of the elements in list F are adjusted again. The adjustment formula is as follows:

[0095]

[0096] After confidence normalization, the improved weighted detection box fusion is performed to output a fused final multi-tree species recognition result; it should be pointed out that the recognition results of n different models after confidence normalization in this method, in addition to using the weighted detection box fusion strategy, are also applicable to other strategies for fusing multiple target detection and recognition results.

[0097] 7) Statistics on the number and distribution of various tree species

[0098] The above 6) obtains the position information of individual tree species in the small picture. According to the positional relationship of each small picture in the original orthophoto, the identification result of each tree species is converted into its position in the original orthophoto, and then the position information is converted into longitude and latitude coordinates to count the final number and distribution of various target tree species in the mountain forest area.

[0099] 8) The advantages of the present invention are:

[0100] 1. All the selected object detection models are tested in the multi-tree training set, and some models suitable for this scenario are selected for subsequent recognition and fusion, which overcomes the problem of too long recognition time caused by selecting all object detection models for recognition and fusion, and the negative effects of models not suitable for this scenario in the fusion process;

[0101] 2. When selecting object detection models, comprehensively consider multiple factors such as the recognition accuracy of the object detection model in multiple tree species data sets, the difference in recognition results with other object detection models, and the recognition time of the object detection model, so as to select object detection models with a good balance in recognition performance and accuracy and obtain better fusion effects;

[0102] 3. When fusing the recognition results of the first n object detection models screened out, the confidence of the recognition results of each object detection model is normalized before applying the specific fusion strategy to eliminate the defect of incomparability of confidence values ​​caused by differences in the specific implementation and calculation methods of confidence among different object detection models.

[0103] Of course, the present invention is not limited to the above-mentioned embodiments, and those skilled in the art may make equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications and substitutions are all included in the scope defined by the claims of this application.

Claims

1. A method for identifying and counting tree species in aerial forest images based on multi-model fusion, characterized in that: include: S1. Use a drone with a camera to collect aerial images from a bird's-eye view in a terrain-mimicking flight mode over the target mountain forest area, and stitch the original aerial photos into a large-scale orthophoto with longitude and latitude coordinates; S2. Divide the large-scale orthophoto into small images of equal size, select several small images containing various target tree species in typical areas, annotate the multi-species samples to form a multi-species sample dataset, and divide the multi-species sample dataset into a training set, a validation set, and a test set. When naming the small image files, their position information in the original orthophoto is retained; S3. Select several object detection models based on different technical routes and network structures, train them separately in the training set of the multi-tree sample data set, use the validation set to tune the parameters of each object detection model, and test them in the test set to obtain the recognition results of each object detection model; S4, comprehensively scoring each object detection model, selecting the top n object detection models with the highest scores and applying them to all small images cut out from the orthophoto of the target mountain forest area for multi-tree species recognition, and obtaining n different multi-tree species recognition results; S5, fusing the multi-tree species recognition results output by the n different object detection models in step S4, and outputting a fused final recognition result; S6. According to the positional relationship of each small image in the orthophoto, the position of the tree species in each small image is converted into its position in the orthophoto, and the position information is converted into longitude and latitude coordinates to count the final number and distribution of various target tree species in the mountain forest area.

2. The method for identifying and counting tree species in aerial forest images based on multi-model fusion according to claim 1 is characterized by: In step S3, the object detection model includes at least a single-stage object detection model, a two-stage object detection model, a deep convolutional network-based object detection model and / or a Transformer-based object detection model.

3. The method for identifying and counting tree species in aerial forest images based on multi-model fusion according to claim 1 is characterized in that: In step S4, the comprehensive score calculation formula of the object detection model is as follows: score i =accuracy i *α+diver sity i *β+(time max -time i ) / (time max -time min )*γ Among them, accuracy i The accuracy of the recognition results of each object detection model; divergence i The difference between each object detection model and other object detection models; time max The average recognition time of the model with the longest recognition time among all object detection models; time min The time taken by the model with the shortest average recognition time among all object detection models; time i is the average recognition time of a single picture of model i; α, β, γ are weights.

4. The method for identifying and counting tree species in aerial forest images based on multi-model fusion according to claim 3 is characterized by: The accuracy calculation formula of the object detection model recognition result is as follows: accu racy i =N TP_i / (N TP_i +N FP_i +N FN_i ) Among them, N TP_i N is the number of samples correctly identified by model i in the test set; FP_i is the number of samples misidentified by model i; N FN_i The number of samples missed by model i.

5. The method for identifying and counting tree species in aerial forest images based on multi-model fusion according to claim 3 is characterized by: The formula for calculating the difference between each object detection model and other object detection models is as follows: diver sity i_all +=(1-N overlap_i_j / (N prediction_i +N prediction_j -N overlap_i_j )) diver sity i =diver sity i_all / (N _all -1) Among them, N overlap_i_j is the number of target objects predicted by both model i and model j; N prediction_i is the number of target objects predicted by model i; N prediction_j is the number of target objects predicted by model j; N _all is the total number of candidate models in the model library; divergence i_all It is the sum of the differences between the recognition results of model i and other object detection models, and its value is divided by (N _all -1) is the average difference between the recognition results of model i and other object detection models i .

6. The method for identifying and counting tree species in aerial forest images based on multi-model fusion according to claim 1 is characterized by: In step S4, each object detection model recognition result includes at least the file name of the thumbnail, the type of target object recognized in the thumbnail, the position and size of the recognition box, and the confidence information of each recognition prediction result.

7. The method for identifying and counting tree species in aerial forest images based on multi-model fusion according to claim 1 is characterized in that: In step S5, the steps of fusing the multi-tree species recognition results output by n different object detection models are as follows: S5.

1. Normalize the confidences corresponding to the recognition results of n object detection models, and use all the recognition outputs of each object detection model as the input of the weighted target box fusion strategy; S5.

2. Each prediction box in each object detection model is added to a separate list B in turn, and then the prediction boxes in list B are sorted in descending order according to their corresponding confidence C; S5.

3. Define an empty list L and a list F to represent the prediction box cluster and the fused prediction box respectively; S5.4, loop through list B and find a matching prediction box in list F. If no matching prediction box is found, add it to the end of list L and list F. If a matching prediction box is found, add the box to list L, and the added position is the index position index of the matching box of the prediction box in list F. When the intersection and union ratio of two prediction boxes is greater than a certain threshold, the two are considered to match. S5.

5. After the traversal is completed, the confidence values ​​of the elements in list F are adjusted again, and the improved weighted detection frames are fused after confidence normalization to output a fused final multi-tree species recognition result.

8. The method for identifying and counting tree species in aerial forest images based on multi-model fusion according to claim 7 is characterized in that: In step S5.4, if there are multiple different prediction boxes at each position in the list L, the value of the corresponding fused box F[index] is updated according to the multiple prediction boxes. The update formula is as follows: Among them, C is the updated confidence of the corresponding fusion frame, X1,2 is the coordinate value of the two points in the upper left corner and the lower right corner of the fusion frame in the horizontal direction; Y1,2 is the coordinate value of the two points in the upper left corner and the lower right corner of the fusion frame in the vertical direction.

9. The method for identifying and counting tree species in aerial forest images based on multi-model fusion according to claim 1 is characterized by: The ratio of the training set, validation set and test set is 4:1:1.

Images