Tree species labeling and recognition system based on deep learning

Abstract

The invention discloses a tree species annotation and recognition system based on deep learning in the field of tree image recognition, and the system comprises an image annotation module which employs an interactive image segmentation technology to extract a key region in an image, and the interactive image segmentation technology comprises automatic segmentation through a segmentation model and user-defined frame selection segmentation; after segmentation is completed, classification labeling is carried out on the segmented areas according to categories selected by a user, and training set data is generated to be used by a tree species classification model; the tree species identification module is used for carrying out tree species identification and quantity statistics on the input image by utilizing the trained tree species classification model and generating a visual output or data file containing a classification result; and the tree species classification model is trained by using training set data provided by the image annotation module. According to the tree species labeling and recognition system based on deep learning provided by the invention, efficient and accurate tree species classification and quantity statistics are realized through cooperative work of the image labeling module and the tree species recognition module.

Description

Technical Field

[0001] The invention relates to a tree species marking and identification system based on deep learning, and belongs to the field of tree image recognition. Background Art

[0002] With the increasing importance of ecological environment protection and forestry resource management, accurate tree monitoring and identification has become a research hotspot. Traditional tree identification methods rely on manual surveys, which are inefficient and easily affected by human factors. The rapid development of drones has provided new perspectives and technical means for forestry monitoring. Through high-resolution image acquisition, drones can conduct comprehensive monitoring of tree canopies at different heights. This method not only improves the efficiency of data acquisition, but also covers a wider area and obtains more detailed information. However, existing drone image processing systems often lack efficient intelligent recognition mechanisms, resulting in the accuracy and efficiency of data processing and analysis failing to meet actual needs. Summary of the invention

[0003] The purpose of the present invention is to overcome the deficiencies in the prior art and to provide a tree species labeling and identification system based on deep learning, which realizes efficient and accurate tree species classification and quantity statistics through the collaborative work of an image labeling module and a tree species identification module.

[0004] To achieve the above object, the present invention is implemented by adopting the following technical solutions: The present invention provides a tree species labeling and identification system based on deep learning, comprising: The image annotation module uses interactive image segmentation technology to extract key areas in the image. The interactive image segmentation technology includes automatic segmentation using a segmentation model and user-defined box selection segmentation, and supports users to fine-tune the segmentation results of the segmentation model. After the segmentation is completed, the segmented areas are classified and annotated according to the categories selected by the user, and training set data is generated for use by the tree species classification model; The tree species identification module uses the trained tree species classification model to identify and count the tree species in the input image, and generates a visual output or data file containing the classification results; the tree species classification model is trained using the training set data provided by the image annotation module.

[0005] Furthermore, the tree species marking and identification system also includes: Image acquisition module, used to collect high-resolution images of tree canopies at different heights and transmit the images to the ground workstation; The image preprocessing module is used to screen the collected images and remove low-quality, abnormal or irrelevant images; The user interaction module is used to implement image segmentation, tree species classification and result display through a graphical user interface. The interface supports image browsing, segmentation result fine-tuning, classification result viewing and file export operations.

[0006] Furthermore, the interactive image segmentation technology is used to extract the key areas in the image, including: Receiving an interactive segmentation instruction input by a user; determining a region to be segmented in the image based on the instruction; automatically or manually segmenting the region to be segmented, and allowing the user to adjust the result of the automatic segmentation; The receiving of the interactive segmentation instruction input by the user includes: automatically determining the entire segmentation area by using the SAM segmentation model after clicking the area to be segmented by the mouse; or manually selecting the segmentation area by using a user-defined box.

[0007] Furthermore, the tree species classification model is constructed based on a ResNet or DenseNet network structure.

[0008] Furthermore, the tree species classification model is trained using the training set data provided by the image annotation module, including: Performing data enhancement on the training set data, wherein the data enhancement includes adding Gaussian noise, random flipping, brightness adjustment, and geometric transformation; After data augmentation, the processed data is fed into the tree classification model to perform the training steps of forward propagation, loss calculation, backpropagation, and weight update; The cross entropy loss function and learning rate scheduler are used to optimize the tree species classification model. After training, the classification accuracy of the tree species classification model is evaluated, and the tree species classification model with the highest classification accuracy is saved.

[0009] Furthermore, the tree species classification model with the highest classification accuracy is tested by the following steps: Standardize the image data of the test set, including adjusting the image size, center cropping, tensor quantization, and normalization; The processed test images are input into the tree species classification model, the prediction results are calculated using forward propagation, and the performance of the tree species classification model is evaluated by comparing the accuracy of the actual category and the predicted category; Generates an output file containing the classification results of each test sample, which is used to record the statistical results of tree species classification in the test set.

[0010] Furthermore, after training and testing, the tree species classification model is deployed on an edge computing device or server to realize offline tree species identification and real-time classification functions.

[0011] Furthermore, the tree species classification model is used to identify and count the tree species in the input image, and a visual output or data file containing the classification results is generated, including: Input the canopy image to be classified into the trained tree species classification model; Calculate the probability distribution of the classification results of the input image and extract several categories with the highest classification probability; The classification results are visualized through the user interaction module, including the name of each category, the corresponding probability, and the number of trees.

[0012] Compared with the prior art, the present invention has the following beneficial effects: The tree species labeling and identification system based on deep learning provided by the present invention generates high-quality training data sets through interactive image segmentation technology and classification labeling functions, providing a reliable data basis for the training of tree species classification models. Using the trained tree species classification model to identify and count tree species in the input image not only achieves high accuracy of the classification results, but also supports the visualization and data export of the classification results, providing efficient and convenient technical support for forest resource management and ecological monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is an example of a photo of a tree canopy taken by a drone according to the second embodiment of the present invention; Figure 2 A schematic diagram of the main interface of the image annotation module of the tree species annotation and identification system based on deep learning provided in the second embodiment of the present invention; Figure 3 A schematic diagram of classification results output by a tree species identification module of the deep learning-based tree species labeling and identification system provided in Embodiment 2 of the present invention. DETAILED DESCRIPTION

[0014] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical solution of the present application, rather than limitations on the technical solution of the present application. In the absence of conflict, the embodiments of the present application and the technical features in the embodiments can be combined with each other.

[0015] Example 1

[0016] This embodiment provides a tree species labeling and identification system based on deep learning, and the system includes an image labeling module and a tree species identification module.

[0017] In the image annotation module, interactive image segmentation technology is used to extract key areas in the image. Specifically, the user can click on the area to be segmented with the mouse, and the system will call the segmentation model to automatically identify and generate the corresponding segmentation area. At the same time, the system also supports users to customize the segmentation area by polygonal selection, which is used to deal with situations where the segmentation model cannot accurately identify in complex scenes. After the segmentation is completed, the user can fine-tune the segmentation result and select the category corresponding to the segmentation area in the category module of the system interface. After the annotation is completed, the system binds the segmentation area with the category information to generate high-quality training set data for training tree species classification models.

[0018] In the tree species identification module, the trained tree species classification model is used to identify the tree species category and count the number of tree species in the input image. The tree species classification model is trained by using the training set data generated by the image annotation module to extract the characteristic information of different tree species and complete the model optimization. The trained model can accurately identify the tree species of the input canopy layer image and count the number of each tree species. Finally, the classification results are displayed in a visual way on the interface and can be exported as a data file in a standard format for further analysis by users.

[0019] Through the above implementation methods, the system realizes the efficient segmentation, labeling and classification functions of canopy images, providing reliable data support and technical guarantee for subsequent forest resource monitoring and management.

[0020] Example 2

[0021] In this embodiment, in response to the actual needs of forest resource monitoring, the deep learning-based tree species labeling and identification system not only includes an image labeling module and a tree species identification module, but also has expanded designs of other functional modules, such as an image acquisition module, an image preprocessing module, and a user interaction module, to improve the overall performance of the system and user experience.

[0022] The image acquisition module is used to collect high-resolution canopy images through drones and transmit them to ground workstations; the image preprocessing module is used to screen, crop and optimize the clarity of the collected images to ensure data quality; the user interaction module supports users to complete the entire process from image browsing, segmentation and annotation to classification result viewing through a friendly interface design. The collaborative work of these modules enables the system to not only efficiently realize tree species labeling and identification, but also provide users with stronger customized operation and analysis capabilities.

[0023] In this embodiment, a DJI Air 3 drone is used to take photos of the tree canopy at a fixed rate every 2 seconds at a fixed altitude of 60 meters and 85 meters, focusing on capturing the top structure and leaf distribution of the trees. The collected pictures are as follows: Figure 1As shown. In order to ensure the accuracy of subsequent tree species classification, the images need to be initially screened after collection. First, remove photos with low clarity or blur to ensure that the image quality meets the classification requirements; second, filter out images that are not related to the target task and only retain those that meet the shooting requirements; finally, remove abnormal images caused by factors such as cloud shadows or aircraft shadows. After these screening steps, a set of clear, relevant, and non-abnormal initial images is finally formed, laying a high-quality data foundation for subsequent tree species labeling and classification.

[0024] In this embodiment, the image annotation module, the tree species identification module and the user interaction module are developed using PyQT technology, wherein the main interface of the image annotation module is as follows: Figure 2 As shown in the figure, it has rich functions to support image annotation. The toolbar at the top of the interface integrates a variety of operation options, including file management, editing, view adjustment, and tool functions, which can quickly open image folders, switch interactive and custom segmentation modes, zoom in or out images, and support fast export of segmentation results.

[0025] In addition, the toolbar also provides a segmentation model selection function, and users can switch between different segmentation models. In this embodiment, the ViT-H (Vision Transformer-Huge) model is used by default to process high-resolution images. This model has powerful feature extraction capabilities and segmentation accuracy. If you are in an environment with limited resources, such as running on a mobile device, you can select the ViT-B (Vision Transformer-Base) model to meet performance requirements; when running on a medium-power device, the system provides a ViT-L (Vision Transformer-Large) model option to strike a balance between inference speed and segmentation accuracy.

[0026] The left area of ​​the interface is the annotation display / hide module, which provides the management function of the annotation content. Users can view the annotation list of the current image and display or hide specific annotation content by checking the checkbox, which is convenient for users to quickly locate and adjust the annotation results. The upper right corner of the interface is the category module, which displays annotations of different categories and their color markings. Users can add or delete categories as needed and set different colors for each category to intuitively distinguish the annotation areas. The file list module is located in the lower right corner of the interface, which displays all image files of the current project, supports users to quickly switch images, and provides the function of jumping to specific files to improve operational efficiency. The main display area of ​​the image is located in the center of the interface, which is used to display the currently loaded image. Users can directly segment and annotate in this area. The segmentation results will be highlighted in different colors. At the same time, it supports zooming in, zooming out, and panning operations, making local annotations more accurate.

[0027] To ensure smooth operation, the log area at the bottom displays the running status of the software in real time, such as the image root directory and loading path. These log contents not only provide users with clear operation feedback, but also help locate and solve related problems when problems arise. Overall, this module provides great convenience for the annotation of tree crown images through its feature-rich interface design and flexible operation mode.

[0028] In this embodiment, the user uses the interactive image segmentation technology to extract the key area in the image. The specific operation includes: after the system receives the segmentation instruction input by the user, it determines the area to be segmented in the image according to the instruction, and automatically or manually segments the area, and allows the user to adjust the automatic segmentation result. The user can choose to trigger the segmentation model to identify the area by clicking the image with the mouse, or customize the target area to achieve more flexible segmentation needs.

[0029] For mouse-click segmentation, the user clicks the area to be segmented with the left mouse button. The system will record the click position and use it as the input point. The segmentation model will be called to calculate the area around the point, and finally a segmentation mask will be generated. The mask will be superimposed on the original image to show the segmentation effect. For the details of the boundary of the segmented area, the system supports users to make subsequent fine-tuning to improve the segmentation accuracy.

[0030] In the custom frame segmentation mode, the user clicks on multiple points in the image in sequence to draw a polygonal area as the segmentation boundary. When the user has completed the selection of all boundary points, the system will use the information of these points to generate the segmentation results of the corresponding area and highlight them on the image to help users intuitively view the annotation effect.

[0031] For different types of tree crown images, users can choose the appropriate segmentation method according to the actual situation. For images with clear tree crown boundaries and easy to distinguish, it is suitable to use mouse point-and-click segmentation; users can add multiple annotation points by clicking different positions, and the system will generate corresponding segmentation areas based on these points. For images with blurred boundaries between tree crowns and difficult to distinguish, it is recommended to use custom frame segmentation. Users can gradually draw polygonal boundaries to accurately mark the area.

[0032] After completing the segmentation, users can select corresponding labels for the segmented areas in the "Category" module of the interface to indicate the tree species category. After labeling is completed, the system supports users to save the results through the "Save Segmentation" function, and use the quick export function in the toolbar to store the classification results in corresponding folders according to the labeled categories, thereby efficiently completing the construction of the image dataset. Through the above process, the system realizes interactive segmentation and labeling. Combined with automatic segmentation and custom box selection segmentation functions, the system can flexibly adapt to different types of crown images. Users can quickly generate segmentation areas with simple mouse clicks, or use polygonal box selection to accurately label complex scenes, which improves the efficiency and accuracy of image segmentation and provides high-quality input data for subsequent tree species classification model training.

[0033] The tree species classification model in this embodiment is based on the ResNet or DenseNet network structure. The tree species classification models based on different network structures are trained respectively. Finally, each tree species classification model is evaluated, and the tree species classification model with the best classification effect is selected to invade the tree species recognition module of this system. The specific model training process is as follows: Data enhancement is performed on the training set data, and the data enhancement includes adding Gaussian noise, random flipping, brightness adjustment and geometric transformation. Random flipping includes horizontal and vertical flipping, and geometric transformation includes rotation, translation, scaling and other operations, so as to enhance the adaptability of the tree species classification model to the crown structure in different directions. Brightness adjustment simulates different lighting conditions by increasing or decreasing the brightness and contrast of the crown image. The brightness and contrast are randomly changed during the enhancement process, so that the tree species classification model can handle images in complex environments such as sunny days, cloudy days or changes in sunlight. Noise is added to simulate slight distortions that may occur during shooting, such as slight blur or random pixel changes. Gaussian noise is used to simulate sensor noise or environmental interference, which makes the tree species classification model more adaptable to unstable image quality and improves its generalization ability.

[0034] After data enhancement, the processed data is input into the tree species classification model to perform the training steps of forward propagation, loss calculation, back propagation and weight update. The model is optimized using the cross entropy loss function and the learning rate scheduler. After the training is completed, the classification accuracy of the tree species classification model is evaluated, and the tree species classification model with the highest classification accuracy is saved.

[0035] After the training is completed, the tree species classification model with the highest classification accuracy is tested by the following steps: First, the images in the test set are resized and cropped to a fixed size, and the pixel values ​​are normalized to keep the size and color distribution of the images consistent with the input requirements during model training. The preprocessed images are converted into a format suitable for deep learning model processing and input into the tree species classification model. The prediction results are calculated using forward propagation, and the model performance is evaluated by comparing the accuracy of the actual category and the predicted category; an output file containing the classification results of each test sample is generated to record the statistical results of the tree species classification in the test set.

[0036] When the system is used for tree species identification, the canopy to be classified is first preprocessed, and then the tree species classification model infers the preprocessed canopy image to generate prediction scores for each category. These prediction scores are then converted into probability distributions, representing the prediction confidence of the tree species classification model for each category. To facilitate the display of results, the system extracts several categories with the highest prediction confidence, such as the three most likely tree species, and sorts them from high to low according to confidence, ensuring that the classification results have clear priorities.

[0037] The system further processes these classification results, mapping the category numbers to specific tree species names and converting the confidence level for each tree species into a percentage. Figure 3 As shown in the figure, the classification results include the name of each tree species and its corresponding confidence level. This information is presented to the user in a clear format and saved as a result file for subsequent analysis. In this way, the generation process of classification results is both scientific and standardized, and easy for users to understand and use.

[0038] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A tree species labeling and identification system based on deep learning, characterized in that: include: The image annotation module uses interactive image segmentation technology to extract key areas in the image. The interactive image segmentation technology includes automatic segmentation using a segmentation model and user-defined box selection segmentation, and supports users to fine-tune the segmentation results of the segmentation model. After the segmentation is completed, the segmented areas are classified and annotated according to the categories selected by the user, and training set data is generated for use by the tree species classification model; The tree species identification module uses the trained tree species classification model to identify and count the tree species in the input image, and generates a visual output or data file containing the classification results; The tree species classification model is trained using the training set data provided by the image annotation module.

2. The tree species labeling and identification system based on deep learning according to claim 1, characterized in that: Also includes: Image acquisition module, used to collect high-resolution images of tree canopies at different heights and transmit the images to the ground workstation; The image preprocessing module is used to screen the collected images and remove low-quality, abnormal or irrelevant images; The user interaction module is used to implement image segmentation, tree species classification and result display through a graphical user interface. The interface supports image browsing, segmentation result fine-tuning, classification result viewing and file export operations.

3. The tree species labeling and identification system based on deep learning according to claim 1, characterized in that: Use interactive image segmentation technology to extract key areas in the image, including: Receiving an interactive segmentation instruction input by a user; determining a region to be segmented in the image based on the instruction; automatically or manually segmenting the region to be segmented, and allowing the user to adjust the result of the automatic segmentation; The receiving of the interactive segmentation instruction input by the user includes: automatically determining the entire segmentation area by using the SAM segmentation model after clicking the area to be segmented by the mouse; or manually selecting the segmentation area by using a user-defined box.

4. The tree species labeling and identification system based on deep learning according to claim 1, characterized in that: The tree species classification model is constructed based on a ResNet or DenseNet network structure.

5. The tree species labeling and identification system based on deep learning according to claim 4, characterized in that: The tree species classification model is trained using the training set data provided by the image annotation module, including: Performing data enhancement on the training set data, wherein the data enhancement includes adding Gaussian noise, random flipping, brightness adjustment, and geometric transformation; After data augmentation, the processed data is fed into the tree classification model to perform the training steps of forward propagation, loss calculation, backpropagation, and weight update; The cross entropy loss function and learning rate scheduler are used to optimize the tree species classification model. After training, the classification accuracy of the tree species classification model is evaluated, and the tree species classification model with the highest classification accuracy is saved.

6. The tree species labeling and identification system based on deep learning according to claim 5, characterized in that: The method also includes testing the tree species classification model with the highest classification accuracy by the following steps: Standardize the image data of the test set, including adjusting the image size, center cropping, tensor quantization, and normalization; The processed test images are input into the tree species classification model, the prediction results are calculated using forward propagation, and the performance of the tree species classification model is evaluated by comparing the accuracy of the actual category and the predicted category; Generates an output file containing the classification results of each test sample, which is used to record the statistical results of tree species classification in the test set.

7. The tree species labeling and identification system based on deep learning according to claim 1, characterized in that: After training and testing, the tree species classification model is deployed on an edge computing device or server to realize offline tree species identification and real-time classification functions.

8. The tree species labeling and identification system based on deep learning according to claim 1, characterized in that: Use the tree species classification model to identify and count tree species in the input image and generate a visual output or data file containing the classification results, including: Input the canopy image to be classified into the trained tree species classification model; Calculate the probability distribution of the classification results of the input image and extract several categories with the highest classification probability; The classification results are visualized through the user interaction module, including the name of each category, the corresponding probability, and the number of trees.

Images