A pine wilt disease tree detection method based on unmanned aerial vehicle vision

Abstract

The application discloses a pine wood nematode disease tree detection method based on unmanned aerial vehicle vision, which comprises the following steps: acquiring pine forest images and geographic information collected by unmanned aerial vehicle aerial photography; obtaining a fused pine forest image by fusion; dividing the fused pine forest image into a plurality of image blocks with the same size; obtaining an image block set after data cleaning; labeling each image block to obtain a training set; constructing a target detection model based on a Color-ViT-YOLOv8 algorithm; replacing the backbone network in YOLOv8 with a ViT module and adding a Color module before the ViT module; processing the input image by using a channel selection and fusion technology of a multi-dimensional color space in the Color module; training the target detection model by using the training set to obtain a trained target detection model, which is used for pine wood nematode disease tree detection on a to-be-detected pine forest image. The application can solve the false detection and missed detection problems in the pine wood nematode disease tree detection in a complex scene.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of image recognition, and particularly relates to a pine wood nematode disease tree detection method based on unmanned aerial vehicle vision. BACKGROUND

[0002] Unmanned aerial vehicle remote sensing technology has developed rapidly and plays an important role in geological exploration, emergency rescue and agricultural management. In forest protection work, unmanned aerial vehicles can provide a new macroscopic perspective and an efficient data acquisition method. On the one hand, compared with traditional manual patrol, unmanned aerial vehicles have a wide aerial perspective and can cover a large area of pine forest area and obtain multi-dimensional geographic information. On the other hand, unmanned aerial vehicles are highly flexible and can adapt to various shooting needs and complex and variable environmental conditions. Therefore, developing disease tree detection technology based on unmanned aerial vehicle remote sensing information is crucial for efficient and economic forest protection work in the future.

[0003] Currently, some studies have used unmanned aerial vehicle remote sensing images to identify pine wood nematode disease trees. This method can collect images of a large area of pine forest in a short time, greatly improving the monitoring efficiency. However, in actual operation, image analysis still highly depends on human visual judgment, which is labor-intensive, highly subjective and limited in accuracy. In view of the above problems, it is urgent to develop a reliable, efficient and intelligent pine wood nematode disease tree detection method. SUMMARY

[0004] In order to solve the above problems in the prior art, the application provides a pine wood nematode disease tree detection method based on unmanned aerial vehicle vision. The technical problem to be solved by the application is solved by the following technical scheme:

[0005] A pine wood nematode disease tree detection method based on unmanned aerial vehicle vision, comprising:

[0006] obtaining pine forest images and geographic information collected by unmanned aerial vehicle aerial photography;

[0007] fusing all the pine forest images using the geographic information to obtain a fused pine forest image;

[0008] dividing the fused pine forest image into a plurality of image blocks of the same size; performing data cleaning on the plurality of image blocks of the same size to obtain an image block set; and labeling each image block in the image block set with a pine wood nematode disease tree label to obtain a training set according to the labeled image block set;

[0009] A target detection model based on the Color-ViT-YOLOv8 algorithm is constructed; wherein the target detection model replaces the backbone network in YOLOv8 with a ViT module, and a Color module is added before the ViT module; the Color module processes the input image by using a multi-dimensional color space channel selection and fusion technology;

[0010] The target detection model is trained using the training set, and a trained target detection model is obtained after a training stop condition is reached, for pine wilt disease tree detection on a to-be-tested pine forest image.

[0011] The beneficial effects of the present application are:

[0012] The present application embodiment proposes a pine wilt disease tree detection method based on unmanned aerial vehicle vision, which is realized based on a target detection model of the Color-ViT-YOLOv8 algorithm. The color model is combined with the ViT detection model and the YOLOv8 detection model, different color features are extracted for different development stages of the diseased trees, and the influence of different development stages of the diseased trees, changes in environmental light and different growth cycles of the trees is effectively coped with. At the same time, the model uses a complex multi-dimensional color space fusion technology, which can extract more color information and avoid the limitations of a single color space. It has a significant advantage in dealing with complex environments and different development stages of the diseased trees, and can solve the problems of false detection and missed detection in the current pine wilt disease tree detection under complex scenes. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1 A flowchart of a pine wilt disease tree detection method based on unmanned aerial vehicle vision provided by the present application embodiment is shown in the figure;

[0014] Figure 2 A process diagram of a pine wilt disease tree detection method based on unmanned aerial vehicle vision provided by the present application embodiment is shown in the figure;

[0015] Figure 3 A specific structure diagram of the target detection model provided by the present application embodiment is shown in the figure;

[0016] Figure 4 The output results of the pine wilt disease tree in nine channels are shown in the figure;

[0017] Figure 5 The figure is a comparison between the original pine wilt disease tree image and the R, H and A three-channel fusion results;

[0018] Figure 6 The figure is a pine wilt disease tree detection flowchart based on Color-ViT-YOLOv8. DETAILED DESCRIPTION

[0019] The application will be described in further detail below with reference to specific embodiments, but the embodiments of the application are not limited thereto.

[0020] The embodiment of the application provides a pine wood nematode disease tree detection method based on unmanned aerial vehicle vision, please refer to Figure 1 and Figure 2 The method can comprise the following steps:

[0021] S1, acquiring pine forest images and geographic information collected by unmanned aerial vehicle aerial photography;

[0022] Specifically, S1 comprises:

[0023] Acquiring the pine forest images and geographic information collected by aerial photography of the target pine forest region at different positions by the same model unmanned aerial vehicle flying at a preset height range from the ground within a preset time period.

[0024] The target pine forest region is a large area of pine forest region, the same model unmanned aerial vehicle continuously flies above the target pine forest region, aerial photography is performed for different positions, and for each time point of collection, the pine forest images and geographic information of the corresponding region can be collected. In the aerial photography process, the unmanned aerial vehicle can fly along a certain trajectory, so as to complete the traversal of each position of the target pine forest region.

[0025] The preset time period is a working time period of aerial photography, in order to ensure sufficient illumination and exclude irrelevant factors, the preset time period can be selected as part of the daytime. For example, in an optional embodiment, the preset time period is from 10:00 to 17:00; that is, the collection time point is from 10:00 to 17:00. Of course, a time period can also be selected between 10:00 and 17:00, such as from 10:00 to 12:00, which is reasonable.

[0026] In order to ensure the consistency of the resolution, size and other parameters of the collected images, the height from the ground when collecting the pine forest images should be kept within a preset height range , and the value can be set according to the requirement. For example, in an optional embodiment, the preset height range is 60 to 100 meters. That is . Of course, a fixed value can also be selected within the preset height range, such as 80 meters, so that the unmanned aerial vehicle can fly at the same height.

[0027] The collected pine forest images are stored in TIF format, the geographic information is stored in TFW format, and the image coordinate system is stored in WCG84.

[0028] S2, fusing all the pine forest images by using the geographic information to obtain a fused pine forest image;

[0029] The purpose of S2 is to obtain a fused pine forest image by fusing all the collected pine forest images of different positions, so as to represent a complete image of the target pine forest area. Specifically, S2 can include:

[0030] S21, obtaining image boundary coordinates of each pine forest image by using corresponding geographic information;

[0031] Specifically, the image boundary coordinates are calculated by analyzing the coordinate information of the TFW file.

[0032] S22, continuously performing image fusion processing on pine forest images adjacent to the image boundary coordinates until a fused pine forest image is obtained.

[0033] Wherein, the image fusion processing on two pine forest images adjacent to the image boundary coordinates includes:

[0034] 1) using the image boundary coordinates to determine whether the edge region of the two pine forest images adjacent to the image boundary coordinates appears discontinuity;

[0035] If the values of the image boundary coordinates of the two pine forest images are continuous and no gaps appear, the edge region does not appear discontinuity.

[0036] 2) if no discontinuity appears, the two pine forest images adjacent to the image boundary coordinates are spliced and fused according to the corresponding image boundary coordinates;

[0037] 3) if discontinuity appears, the two pine forest images adjacent to the image boundary coordinates are spliced and fused according to the corresponding image boundary coordinates, and then the pixel points of the discontinuous part are converted from three channels RGB to four channels RGBA, and the A channel transparency is defined as 0.

[0038] It can be understood that the image fusion processing can be performed on two pine forest images adjacent to the image boundary coordinates, and then a pine forest image adjacent to the image boundary coordinates is fused, and the process is repeated until a fused pine forest image is obtained.

[0039] S3, dividing the fused pine forest image into a plurality of image blocks of the same size; obtaining an image block set after data cleaning of the plurality of image blocks of the same size; and labeling each image block in the image block set with a pine wood nematode disease tree label, to obtain a training set according to the labeled image block set;

[0040] Dividing the fused pine forest image into a plurality of image blocks of the same size is to perform image framing on the fused pine forest image. The specific process includes:

[0041] According to a preset step length and a preset image block size, the fused pine forest image is overlaid and divided to obtain a plurality of divided image blocks.

[0042] The preset image block size can be represented as The preset step size can be expressed as ;

[0043] ;

[0044] in, This is a scaling factor, which is related to the size of the image patch; The length of the image block.

[0045] The preset image patch size can be set as needed, for example, it can be set to... ; The value can be 0.5, which sets the preset step size. .

[0046] In S3, after preprocessing the plurality of image blocks of the same size, an image block set is obtained, including:

[0047] 1) Delete noisy image blocks from among the multiple image blocks of the same size;

[0048] 2) Convert the retained image blocks from four-channel RGBA to three-channel RGB, and the converted image blocks constitute an image block set.

[0049] Because drone aerial images, after being segmented, will result in two types of image patches: noise-free image patches and noisy image patches, the segmented images need to undergo the data cleaning process described above to obtain a set of image patches.

[0050] Labeling of each image patch in the image patch set with pine wilt disease trees can be done using labeling tools and with the help of expert experience. The labeled image patch set can then be divided into a training set and a test set according to a certain ratio. The ratio can be selected as needed, for example, 8:2.

[0051] S4, construct an object detection model based on the Color-ViT-YOLOv8 algorithm; wherein the object detection model replaces the backbone network in YOLOv8 with a ViT module, and adds a Color module before the ViT module; the Color module processes the input image using channel selection and fusion techniques in a multi-dimensional color space;

[0052] This invention improves upon YOLOv8 as the basic framework by replacing the backbone network in YOLOv8 with the ViT module (Vision Transformer module) to enhance global feature extraction capabilities, and adds a Color module before the ViT module.

[0053] In the YOLOv8 model, the traditional CNN backbone network is mainly used to extract local features, which is efficient but has the problems of insufficient global information and weak long-distance dependence. The present application replaces the backbone network with a ViT module, which can significantly improve the global feature extraction capability. ViT captures long-distance dependence through self-attention mechanism, solves the limitations of traditional CNN (Convolutional Neural Network) in local feature extraction in complex background, and enhances the adaptability to different development stages of the diseased tree. In addition, ViT can effectively reduce background interference and improve the robustness of the model in complex environment, making the diseased tree detection more accurate and stable. The specific structure of the target detection model of the present application is shown in Figure 3 . Figure 3 The abbreviations of the modules in the prior art are described in the prior art, and will not be described one by one here. EfficientViT is one of the ViT modules, which is only used as an example here.

[0054] At present, the research on the detection method of pine wood nematode diseased trees has made certain progress, but the existing color model mainly relies on static feature extraction, and still has certain limitations, and technical breakthrough is urgently needed. The Color module of the embodiment of the present application can extract different color features for different development stages of the diseased tree, effectively cope with the influence of different development stages of the diseased tree, changes of environmental light and different growth cycles of the tree, and avoid the limitations of traditional static color model. At the same time, the Color module adopts the channel selection and fusion technology of multi-dimensional color space, which can extract more color information and avoid the limitations of single color space, and has significant advantages in dealing with complex environment.

[0055] Specifically, the Color module is used to:

[0056] 1) For the input image, the R channel output in the RGB color space is selected, the H channel output in the HSV color space is selected, and the A channel output in the LAB color space is selected;

[0057] The Color module highlights the features of the diseased tree and reduces background interference. In different color spaces, specific channels with large differences between foreground diseased trees and background are selected to adapt to the task of diseased tree detection. In the RGB color space, the R channel can capture the red information of the lesion, which has a significant effect in the detection of pine wood nematode diseased trees. In different development stages of the diseased tree, red-brown or yellowish changes may occur, and the R channel can effectively capture such color features. In the HSV color space, the H channel can capture stable hue information. The color of the diseased tree may change in brightness under different light conditions, and the H channel is not easily affected by brightness and can adapt to different light conditions. In the LAB color space, the A channel can capture red-green contrast characteristics, effectively distinguish non-diseased tree and diseased tree areas, reduce background interference, and improve the attention of the model to the diseased tree area.

[0058] Please refer to Figure 4 Understand, Figure 4 is the output result of the nine channels of the disease tree.

[0059] 2) The outputs of the R channel, the H channel and the A channel are fused after being normalized to the same channel value interval to obtain a new fused three-channel image;

[0060] The selected R, H and A channels are fused and output. The channel value ranges of different color spaces are different, and they are normalized to the same channel value interval [0, 255] and fused into a new three-channel image. Compared with a single color space, the fused image can enhance the features of the disease tree, highlight the disease area and reduce environmental interference.

[0061] Please refer to Figure 5 Understand, Figure 5 is the original image of the pine wood nematode disease tree (see Figure 5 (a) figure) and the fusion result of the R, H and A three channels (see Figure 5 (b) figure) comparison chart.

[0062] 3) The new fused three-channel image is converted into a grayscale image, and the OTSU (Nobuyuki Otsu method) method is used to adaptively calculate the best threshold value to obtain a binary image;

[0063] This step can further enhance the saliency of the disease tree area, reduce background interference, improve the discrimination of the disease area, and make the disease tree area clearer in the binary result. The OTSU method can adaptively select the best threshold value, and can stably segment the disease tree area clearly under different conditions.

[0064] The specific process can be seen in the OTSU related technology, which is not described in detail here.

[0065] 3) The binary image is subjected to morphological operation to correct the connectivity of the disease area and remove noise;

[0066] Morphological processing can correct the connectivity of the disease area, remove noise and improve segmentation accuracy.

[0067] 4) The image after morphological operation is combined with an area screening strategy to remove invalid targets, and finally retains the detection area that meets the conditions.

[0068] The area screening strategy can ignore areas with too small area. Specifically, a set area threshold can be used to achieve this. Through screening, invalid targets can be removed, and the remaining targets are retained as detection areas that meet the conditions.

[0069] The above process can be seen in 6 Figure 6A flowchart of a pine wilt disease tree detection process based on Color-ViT-YOLOv8. Figure 6 The image after the result identification is the final image after retaining the eligible detection area, which will be used as the input of the subsequent model structure.

[0070] S5, training the target detection model using the training set, obtaining a trained target detection model after reaching a training stop condition, for pine wilt disease tree detection on a pine forest image to be tested.

[0071] The training stop condition is:

[0072] In the continuous training rounds, the detection accuracy of the target detection model always changes by less than or equal to the threshold .

[0073] The formula is:

[0074] ;

[0075] Wherein, represents the accuracy of the model at the training round; is a preset threshold; if the performance of the model always changes by less than or equal to the threshold in the continuous training rounds, it means that the accuracy of the model on the training set tends to be stable, the training is stopped, and the current model is saved as the final training result. Wherein, and can be set as needed, such as 0.1%, and 50.

[0076] The specific training process can be understood by referring to the general model training process, which will not be described in detail here.

[0077] After obtaining the trained target detection model, pine wilt disease tree detection can be performed on the pine forest image to be tested, which can come from the test set or other pine forest images to be detected, thereby obtaining the detection result. Denoted as:

[0078] ;

[0079] Wherein, represents the pine forest image to be tested; represents model inference; represents the final detection result of the pine wilt disease tree; is the probability that the preselected frame is a disease tree; is a set disease tree probability threshold; when the predicted probability of the preselected frame is less than the disease tree probability threshold, it means that the preselected frame is not a disease tree. exceeding a threshold value When the value exceeds the threshold value, the tree is marked as a diseased tree. A Color-ViT-YOLOv8-based diseased tree detection flowchart is shown in Figure 6

[0080] In an optional embodiment, after the pine forest image to be tested is subjected to pine wood nematode diseased tree detection, the method further comprises:

[0081] If there is a difference in external environmental conditions between the pine forest image to be tested and the pine forest images in the training set, the detection result corresponding to the pine forest image to be tested is processed and used as new training data for iterative training of the target detection model, so as to update the model parameters.

[0082] The application can use the trained target detection model to infer the pine forest image to be tested to obtain a final detection result, and then determine the inferred pine forest image to be tested. If the time, illumination, weather, and other external environmental conditions are consistent with the original training set of images, the image is not made into a training set. If there is a difference, the detection result of the pine forest image to be tested is processed to make a new training set, and then the new training set is mixed with the original training set to repeatedly iterate and train the model and update the model parameters.

[0083] The application embodiment proposes a pine wood nematode diseased tree detection method based on unmanned aerial vehicle vision, and a target detection model based on a Color-ViT-YOLOv8 algorithm is implemented. The Color module and the ViT module are introduced into the YOLOv8 detection model, different color features are extracted for different development stages of diseased trees, and the influence of different development stages of diseased trees, changes in environmental illumination, and different growth cycles of trees is effectively coped with. At the same time, the model uses a complex multi-dimensional color space fusion technology, can extract more color information, avoids the limitation of a single color space, has a significant advantage in coping with complex environments and different development stages of diseased trees, and can solve the false detection and missed detection problems in the current pine wood nematode diseased tree detection under complex scenes.

[0084] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in the present specification.

[0085] ​The above merely provides the preferred embodiments of the application, and not intended to limit the protection scope of the application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the application shall fall within the protection scope of the application.

Claims

1. A method for detecting pine wilt disease in trees based on UAV vision, characterized in that, include: Acquire aerial images and geographic information of pine forests captured by drones; By using geographic information, all images of pine forests are merged to obtain a single merged image of the pine forest. The fused pine forest image is divided into multiple image blocks of the same size; after data cleaning of the multiple image blocks of the same size, an image block set is obtained; and each image block in the image block set is labeled with a tree with pine wilt disease, and a training set is obtained based on the labeled image block set; A target detection model based on the Color-ViT-YOLOv8 algorithm is constructed; wherein the target detection model replaces the backbone network in YOLOv8 with a ViT module, and a Color module is added before the ViT module; the Color module processes the input image using channel selection and fusion techniques in a multi-dimensional color space; The target detection model is trained using the training set. After the training stopping condition is met, the trained target detection model is obtained and used to detect pine wilt disease trees in the pine forest images to be tested. The Color module is used for: For the input image, select the R channel in the RGB color space for output, select the H channel in the HSV color space for output, and select the A channel in the LAB color space for output; The outputs of the R channel, H channel, and A channel are normalized to the same channel value range and then fused to obtain a new fused three-channel image. The fused new three-channel image is converted into a grayscale image, and the optimal threshold is adaptively calculated using the OTSU method to obtain a binarized image. The binarized image is subjected to morphological operations to correct the connectivity of the diseased area and remove noise; The image after morphological operations is combined with an area screening strategy to remove invalid targets and finally retain the detection areas that meet the criteria.

2. The method according to claim 1, characterized in that, The acquisition of pine forest images and geographic information collected by drone aerial photography includes: The system acquires aerial images and geographic information of pine forests at different locations within a target pine forest area by using the same type of drone flying within a preset time period and at a preset altitude.

3. The method according to claim 2, characterized in that, The preset time period is from 10:00 to 17:00; the preset altitude range is from 60 to 100 meters.

4. The method according to claim 1, characterized in that, The method of fusing all pine forest images using geographic information to obtain a single fused pine forest image includes: Using the corresponding geographic information, the image boundary coordinates of each pine forest image are obtained; The images of pine forests with adjacent boundary coordinates are continuously fused until a single fused pine forest image is obtained.

5. The method according to claim 4, characterized in that, Image fusion processing is performed on two adjacent pine forest images with adjacent image boundary coordinates, including: Using image boundary coordinates, determine whether there is a discontinuity in the edge region of two adjacent pine forest images; If no discontinuity is found, the two pine forest images with adjacent image boundary coordinates are stitched and merged according to their corresponding image boundary coordinates. If discontinuities occur, the two pine forest images with adjacent image boundary coordinates are stitched and merged according to their corresponding image boundary coordinates. The pixels of the discontinuous parts are converted from three-channel RGB to four-channel RGBA, and the transparency of the A channel is defined as 0.

6. The method according to claim 1, characterized in that, The fused pine forest image is divided into multiple image blocks of the same size, including: The fused pine forest image is overlaid and divided according to a preset step size and a preset image block size to obtain multiple image blocks.

7. The method according to claim 1, characterized in that, Data cleaning is performed on the multiple image blocks of the same size to obtain an image block set, including: Delete noisy image blocks from among the plurality of image blocks of the same size; The retained image blocks are converted from four-channel RGBA to three-channel RGB, and the converted image blocks constitute an image block set.

8. The method according to claim 1, characterized in that, The training termination condition is as follows: In continuous Throughout the training epochs, the change in the detection accuracy of the target detection model is always less than or equal to the threshold. .

9. The method according to claim 1, characterized in that, After detecting pine wilt disease trees in the images of the pine forest to be tested, the method further includes: If the external environmental conditions of the pine forest image to be tested differ from those of the pine forest images in the training set, the detection results corresponding to the pine forest image to be tested are processed and used as new training data for iterative training of the target detection model, thereby updating the model parameters.

Images