Tree disease detection method based on multispectral and visible light images from UAV

Abstract

The present invention provides a tree disease detection method based on multispectral and visible light images carried by unmanned aerial vehicles, comprising the following steps: using unmanned aerial vehicles to collect data to obtain multispectral images; performing feature matching and alignment across images based on the multispectral images, and establishing a mosaic transformation model; using the mosaic transformation model and multi-band mixing technology to mosaic the multispectral images into an overall remote sensing image; obtaining a single tree segmentation result map based on the overall remote sensing image, and then fusing the single tree segmentation result map with the multispectral image to obtain a multispectral single tree distribution map; slicing the multispectral single tree distribution map and calculating the average radiation eigenvalue of each tree in different bands, and using the average radiation eigenvalue to judge the growth status and specific disease of each tree. The present invention effectively overcomes the problem of loss in calculation of non-tree parts in multispectral images, and by fusing the single tree segmentation result map and the multispectral image, efficient single tree disease detection is achieved, which greatly reduces labor costs.

Description

Technical Field

[0001] The present invention relates to the technical field of tree disease detection, and in particular to a tree disease detection method based on multi-spectral and visible light images carried by unmanned aerial vehicles. Background Art

[0002] Multispectral disease detection technology can identify crop diseases and damages at an early stage through spectral data in different bands. This is faster and more accurate than traditional visual inspections, and helps to take preventive measures in a timely manner and reduce losses. Multispectral disease detection technology can cover large areas of farmland, provide a comprehensive assessment of crop health, and improve the efficiency of agricultural production. In addition, this technology can also accurately locate the area where the disease occurs, reduce the amount and scope of pesticide use, thereby reducing environmental pollution and promoting sustainable agricultural development. In the past, disease detection tasks were basically traditional visual inspections. This method relies on manual means, which is costly, inefficient, and cannot accurately locate specific trees.

[0003] Due to the booming development of the drone industry in recent years, many tasks have been proposed in the field to use drone-mounted cameras to detect disease symptoms. At present, the algorithms for disease detection are roughly divided into traditional target detection algorithms and deep learning-based algorithm detection. Mainstream deep learning is divided into YOLO-based models and attention-based models. We found that these studies generally use complete multispectral data, which will lead to the following problems: 1) The direct use of complete multispectral images in the model will lead to additional computing power loss and reduce the calculation effect of the model. 2) The regional-level growth status evaluation is too broad, and the disease detection of specific individual trees still requires manual judgment, resulting in additional labor costs. Summary of the invention

[0004] In order to overcome the shortcomings of the prior art, the purpose of the present invention is to provide a tree disease detection method based on drone-borne multispectral and visible light images, which effectively overcomes the problem of calculation loss of non-tree parts in multispectral images, and realizes efficient single tree disease detection by fusing single tree segmentation result images and multispectral images, greatly reducing labor costs.

[0005] To achieve the above object, the present invention provides the following scheme: a tree disease detection method based on multispectral and visible light images carried by unmanned aerial vehicles, comprising the following steps:

[0006] The data of trees were collected using a drone equipped with a multispectral camera, and a multispectral image with four bands was obtained;

[0007] Performing feature point matching and feature alignment across images based on the multispectral image, and then using the matched feature points to establish a stitching transformation model between images;

[0008] Using the stitching transformation model to stitch the aligned multispectral images into an overall remote sensing image, and using multi-band blending technology to smooth the transition and reduce the visibility of the stitching edge;

[0009] Performing single-tree segmentation on the visible light image in the overall remote sensing image to obtain a single-tree segmentation result map of each tree, and then fusing the single-tree segmentation result map with the multispectral image to obtain a multispectral single-tree distribution map;

[0010] The multispectral single tree distribution map is sliced ​​to obtain a plurality of slice maps of fixed size, and the average radiation characteristic value of each tree in different bands is calculated based on the plurality of slice maps, and then the average radiation characteristic value is used to judge the growth condition and specific disease of each tree.

[0011] Optionally, performing feature point matching and feature alignment across images based on the multispectral image includes:

[0012] Using a feature extraction algorithm to detect feature points in the multispectral image, and generating a feature descriptor for feature matching for each feature point;

[0013] The KNN algorithm is used to match feature points, and the feature similarity points are determined by comparing the distances between corresponding feature descriptors; in the matching process, the cross-validation method and the random extraction consistency algorithm are used to eliminate and filter the wrong matching points;

[0014] Based on the matched feature points, the images of different bands are geometrically corrected and aligned, so that the multispectral images are aligned at the pixel level.

[0015] Optionally, a stitching transformation model between images is established using the matched feature points, including:

[0016] Based on the matched feature point pairs, an initial transformation model is calculated using the least squares method, and then the initial transformation model is optimized using a random sampling consensus algorithm;

[0017] Based on the optimized initial transformation model, the transformation parameters of the initial transformation model are adjusted using a nonlinear optimization method, and the image alignment quality is checked and corrected to obtain a spliced ​​transformation model.

[0018] Optionally, using the stitching transformation model to stitch the aligned multispectral images into an integrated remote sensing image includes:

[0019] The image of each band is geometrically corrected and color balanced by using the stitching transformation model to complete image preprocessing;

[0020] All pre-processed images are transformed into the same geographic reference frame and projected into the same coordinate reference system, and then the resolution of the images is unified using resampling technology;

[0021] The overlapping images are stitched together using an automatic stitching algorithm, and multi-band fusion is performed on the overlapping areas after stitching. Image processing software is then used to perform detail evaluation and local correction on the fused images to obtain the overall remote sensing image.

[0022] The overall remote sensing image is saved as a high-resolution image file and format converted so as to be read by different analysis tools.

[0023] Optionally, utilize multi-band blending technology to smooth transitions and reduce the visibility of splice edges, including:

[0024] In the coordinate reference system, each image of the same resolution is decomposed into images of different frequency bands, and then the images of different frequency bands are decomposed layer by layer using a multi-scale analysis technique, so that different details in the images of different frequency bands are separated into corresponding frequency bands;

[0025] Introducing the Laplacian pyramid into the automatic stitching algorithm, and fusing the Laplacian pyramid into each image to establish a detail level; wherein, in the overlapping area after the image stitching, the corresponding frequency band level characteristics are weighted fused;

[0026] In the image stitching process, the low-frequency image and high-frequency details of the Laplacian pyramid are fused layer by layer to make the image features of the stitching transition area transition evenly;

[0027] The inverse pyramid transform operation is used to reconstruct the layer-by-layer fused image back to the original image space and merge the high-frequency and low-frequency parts to retain the detailed features of each band and reduce the edge seams;

[0028] Use manual or filtering techniques to adjust small color differences and rough edge areas in detail.

[0029] Optionally, performing single tree segmentation on the visible light image in the overall remote sensing image to obtain a single tree segmentation result map of each tree includes:

[0030] Performing color correction, noise removal, and spectral band standardization on the visible light image to unify the image scale, and then calculating the vegetation index and extracting texture information based on the processed visible light image to achieve the recognition of the tree crown structure;

[0031] The tree area is identified by using the threshold segmentation method based on the vegetation index. Then, starting from each initial threshold segmentation area, similar area pixels are merged in a recursive manner, and the contour structure is found by edge detection. Single tree segmentation is performed based on the contour structure to obtain an initial segmentation map.

[0032] Integrate and divide the relevant data for generating the initial segmentation map into a training set and a validation set, use the training set to perform deep learning to obtain a pre-trained model, and then use the validation set to adjust the pre-trained model and the parameters of the pre-trained model to perform model optimization to obtain a segmentation model;

[0033] The overall remote sensing image is input into the segmentation model for forward propagation, the category to which each pixel belongs is output and a segmentation mask is generated. Based on the segmentation mask, morphological operations are used to refine the crown edge, segmentation optimization tools are used to enhance edge clarity, and connected component analysis is used to eliminate abnormal segmentation areas, finally obtaining a single tree segmentation result map for each tree.

[0034] Optionally, fusing the single tree segmentation result map with the multispectral image to obtain the multispectral single tree distribution map includes:

[0035] Using geographic registration technology to align the coordinates and resolutions of the single tree segmentation result map and the multispectral image, and then extracting the crown mask in the single tree segmentation result map and the relevant band information in the multispectral image;

[0036] Extracting multispectral pixel values ​​corresponding to individual trees in the multispectral image based on the tree crown mask, and then integrating the tree crown mask and the multispectral pixel values ​​into a data matrix to construct a spectral characteristic map of each tree;

[0037] The tree IDs are marked, and the spectral characteristic map is integrated with the geographic information of the corresponding single tree to obtain a multi-spectral single tree distribution map.

[0038] Optionally, slicing the single tree distribution map to obtain a plurality of slice maps of fixed sizes, and calculating the average characteristic value of radiation of each tree in different bands based on the plurality of slice maps, including:

[0039] Determine a slice size and traverse each tree in the single tree distribution map, and generate a plurality of slices in the multispectral image according to the central coordinates of the single tree distribution map;

[0040] The crown mask is used to extract relevant band data in each slice and calculate the average radiation eigenvalues ​​in different bands. The average radiation eigenvalues, single tree IDs and single tree positions are associated to obtain single tree comprehensive results, which are then formatted and saved.

[0041] Optionally, the average radiation characteristic value is used to determine the growth status and specific disease condition of each tree, including:

[0042] The radiation average characteristic values ​​of all bands are standardized and integrated to obtain a standard characteristic vector, and the standard characteristic vector is combined with the healthy tree sample data to obtain an analysis data set;

[0043] Setting a health state and a health state threshold in the analysis model, and training the analysis model using the analysis data set and the abnormality recognition algorithm to obtain a health state model;

[0044] The single tree comprehensive results are input into the health status model to determine the growth status and specific disease symptoms of each tree, and the determination results are combined with geographic information to generate a spatial risk distribution map.

[0045] The present invention discloses the following technical effects by providing a tree disease detection method based on multi-spectral and visible light images carried by unmanned aerial vehicles:

[0046] 1. The present invention can effectively match the cross-image feature points of multispectral images of different bands and accurately align them at the pixel level to ensure the correct fusion and utilization of multi-band information. This process is the basis of the automated pathology detection system and the prerequisite for subsequent accurate analysis. It can minimize errors and improve the sensitivity and accuracy of detection.

[0047] 2. The use of multi-band mixing technology can effectively retain the detailed information of different image sources in the process of stitching multiple sets of images, reduce stitching traces, ensure the integrity and natural transition of the stitched images, and generate high-quality panoramic broad-leaved tree forest remote sensing images. This provides a reliable data basis for subsequent image analysis and tree health monitoring.

[0048] 3. Segmenting individual trees in multi-band images to obtain segmentation results for each tree is an important step in tree disease detection and provides solid data support for accurate plant disease monitoring and tree health assessment. This process combines traditional image processing technology with modern deep learning methods to cope with complex forest environments.

[0049] 4. Deeply integrating the results of single tree segmentation with multispectral images can effectively integrate information from different bands into each tree, making subsequent tree disease analysis and the development of decision support systems more accurate and efficient.

[0050] 5. Calculating and analyzing the radiation characteristics of each tree in different bands can provide accurate data support for tree disease monitoring and ecological environment assessment. At the same time, based on the calculated radiation characteristics, a fixed-size slice analysis is performed on each tree, and the multi-spectral analysis process is refined to accurately calculate and evaluate the spectral characteristics of each tree, laying the foundation for intelligent and precise forestry disease monitoring.

[0051] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0053] Figure 1 A flow chart of a method provided by an embodiment of the present invention;

[0054] Figure 2 A schematic diagram of the fusion process of a single tree segmentation result image and a multispectral image provided by an embodiment of the present invention;

[0055] Figure 3 A schematic diagram of a process for calculating the average characteristic value of radiation provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0058] like Figure 1-3 As shown, the present invention provides a tree disease detection method based on multi-spectral and visible light images carried by unmanned aerial vehicles, comprising the following steps:

[0059] 1. Use a drone equipped with a multispectral camera to collect tree data and obtain a multispectral image with 4 bands.

[0060] Choose a multispectral camera with 4 bands. Common bands include blue light, green light, red light, and near infrared light (NIR). These bands help identify the health of vegetation. Perform spectral calibration on the multispectral camera before flight, using a standard grayscale plate or calibration plate to ensure the accuracy of reflectance data. Perform black plane calibration and white plane calibration to offset the impact of changing light conditions on measurements.

[0061] The flight route is formulated according to the geographical size and characteristics of the coverage area to ensure that each flight can cover the entire target area. When conditions permit, a fixed-route drone is used to collect spectral data at a flight altitude of 60-80 meters, with a 60% overlap in the heading and 60% in the lateral direction to ensure the quality of subsequent image stitching.

[0062] During the flight, the multispectral camera will continuously capture multispectral images on the ground, which will be automatically saved to the camera's storage device. After the flight mission is completed, the aircraft will land safely and download the image data stored in the multispectral camera. According to the platform specifications, the image data will be managed and classified, usually stored by location, time, and band. At the same time, the original image will be preliminarily processed to check the image quality and clarity, and retaken if necessary.

[0063] 2. Based on the multispectral image, feature point matching and feature alignment are performed across images, and then a stitching transformation model between images is established using the matched feature points.

[0064] 2.1 The process of feature point matching and feature alignment is:

[0065] 2.11 Use feature extraction algorithms to detect feature points in the multispectral image. Such algorithms include SIFT (Scale Invariant Feature Transform), ORB (Rapid Feature Points), SURF (Speeded Up Robust Features), etc. These algorithms can detect stable feature points in different images. These feature points should have good repeatability, that is, they should remain consistent under different viewing angles and lighting conditions.

[0066] 2.12 Generate a feature descriptor for feature matching for each feature point. These descriptors are mathematical representations of the feature points and will be used for matching later.

[0067] 2.13 Use the KNN algorithm to match feature points, and determine feature similarities by comparing the distances between corresponding feature descriptors; in the matching process: 1) Use algorithms such as FLANN (Fast Nearest Neighbor Search) to speed up the matching process, especially when dealing with a large number of feature points; 2) Use ratio tests, cross-validation methods and other techniques to eliminate mismatched points to ensure that only high-quality matching pairs are retained. 3) Apply RANSAC (Random Sampling Consensus Algorithm) to further filter out mismatches, especially in the case of perspective changes or noise, to improve the robustness of matching.

[0068] 2.14 Based on the matched feature points, geometric correction and alignment are performed on the images of different bands so that the multispectral images are aligned at the pixel level. The alignment results can then be visualized and the accuracy of the feature point matching can be verified by superimposing the images of different bands (for example, using pseudo-color display).

[0069] 2.2 Use the matched feature points to establish a stitching transformation model between images, including:

[0070] 2.21 Based on the matched feature point pairs, the initial transformation model is calculated using the least squares method, and then the random sampling consistency algorithm is used to optimize the initial transformation model.

[0071] 1) The initial transformation model can be an affine transformation model or a homography transformation model; where:

[0072] Affine transformation model: Applicable to linear transformations such as rotation, scaling, and translation. It is selected if the image deformation can be described by the above simple transformations.

[0073] Homography transformation model: Applicable to situations where there are changes in perspective and perspective projection, especially the possible tilt and height changes in drone images.

[0074] 2) The random sampling consensus algorithm randomly selects subset feature point pairs, fits the transformation model, and verifies the model's adaptability to the overall data set. The model with the least number of mismatched pairs is selected as the best model.

[0075] 2.22 Based on the optimized initial transformation model, the transformation parameters of the initial transformation model are adjusted using a nonlinear optimization method, and the image alignment quality is checked and corrected to obtain a spliced ​​transformation model.

[0076] 3. Using the stitching transformation model to stitch the aligned multispectral images into an overall remote sensing image, and using multi-band blending technology to smooth the transition and reduce the visibility of the stitching edges.

[0077] 3.1 Stitching into a whole remote sensing image, including:

[0078] 3.11 The stitching transformation model is used to perform geometric correction and color balance on the images of each band to reduce the impact of illumination changes and complete image preprocessing.

[0079] 3.12 Transform all preprocessed images into the same geographic reference frame and project them into the same coordinate reference system, then use resampling techniques (such as bilinear interpolation and bicubic interpolation) to unify the image resolution and ensure pixel grid alignment for subsequent fusion.

[0080] 3.13 Use an automatic stitching algorithm (such as the Stitcher API in the OpenCV library) to stitch overlapping images, and perform multi-band fusion on the overlapping areas after stitching. Then use image processing software to perform detail evaluation and local correction on the fused image to obtain the overall remote sensing image.

[0081] 3.14 Save the overall remote sensing image as a high-resolution image file and convert the format so as to be read by different analysis tools.

[0082] 3.2 Utilize multi-band blending techniques to smooth transitions and reduce the visibility of splicing edges, including:

[0083] 3.21 In the coordinate reference system, each image of the same resolution is decomposed into images of different frequency bands, and then the images of different frequency bands are decomposed layer by layer using multi-scale analysis technology, so that different details in the images of different frequency bands are separated into corresponding frequency bands.

[0084] 3.22 Introduce the Laplacian pyramid in the automatic stitching algorithm, and fuse the Laplacian pyramid into each image to establish a detail level; wherein, in the overlapping area after the image stitching, the corresponding frequency band level characteristics are weighted fused; usually higher weights are used in the detail layer and lower weights are used in the background.

[0085] 3.23 During the image stitching process, the low-frequency image and high-frequency details of the Laplacian pyramid are fused layer by layer. This combination process is gradual, from the bottom layer to the top layer of the pyramid, to ensure that the image features in the stitching transition area are evenly transitioned.

[0086] 3.24 The inverse pyramid transform operation is used to reconstruct the layer-by-layer fused image back to the original image space and merge the high-frequency and low-frequency parts to retain the detailed features of each band and reduce the edge seams.

[0087] 3.25 Use manual or filtering technology to make detailed adjustments to small color differences and uneven edge areas to ensure that the fused image has a natural transition and maintains consistent details and tones.

[0088] 4. If Figure 2 As shown, the visible light image in the overall remote sensing image is segmented into individual trees to obtain an individual tree segmentation result map of each tree, and then the individual tree segmentation result map is fused with the multispectral image to obtain a multispectral individual tree distribution map;

[0089] 4.1 Obtain the single tree segmentation result map for each tree, including:

[0090] 4.11 In order to improve the accuracy of segmentation, the visible light image is first subjected to color correction, noise removal and spectral band standardization processing to unify the image scale, and then the vegetation index is calculated and texture information is extracted based on the processed visible light image to realize the recognition of the tree crown structure; wherein:

[0091] Vegetation index calculation: Utilize the characteristics of multispectral images to calculate commonly used vegetation indices such as NDVI (Normalized Difference Vegetation Index). NDVI can effectively distinguish between vegetated and non-vegetated areas.

[0092] Texture feature analysis: Apply texture analysis algorithms (such as Gabor filter and gray-level co-occurrence matrix) to extract different texture information to help identify the crown structure.

[0093] 4.12 Perform image segmentation

[0094] Threshold segmentation: Simple threshold segmentation based on the vegetation index map can help quickly identify obvious tree areas.

[0095] Region growing method: Starting from each initial threshold segmentation region, it grows recursively or iteratively and merges similar neighboring pixels.

[0096] Edge detection and segmentation: Use edge detection (such as Canny edge detection) to find the contour structure, and then perform single tree segmentation based on the contour structure to obtain the initial segmentation map.

[0097] 4.13 Depth Segmentation

[0098] The relevant data for generating the initial segmentation map are integrated and divided into a training set and a validation set, and the training set is used for deep learning to obtain a pre-trained model; large data sets (such as ImageNet, COCO) are selected to pre-train weights, which contain rich feature information, help accelerate convergence and improve segmentation effects.

[0099] The validation set is then used to adjust the pre-trained model (involving freezing some initial layers and only training specific layers or newly added layers to complete the adaptation of specific tasks) and the parameters of the pre-trained model (such as learning rate, batch size, etc.) to optimize the model and obtain a segmentation model; a suitable loss function can also be selected, such as cross entropy loss combined with Dice coefficient, to improve segmentation accuracy.

[0100] The overall remote sensing image is input into the segmentation model for forward propagation, the category (crown or background) to which each pixel belongs is output and a fine segmentation mask is generated. Based on the segmentation mask, morphological operations are used to refine the crown edge, segmentation optimization tools are used to enhance edge clarity, and connected component analysis is used to eliminate abnormal segmentation areas (such as segmentation areas with unreasonable size and shape filtering), and finally a single tree segmentation result map of each tree is obtained.

[0101] 4.2 Obtaining the multi-spectral single tree distribution map, including:

[0102] 4.21 Standardize the coordinates and resolution of the single tree segmentation result map and the multispectral image using geo-registration techniques to ensure perfect pixel alignment between the images. The alignment between the stitched multispectral image and the segmentation result image can be re-evaluated to ensure that there is no scale or rotation offset.

[0103] 4.22 Extract the crown mask in the single tree segmentation result image, and the mask is displayed in the form of a binary image, with the tree area as 1 (or True) and the background as 0 (or False). Then extract the relevant band information (such as red edge, near infrared) in the multispectral image according to the task requirements; these band information can effectively provide information indicating the health status.

[0104] 4.23 extracting the multispectral pixel value of the corresponding single tree in the multispectral image based on the tree crown mask; the extraction process is equivalent to performing a pixel-by-pixel logical 'AND' operation on the tree crown mask and the multispectral image;

[0105] The tree crown mask and the multispectral pixel values ​​are then integrated into a data matrix, which generally involves arranging and extracting the values ​​of different bands to form a multispectral feature vector of a single tree, and then constructing a spectral characteristic map of each tree;

[0106] Label tree IDs, automatically or manually confirm the independence of each tree segmentation result, and mark its ID in the spatial data to facilitate subsequent analysis;

[0107] The spectral characteristic map is integrated with the geographic information of the corresponding single tree to obtain a complete multi-spectral single tree distribution map, which includes the geographic location of each tree and its spectral characteristic data.

[0108] 5. If Figure 3 As shown, the multispectral single tree distribution map is sliced ​​to obtain a plurality of fixed-size slice maps, and the average radiation characteristic value of each tree in different bands is calculated based on the plurality of slice maps, and then the average radiation characteristic value is used to judge the growth condition and specific disease condition of each tree.

[0109] 5.1 Calculate the average characteristic value of radiation, including:

[0110] 5.11 Determine the slice size, which should be able to accommodate and cover the maximum projected area of ​​a single tree. At the same time, the slice size should be adapted to the image resolution and the density of individual trees to ensure that the slices cover the target without overlapping too much.

[0111] 5.12 Traverse each tree in the single tree distribution map, and for each ID, determine its center point or positioning point. Generate multiple slices in the multispectral image according to the center coordinates of the single tree distribution map. The slice can usually be a square or rectangular area to ensure that the slice contains a complete crown part. Edge problems can be solved by mirror filling or skipping.

[0112] 5.13 Use the tree crown mask to extract the relevant band data in each slice and calculate the average characteristic value of radiation in different bands; the role of the mask here is to remove the background noise and only process the multispectral pixels of the trees.

[0113] 5.14 Associate the radiation average characteristic value, the single tree ID and the single tree position to obtain the single tree comprehensive result, and then format and save the single tree comprehensive result.

[0114] 5.2 Determine the growth status and specific disease symptoms of each tree, including:

[0115] The radiation average characteristic values ​​of all bands are standardized and integrated to obtain a standard characteristic vector, and the standard characteristic vector is combined with the healthy tree sample data to obtain an analysis data set;

[0116] The health status (i.e., healthy, sub-healthy, unhealthy, etc.) and health status threshold are set in the analysis model, and the analysis model is trained using the analysis data set and the abnormality recognition algorithm to obtain a health status model; the abnormality recognition algorithm compares the predicted health index or probability with a preset threshold, and marks trees that exceed or are below the threshold as abnormal, indicating that a disease may exist.

[0117] The comprehensive results of individual trees are input into the health status model to determine the growth status and specific disease symptoms of each tree, and the determination results are combined with geographic information to generate a spatial risk distribution map, which can help quickly identify areas where the disease is concentrated.

[0118] Therefore, the present invention provides a tree disease detection method based on drone-borne multispectral and visible light images, which effectively overcomes the problem of calculation loss of non-tree parts in multispectral images. By fusing single tree segmentation result images and multispectral images, efficient single tree disease detection is achieved, greatly reducing labor costs.

[0119] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0120] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A tree disease detection method based on multispectral and visible light images carried by unmanned aerial vehicles, characterized in that: The following steps are involved: The data of trees were collected using a drone equipped with a multispectral camera, and a multispectral image with four bands was obtained; Performing feature point matching and feature alignment across images based on the multispectral image, and then using the matched feature points to establish a stitching transformation model between images; Using the stitching transformation model to stitch the aligned multispectral images into an overall remote sensing image, and using multi-band blending technology to smooth the transition and reduce the visibility of the stitching edge; Performing single-tree segmentation on the visible light image in the overall remote sensing image to obtain a single-tree segmentation result map of each tree, and then fusing the single-tree segmentation result map with the multispectral image to obtain a multispectral single-tree distribution map; Slicing the multispectral single tree distribution map to obtain a plurality of slice maps of fixed size, calculating the average radiation characteristic value of each tree in different bands based on the plurality of slice maps, and then using the average radiation characteristic value to determine the growth status and specific disease condition of each tree; Using the stitching transformation model to stitch the aligned multispectral images into an overall remote sensing image includes: The image of each band is geometrically corrected and color balanced by using the stitching transformation model to complete image preprocessing; All pre-processed images are transformed into the same geographic reference frame and projected into the same coordinate reference system, and then the resolution of the images is unified using resampling technology; The overlapping images are stitched together using an automatic stitching algorithm, and multi-band fusion is performed on the overlapping areas after stitching. The fused images are then evaluated for details and corrected locally using image processing software to obtain an overall remote sensing image. The integrated remote sensing image is saved as a high-resolution image file and converted into a format for reading by different analysis tools; Leverages multi-band blending technology to smooth transitions and reduce the visibility of splice edges, including: In the coordinate reference system, each image of the same resolution is decomposed into images of different frequency bands, and then the images of different frequency bands are decomposed layer by layer using a multi-scale analysis technique, so that different details in the images of different frequency bands are separated into corresponding frequency bands; Introducing the Laplacian pyramid into the automatic stitching algorithm, and fusing the Laplacian pyramid into each image to establish a detail level; wherein, in the overlapping area after the image stitching, the corresponding frequency band level characteristics are weighted fused; In the image stitching process, the low-frequency image and high-frequency details of the Laplacian pyramid are fused layer by layer to make the image features of the stitching transition area transition evenly; The inverse pyramid transform operation is used to reconstruct the layer-by-layer fused image back to the original image space and merge the high-frequency and low-frequency parts to retain the detailed features of each band and reduce the edge seams; Use manual or filtering techniques to adjust small color differences and rough edge areas in detail.

2. The tree disease detection method based on multispectral and visible light images onboard drones according to claim 1 is characterized in that: Performing feature point matching and feature alignment across images based on the multispectral image includes: Using a feature extraction algorithm to detect feature points in the multispectral image, and generating a feature descriptor for feature matching for each feature point; The KNN algorithm is used to match feature points, and the feature similarity points are determined by comparing the distances between corresponding feature descriptors; in the matching process, the cross-validation method and the random extraction consistency algorithm are used to eliminate and filter the wrong matching points; Based on the matched feature points, the images of different bands are geometrically corrected and aligned, so that the multispectral images are aligned at the pixel level.

3. The tree disease detection method based on multispectral and visible light images onboard drones according to claim 2 is characterized in that: The image stitching transformation model is established using the matched feature points, including: Based on the matched feature point pairs, an initial transformation model is calculated using the least squares method, and then the initial transformation model is optimized using a random sampling consensus algorithm; Based on the optimized initial transformation model, the transformation parameters of the initial transformation model are adjusted using a nonlinear optimization method, and the image alignment quality is checked and corrected to obtain a spliced ​​transformation model.

4. The tree disease detection method based on multi-spectral and visible light images onboard drones according to claim 3 is characterized in that: Performing single tree segmentation on the visible light image in the overall remote sensing image to obtain a single tree segmentation result map of each tree, including: Performing color correction, noise removal, and spectral band standardization on the visible light image to unify the image scale, and then calculating the vegetation index and extracting texture information based on the processed visible light image to achieve the recognition of the tree crown structure; The tree area is identified by using the threshold segmentation method based on the vegetation index. Then, starting from each initial threshold segmentation area, similar area pixels are merged in a recursive manner, and the contour structure is found by edge detection. Single tree segmentation is performed based on the contour structure to obtain an initial segmentation map. Integrate and divide the relevant data for generating the initial segmentation map into a training set and a validation set, use the training set to perform deep learning to obtain a pre-trained model, and then use the validation set to adjust the pre-trained model and the parameters of the pre-trained model to perform model optimization to obtain a segmentation model; The overall remote sensing image is input into the segmentation model for forward propagation, the category to which each pixel belongs is output and a segmentation mask is generated. Based on the segmentation mask, morphological operations are used to refine the crown edge, segmentation optimization tools are used to enhance edge clarity, and connected component analysis is used to eliminate abnormal segmentation areas, finally obtaining a single tree segmentation result map for each tree.

5. The tree disease detection method based on multi-spectral and visible light images onboard drones according to claim 4 is characterized in that: The single tree segmentation result map is fused with the multispectral image to obtain the multispectral single tree distribution map, including: Using geographic registration technology to align the coordinates and resolutions of the single tree segmentation result map and the multispectral image, and then extracting the crown mask in the single tree segmentation result map and the relevant band information in the multispectral image; Extracting multispectral pixel values ​​corresponding to individual trees in the multispectral image based on the tree crown mask, and then integrating the tree crown mask and the multispectral pixel values ​​into a data matrix to construct a spectral characteristic map of each tree; The tree IDs are marked, and the spectral characteristic map is integrated with the geographic information of the corresponding single tree to obtain a multi-spectral single tree distribution map.

6. The tree disease detection method based on multi-spectral and visible light images onboard drones according to claim 5 is characterized in that: Slicing the single tree distribution map to obtain a plurality of slice maps of fixed size, and calculating the average characteristic value of radiation of each tree in different bands based on the plurality of slice maps, including: Determine a slice size and traverse each tree in the single tree distribution map, and generate a plurality of slices in the multispectral image according to the central coordinates of the single tree distribution map; The crown mask is used to extract relevant band data in each slice and calculate the average radiation eigenvalues ​​in different bands. The average radiation eigenvalues, single tree IDs and single tree positions are associated to obtain single tree comprehensive results, which are then formatted and saved.

7. The tree disease detection method based on multi-spectral and visible light images onboard drones according to claim 6, characterized in that: The radiation average characteristic value is used to determine the growth status and specific disease symptoms of each tree, including: The radiation average values ​​of all bands are standardized and integrated to obtain a standard feature vector, and the standard feature vector is combined with healthy tree sample data to obtain an analysis data set; Setting a health state and a health state threshold in the analysis model, and training the analysis model using the analysis data set and the abnormality recognition algorithm to obtain a health state model; The single tree comprehensive results are input into the health status model to determine the growth status and specific disease symptoms of each tree, and the determination results are combined with geographic information to generate a spatial risk distribution map.

Images