Agricultural crop pest detection method and system based on image segmentation

Abstract

The invention relates to the technical field of image recognition, in particular to an agricultural crop disease and pest detection method and system based on image segmentation, and the method comprises the steps: collecting a vertical view angle image and an inclined view angle image of a tea garden through an unmanned aerial vehicle, carrying out the image registration through SIFT feature extraction, KNN matching and RANSAC filtering, and carrying out the alignment to a reference image coordinate system. And segmenting the foreground region by adopting a U-Net algorithm, and performing post-processing optimization. Positioning a disease area through a color detection method, extracting disease features, and identifying a disease stage by using a CNN algorithm; and using a YOLOv3 algorithm to identify insect body areas, extracting insect pest features, and evaluating insect pest degrees. And finally, based on the disease stage and the pest degree, using an LSTM algorithm to predict the pest diffusion trend, and generating a space thermodynamic diagram according to the reference image coordinate system, thereby improving the detection precision and timeliness, and providing technical support for intelligent agricultural prevention and control.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and particularly to a method and system for detecting agricultural crop diseases and pests based on image segmentation. Background Art

[0002] As the second most consumed beverage crop globally after water, the cultivation process of tea trees depends on high-temperature and high-humidity environments, creating favorable conditions for the breeding of diseases and pests, resulting in a complex and diverse range of tea tree diseases. In the large-scale intensive planting mode, once a disease breaks out and is not timely prevented and controlled, it is extremely easy to cause irreversible economic losses to the tea garden ecosystem.

[0003] Tea tree diseases and pests are concealed. Although some existing studies have achieved remarkable results in health assessment based on image recognition technology, since diseased leaf images are easier to obtain than pest images and have a more direct impact on quality, the research mainly focuses on disease recognition. And although these image recognition technologies such as support vector machines and random forest algorithms perform well in terms of feature extraction efficiency and recognition rate, they do not fully consider the dynamic change characteristics of leaf color during the disease evolution process. Then, the detection of tea tree diseases and pests is easily restricted by various factors such as light intensity and background, and the accuracy of pest and disease image segmentation in complex backgrounds still needs to be improved. In addition, these existing studies are mostly limited to static single-point detection, lacking the ability to perform spatio-temporal modeling of the spread path of diseases and pests, resulting in lagging prevention and control strategies and being difficult to achieve precise early warning and prevention.

[0004] In order to further improve the accuracy of tea tree disease and pest detection, a method and system for detecting agricultural crop diseases and pests based on image segmentation are proposed. Summary of the Invention

[0005] The purpose of the present invention is to provide a method and system for detecting agricultural crop diseases and pests based on image segmentation. Vertical and oblique view images of the tea garden are collected by an unmanned aerial vehicle, and image registration is performed using SIFT feature extraction, KNN matching, and RANSAC filtering, and aligned to the reference image coordinate system. The U-Net model is used to segment the foreground area and perform post-processing optimization. The disease area is located by a color detection method, disease features are extracted, and the disease stage is identified using a CNN; the YOLOv3 is used to identify the pest area, pest features are extracted, and the pest degree is evaluated. Finally, based on the disease stage and pest degree, the LSTM is used to predict the spread trend of diseases and pests, and a spatial heat map is generated according to the reference image coordinate system, improving the accuracy of disease and pest detection, thereby providing technical support for the prevention and control of smart agriculture.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A method for detecting agricultural crop diseases and pests based on image segmentation, comprising:

[0008] Set the flight path and shooting angle of the drone to acquire the vertical-view image and the oblique-view image of the tea garden;

[0009] Set a reference image coordinate system for the vertical-view image; use the SIFT algorithm to extract the image feature points of the reference image and the oblique-view image; use the KNN algorithm to match the image feature points, and use the RANSAC algorithm to filter out the wrong matches to obtain the feature point registration result; perform perspective transformation on the oblique-view image according to the feature point registration result to generate the aligned reference image coordinate system;

[0010] Preprocess the vertical-view image and the oblique-view image to generate the first dataset; use the U-Net algorithm to perform foreground segmentation on the first dataset to obtain the foreground region; post-process the foreground region to obtain the optimized foreground region;

[0011] Use the color detection method to extract the disease region from the foreground region, extract the disease characteristics of the disease region, and use the CNN algorithm to identify the disease stage of the disease characteristics; use the YOLOv3 algorithm to locate the insect body region in the foreground region, extract the pest characteristics of the foreground region and the insect body region, and evaluate the pest degree according to the pest characteristics;

[0012] According to the disease stage and the pest degree, use the LSTM algorithm to predict the spread direction and speed of the pests and diseases, and generate a spatial heat map according to the reference image coordinate system.

[0013] Furthermore, the acquisition process of the aligned reference image coordinate system includes:

[0014] Set an oblique-view coordinate system for the oblique-view image;

[0015] Calculate the homography matrix for the feature point registration result;

[0016] Use the homography matrix to perform perspective transformation on the oblique-view image to obtain the image alignment result, and align the oblique-view coordinate system to the reference image coordinate system;

[0017] Calculate the registration error between the image alignment result and the reference image. If the registration error is greater than the preset registration error, adjust the parameters of the RANSAC algorithm and perform registration again.

[0018] Furthermore, the implementation process of the post-processing includes:

[0019] Detect all connected regions in the foreground region using connected component analysis, and calculate the area of each connected region; if the area of the connected region is less than the set area threshold, remove the connected region and update the foreground region to obtain the first foreground region.

[0020] Use morphological skeleton extraction technology to obtain the skeleton extraction result of the first foreground region. If the number of connected regions in the skeleton extraction result is greater than 2, use morphological closing operation to repair the first foreground region and generate the optimized foreground region.

[0021] Further, the extraction process of the disease characteristics includes:

[0022] The disease characteristics include disease color characteristics, disease texture characteristics, and disease morphology characteristics;

[0023] Calculate the average hue, average saturation, and average brightness of the disease region, and analyze the color distribution of the disease region in combination with the color histogram to obtain the disease color characteristics;

[0024] Convert the disease region into a grayscale image and calculate the gray-level co-occurrence matrix to obtain the disease texture characteristics;

[0025] Use a binary mask to extract the disease spots in the disease region, and calculate the area, perimeter, and roundness of the disease spots to obtain the disease morphology characteristics.

[0026] Further, the calculation process of the pest degree includes:

[0027] The pest characteristics include pest body characteristics, wormhole characteristics, and bite mark characteristics;

[0028] Extract the pest body type, pest body size, and pest body quantity in the pest body region to obtain the pest body characteristics;

[0029] Use connected component analysis to detect the wormhole contour in the foreground region, count the number of pixels in the wormhole contour, and calculate the dispersion degree through the centroid position of the wormhole contour to obtain the wormhole characteristics;

[0030] Use an edge detection algorithm to extract the leaf edge contour of the foreground region, generate a smooth edge contour through polynomial fitting, compare the leaf edge contour with the smooth edge contour, and use the contour difference algorithm to determine the area of the concave region to obtain the bite mark characteristics;

[0031] Weighted sum the pest body characteristics, the wormhole characteristics, and the bite mark characteristics to obtain the pest degree.

[0032] An agricultural crop pest and disease detection system based on image segmentation, comprising:

[0033] An image acquisition module for acquiring vertical-view images and tilted-view images of a tea garden;

[0034] A coordinate system alignment module for extracting image feature points of the vertical-view image and the tilted-view image using the SIFT algorithm; matching the image feature points using the KNN algorithm, and filtering out incorrect matches using the RANSAC algorithm to obtain a feature point registration result; performing a perspective transformation on the tilted-view image according to the feature point registration result to generate an aligned reference image coordinate system;

[0035] A foreground segmentation module for preprocessing the vertical-view image and the tilted-view image to generate a first data set; performing foreground segmentation on the first data set using the U-Net algorithm to obtain a foreground region; performing post-processing on the foreground region to obtain an optimized foreground region;

[0036] A pest and disease assessment module for extracting disease regions from the foreground region using a color detection method, extracting disease features from the disease regions, and identifying disease stages using the CNN algorithm for the disease features; using the YOLOv3 algorithm to locate pest body regions in the foreground region, extracting pest and disease features of the foreground region and the pest body regions, and evaluating the degree of pest and disease damage according to the pest and disease features;

[0037] A pest and disease prediction module for predicting the spread direction and spread speed of pest and disease using the LSTM algorithm according to the disease stage and the degree of pest and disease damage, and generating a spatial heat map according to the reference image coordinate system.

[0038] Furthermore, the process of obtaining the aligned reference image coordinate system includes:

[0039] Setting a tilted-view coordinate system for the tilted-view image;

[0040] Calculating a homography matrix for the feature point registration result;

[0041] Performing a perspective transformation on the tilted-view image using the homography matrix to obtain an image alignment result, and aligning the tilted-view coordinate system to the reference image coordinate system;

[0042] Calculating the registration error between the image alignment result and the reference image. If the registration error is greater than a preset registration error, adjust the parameters of the RANSAC algorithm and re-perform the registration.

[0043] Furthermore, the implementation process of the post-processing includes:

[0044] Detect all connected regions in the foreground region using connected component analysis, and calculate the area of each connected region; if the area of the connected region is less than the set area threshold, then eliminate the connected region and update the foreground region to obtain the first foreground region.

[0045] Use morphological skeleton extraction technology to obtain the skeleton extraction result of the first foreground region. If the number of connected regions in the skeleton extraction result is greater than 2, then use morphological closing operation to repair the first foreground region and generate the optimized foreground region.

[0046] Further, the extraction process of the disease characteristics includes:

[0047] The disease characteristics include disease color characteristics, disease texture characteristics, and disease morphology characteristics;

[0048] Calculate the average hue, average saturation, and average brightness of the disease region, and analyze the color distribution of the disease region in combination with the color histogram to obtain the disease color characteristics;

[0049] Convert the disease region into a grayscale image and calculate the gray-level co-occurrence matrix to obtain the disease texture characteristics;

[0050] Use a binary mask to extract the disease spots in the disease region, and calculate the area, perimeter, and roundness of the disease spots to obtain the disease morphology characteristics.

[0051] Further, the calculation process of the pest degree includes:

[0052] The pest characteristics include pest body characteristics, pest hole characteristics, and bite mark characteristics;

[0053] Extract the pest body type, pest body size, and pest body quantity in the pest body region to obtain the pest body characteristics;

[0054] Use connected component analysis to detect the pest hole contour in the foreground region, count the number of pixels in the pest hole contour, and calculate the degree of dispersion through the centroid position of the pest hole contour to obtain the pest hole characteristics;

[0055] Use an edge detection algorithm to extract the leaf edge contour of the foreground region, generate a smooth edge contour through polynomial fitting, compare the leaf edge contour with the smooth edge contour, and use the contour difference algorithm to determine the area of the concave region to obtain the bite mark characteristics;

[0056] Sum the pest body characteristics, the pest hole characteristics, and the bite mark characteristics with weights to obtain the pest degree.

[0057] Compared with the prior art, the beneficial effects of the present invention are:

[0058] 1. Through SIFT feature extraction, KNN matching, and RANSAC filtering methods, the present invention performs feature point matching on the vertical-view images and oblique-view images obtained by the drone, calculates the homography matrix, and then performs perspective transformation to align the oblique-view images to the reference image coordinate system. In addition, based on the registration error feedback, the RANSAC parameters are adjusted, effectively reducing the image distortion caused by the perspective change, thereby improving the usability of the multi-view data, providing a unified reference coordinate system for subsequent pest and disease detection, and improving the consistency and accuracy of the detection results.

[0059] 2. The present invention uses U-Net for foreground segmentation and combines connected component analysis to eliminate small noise regions, avoiding interference from background information such as soil and sky, and improving the accuracy of tea tree region extraction. Combining with the skeleton extraction technology to analyze the connectivity of the tea trees, if the skeleton structure is broken, the morphological closing operation is automatically performed for repair, avoiding mis-segmenting the tea trees into multiple independent regions, thereby improving the integrity of the leaf region and ensuring the integrity of the tea tree structure, providing high-quality input data for subsequent disease detection and pest detection.

[0060] 3. The present invention comprehensively extracts the features of insect bodies, insect holes, and bite marks, and calculates the pest degree based on the weighted summation method. At the same time, combining the color, texture, and morphological features of diseases, the disease stage is identified through the CNN algorithm. Further, using the LSTM algorithm to combine the disease stage and the pest degree, the diffusion trend of pests and diseases is predicted, and a spatial heat map is generated to visualize the future development trend of pests and diseases. The present invention can provide early warnings, accurately evaluate the risks of pests and diseases, reduce the economic losses caused by pests and diseases, and improve the accuracy of tea tree pest and disease detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 It is a schematic flow chart of a method for detecting agricultural crop pests and diseases based on image segmentation proposed by the present invention;

[0062] Figure 2 It is a schematic flow chart of aligning the reference image coordinate system proposed by the present invention;

[0063] Figure 3 It is a schematic structural diagram of a system for detecting agricultural crop pests and diseases based on image segmentation proposed by the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0064] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0065] Please refer to Figures 1 to 3 , the present invention provides a method and system for detecting agricultural crop pests and diseases based on image segmentation, and the technical solution is as follows:

[0066] The first embodiment is as follows:

[0067] The terrain of a certain tea garden is complex and the planting density is high. Traditional pest and disease monitoring methods are difficult to detect in the initial stage, which is likely to lead to large-scale outbreaks. The manifestation forms of different diseases (such as tea blister blight and tea mold) and pests (such as tea geometrid and tea leaf weevil) are complex and the accuracy is insufficient. In addition, the spread of pests and diseases has dynamic changes in time and space, and traditional methods cannot effectively predict their spread trends, and control measures may be lagged, thus affecting the yield and quality of tea.

[0068] In view of the above problems, Figure 1 is a schematic flow chart of a method for detecting agricultural crop pests and diseases based on image segmentation proposed by the present invention. As Figure 1 shown, a method for detecting agricultural crop pests and diseases based on image segmentation is used to improve the accuracy of pest and disease monitoring in the tea garden, including:

[0069] Step 1: Set the flight path and shooting angle of the unmanned aerial vehicle (UAV) to obtain the vertical view image and the oblique view image of the tea garden.

[0070] Specifically, in this embodiment, regular grids are divided according to the terrain of a certain tea garden to ensure full coverage, and the horizontal interval is calculated according to the field of view angle of the UAV camera. For the vertical view image, the UAV camera is kept perpendicular to the ground, and the top view of the tea garden is taken from directly above, flying parallel to the main axis of the tea garden, with a forward overlap rate ≥ 80% and a side overlap rate ≥ 70% for subsequent stitching and registration. For the oblique view image, the angle of the UAV gimbal is adjusted, such as 45° or 60°, to take an oblique shot of the side or back of the tea plant, balancing the coverage range and detail clarity, thus meeting the accuracy requirements for subsequent image recognition.

[0071] Step 2: Set the vertical view image as the reference image and set a reference image coordinate system for the reference image; use the Scale-Invariant Feature Transform (SIFT) algorithm to extract the image feature points of the reference image and the oblique view image; use the K-Nearest Neighbor (KNN) algorithm to match the image feature points, and use the Random Sample Consensus (RANSAC) algorithm to filter out the wrong matches to obtain the feature point registration result; perform perspective transformation on the oblique view image according to the feature point registration result to generate the aligned reference image coordinate system.

[0072] Specifically, the vertical perspective image of the tea garden is parallel to the ground and is not affected by perspective distortion, which can be used as a reference image. Taking the upper left corner of the reference image as the coordinate origin, a two-dimensional Cartesian coordinate system is established. The x-axis represents the horizontal direction of the image, with a range from 0 to the width w of the reference image, and the y-axis represents the vertical direction of the image, with a range from 0 to the height h of the reference image. The unit length corresponds to the actual ground size, making the coordinate system of the reference image consistent with the actual geographical coordinates, which is convenient for subsequent calculations and spatial mapping.

[0073] Next, the SIFT (Scale-Invariant Feature Transform) algorithm is used to extract image feature points, and the KNN algorithm is adopted for feature point matching. For each feature point, two nearest matching points are selected, and the RANSAC (Random Sample Consensus) algorithm is used to eliminate incorrect matches, obtaining the feature point registration result and improving the accuracy of the matching.

[0074] Furthermore, the process of obtaining the coordinate system of the aligned reference image includes:

[0075] Setting an inclined perspective coordinate system for the inclined perspective image;

[0076] Calculating the homography matrix for the feature point registration result;

[0077] Using the homography matrix to perform perspective transformation on the inclined perspective image to obtain the image alignment result, and aligning the inclined perspective coordinate system to the reference image coordinate system;

[0078] Calculating the registration error between the image alignment result and the reference image. If the registration error is greater than the preset registration error, adjust the parameters of the RANSAC algorithm and re-perform the registration.

[0079] Specifically, in this embodiment, Figure 2 This is a schematic flow chart of the process for aligning the reference image coordinate system proposed by the present invention. As Figure 2 shown, after completing the feature point matching, calculate the homography matrix, that is, the transformation matrix from the feature point registration result of the inclined perspective to the feature point registration result of the reference perspective. Then, select a set of feature points in the inclined perspective image, and use the homography matrix to calculate their mapped coordinates P in the reference perspective image and compare them with the actual feature points of the reference perspective. Next, calculate the registration error, that is, measure the Euclidean distance between the mapped point P and the actual feature points of the reference image. If the registration error is greater than the preset registration error (for example, 0.8 pixels), then dynamically adjust the error threshold of the RANSAC algorithm, increase the number of feature points, and re-perform the registration until the registration error converges within the preset registration error range. When the registration error meets the accuracy requirements, output the final image alignment result to complete the establishment of the reference coordinate system.

[0080] By applying the geographic coordinate transformation technology to process the images taken from different angles, the detection results of pests and diseases can be accurately mapped to the actual positions of the tea gardens. On this basis, the SIFT, KNN, and RANSAC algorithms are adopted, and combined with the homography matrix for image registration, so that the images from different perspectives are accurately aligned, reducing the later-stage positioning error of pests and diseases, making the detection of the health status of tea trees more comprehensive, and improving the accuracy of the detection of tea tree pests and diseases.

[0081] Step 3: Preprocess the vertical-view image and the oblique-view image to generate a first data set; use the U-Net algorithm to perform foreground segmentation on the first data set to obtain the foreground area; post-process the foreground area to obtain the optimized foreground area to improve the integrity and accuracy of the foreground area.

[0082] Further, the implementation process of the post-processing includes:

[0083] Use connected component analysis to detect all connected components in the foreground area, and calculate the area of each connected component; if the area of the connected component is less than the set area threshold, then remove the connected component and update the foreground area to obtain the first foreground area;

[0084] Use morphological skeleton extraction technology to obtain the skeleton extraction result of the first foreground area. If the number of connected components of the skeleton extraction result is greater than 2, then use morphological closing operation to repair the first foreground area and generate the optimized foreground area.

[0085] Specifically, in this embodiment, the preprocessing includes unifying the image size and Gaussian filtering for denoising, then loading the trained U-Net model, performing foreground segmentation on the processed first data set, extracting the binary mask of the tea tree, converting the output of the U-Net into a binary image to obtain the preliminary foreground area. Then, in order to further optimize the foreground area, all connected components in the foreground area are identified and the area of each connected component is calculated. If the area of a certain connected component is less than the set area threshold (such as 100 pixels), it is considered as a noise area and removed to obtain the optimized first foreground area. Apply morphological skeleton extraction to the first foreground area to obtain the skeleton extraction result, that is, the skeleton structure of the tea tree branches and leaves. Perform connected component analysis on the skeleton result to determine whether there is a break in the tea tree. If the number of connected components of the skeleton structure is greater than 2, it indicates that the branches or leaves are mis-segmented, then use the closing operation to fill the mis-segmented broken area to generate the final optimized foreground area. The IoU of the optimized foreground area can reach 85%-90%, which is 5%-10% higher than the original U-Net output, improving the continuity of the segmentation result and providing a more accurate foreground input for subsequent pest and disease detection.

[0086] Step 4: Use the color detection method to locate the disease area in the foreground area, extract the disease characteristics of the disease area, and use the CNN algorithm to identify the disease stage based on the disease characteristics; use the YOLOv3 algorithm to locate the insect body area in the foreground area, extract the pest characteristics of the foreground area and the insect body area, and evaluate the pest degree according to the pest characteristics.

[0087] Specifically, tea tree diseases usually cause changes in leaf color (such as yellowing, browning, and black spots, etc.). Therefore, after extracting the foreground area from the original RGB image, the extracted foreground area is converted to the HSV color space. Set the HSV color range according to the disease color characteristics to extract the disease area. For example, the HSV color range of browning is from [10, 50, 50] to [30, 255, 255], and a disease mask is generated to locate the disease area.

[0088] Furthermore, the extraction process of the disease characteristics includes:

[0089] The disease characteristics include disease color characteristics, disease texture characteristics, and disease morphology characteristics;

[0090] Calculate the average hue, average saturation, and average brightness of the disease area, and analyze the color distribution of the disease area in combination with the color histogram to obtain the disease color characteristics;

[0091] Convert the disease area to a grayscale image and calculate the gray-level co-occurrence matrix to obtain the disease texture characteristics;

[0092] Use a binary mask to extract the disease spots in the disease area, and calculate the area, perimeter, and roundness of the disease spots to obtain the disease morphology characteristics.

[0093] Specifically, calculate the average hue, average saturation, and average brightness of the disease area, and analyze the color distribution of the disease area in combination with the color histogram, so as to extract the color characteristics of the disease area. In addition, different diseases will cause texture changes such as rough, smooth, and spotted on the leaf surface. Therefore, the gray-level co-occurrence matrix can be used to extract disease texture characteristics such as contrast, entropy, uniformity, and correlation. In addition, diseases will also cause irregular disease spots, insect holes, or leaf distortion on the leaves. Therefore, a binary mask can be used to extract the contour of the disease area and extract disease morphology characteristics such as area, perimeter, and roundness. For example, for Exobasidium vexans, the color characteristics are generally an average hue of 30, a saturation of 120, and a brightness of 180; the texture characteristics are generally a contrast of 1.2, an entropy of 0.8, a uniformity of 0.9, and a correlation of 0.85; the morphology characteristics are generally a disease area of 400px 2, with a perimeter of 80 px and a roundness of 0.6. By combining the three major characteristics of color, texture, and morphology, a dataset of disease characteristics is constructed, providing a scientific basis for disease classification, thereby effectively distinguishing different types of diseases and improving the accuracy of tea tree disease detection.

[0094] Table 1 Examples of data annotation for different disease stages

[0095]

[0096] Table 2 Comparison of the performance of different algorithms

[0097] Algorithm Disease category accuracy Disease stage accuracy SVM 88.2％ 86.5％ Random forest 80.4％ 79.3％ CNN 91.5％ 89.8％

[0098] The disease characteristics can be directly used in the CNN algorithm for multi-task learning, simultaneously outputting the disease category and disease stage, improving the accuracy of automatic recognition. The tea tree disease images at different stages and with different symptoms collected are annotated and the CNN algorithm is trained, as shown in Table 1. To evaluate the effect of CNN in recognizing the disease stage, a comparison is made with other machine learning algorithms (SVM and random forest), as shown in Table 2. The disease stage is divided into three categories: mild, moderate, and severe. It can be seen that CNN performs better than the SVM and random forest algorithms. Without the need to train the detection task separately, it can automatically learn the color, texture, and morphological characteristics of the disease and accurately monitor the situation of tea tree diseases.

[0099] Furthermore, the calculation process of the pest degree includes:

[0100] The pest characteristics include pest body characteristics, pest hole characteristics, and bite mark characteristics;

[0101] Extract the pest type, pest size, and pest quantity in the pest body area to obtain the pest body characteristics;

[0102] Use connected component analysis to detect the pest hole contours in the foreground area, count the number of pixels of the pest hole contours, and calculate the degree of dispersion through the centroid position of the pest hole contours to obtain the pest hole characteristics;

[0103] Use an edge detection algorithm to extract the leaf edge contours of the foreground area, generate smooth edge contours through polynomial fitting, compare the leaf edge contours with the smooth edge contours, and use the contour difference algorithm to determine the area of the concave area to obtain the bite mark characteristics;

[0104] Weighted sum the pest body characteristics, the pest hole characteristics, and the bite mark characteristics to obtain the pest degree.

[0105] Specifically, first, the YOLOv3 algorithm is used to locate the worm body area, obtain the worm body types corresponding to the worm body area (such as aphids, caterpillars, and leaf beetles), and obtain the worm body size and the number of worm bodies based on the pixel area and quantity of the target detection box respectively. By synthesizing the worm body type, worm body size, and the number of worm bodies, the worm body characteristics are sorted out and the worm body index is calculated, which is expressed as:

[0106]

[0107] Among them, I insect is the worm body index, S insect is the worm body size, N insect is the number of worm bodies, I type is the pest index corresponding to the worm body type. For example, aphids are 0.5, caterpillars are 0.8, leaf beetles are 1.0, and max() is the maximization function.

[0108] Then, the foreground area is read for connected component analysis, the pixel area of the wormholes is counted, and the centroid position of the wormholes is calculated. Then the standard deviation of the centroid position is calculated to obtain the degree of dispersion. If the wormholes are concentrated, the impact is small. If they are dispersed, it indicates that the pest damage is more serious. The wormhole index is calculated based on the pixel area and degree of dispersion of the wormholes, which is expressed as:

[0109]

[0110] Among them, I hole is the wormhole index, A hole is the wormhole area, D hole is the degree of dispersion, and max() is the maximization function.

[0111] Then, the Canny edge detection algorithm is used to obtain the leaf contour, the polynomial fitting is used to smooth the edge contour, and the contour difference algorithm is used to calculate the area of the eaten part of the leaf to obtain the bite mark feature. The area is normalized and then the bite mark index is calculated. Finally, by setting the weighting coefficients of different features, the weighting coefficients of the worm body index, the wormhole index, and the bite mark index can be set to 0.4, 0.3, and 0.3 respectively. After weighted summation, the pest damage degree is obtained.

[0112] Table 3 Examples of pest damage degree

[0113] Coordinates Insect body index Insect hole index Bite mark index Pest infestation level (105，210) 0.65 0.75 0.72 0.72 (220，315) 0.80 0.88 0.85 0.83 (330，420) 0.55 0.62 0.60 0.60

[0114] Meanwhile, obtain the initial coordinates of various pest characteristics, including the center coordinates of the pest detection box, the center coordinates of the insect holes, and the center coordinates of the bitten area. Use DBSCAN clustering to merge these center coordinates. If the distance between two center coordinates is less than 50 pixels, they are merged into one coordinate. As shown in Table 3, calculate the pest severity for each coordinate according to three indices respectively. By comprehensively considering multiple pest characteristics, including pests, insect holes, and leaf damage, an accurate pest assessment is obtained. In addition, through the analysis of the dispersion degree of insect holes, it is also helpful to judge the spread of pests and improve the detection accuracy of tea tree pests.

[0115] Step Five: According to the disease stage and the pest severity, use the LSTM algorithm to predict the spread direction and speed of pests and diseases, and generate a spatial heat map based on the reference image coordinate system.

[0116] Specifically, tea tree pests and diseases have the characteristics of spatio-temporal dynamic changes. Predicting their spread trend in the early stage can improve the prevention and control efficiency and reduce losses. Organize data such as pest characteristics, disease characteristics, disease stage, pest severity, and geographical coordinates into a time series for LSTM algorithm training. The trained LSTM algorithm predicts the coordinates of the next moment. According to the coordinates of the current moment and the coordinates of the next moment, obtain the spread direction and speed. Among them, the spread direction refers to the spatial direction of the spread of pests and diseases, and the spread speed refers to the spread distance of pests and diseases per unit time. Assume that the coordinates of the current moment are (x0, y0), and the predicted coordinates of the next moment are (x1, y1). Then the spread direction can be obtained by calculating the coordinate difference, expressed as:

[0117] dir = arctan2(y1 - y0, x1 - x0);

[0118] where dir is the spread direction, and arctan2() is the two-dimensional arctangent function, which takes into account the positive and negative situations of the coordinates.

[0119] In addition, the spread speed is obtained by calculating the Euclidean distance between the current coordinates and the predicted coordinates, expressed as:

[0120]

[0121] where speed is the spread speed.

[0122] Finally, using the spread direction and speed, and the reference image coordinate system, mark the spread situation of pests and diseases at each moment in the spatial heat map, display the spread of pests and diseases in different regions, and help agricultural managers make reasonable prevention and control decisions.

[0123] The present invention collects vertical-view and oblique-view images through an unmanned aerial vehicle, and uses SIFT, KNN, and RANSAC algorithms for feature point matching and perspective transformation to align images from different perspectives to a unified coordinate system, solving the problem of pest and disease area offset caused by perspective differences and ensuring the consistency of pest and disease detection under different perspectives. Then, the U-Net algorithm is used for foreground segmentation to automatically extract the tea tree area, and morphological skeleton extraction and closing operations are used to repair connectivity to optimize the foreground area, ensuring the accuracy of subsequent pest and disease detection. Next, features such as the color, texture, and morphology of diseases are extracted to identify different disease types and disease stages, and information such as the size, type, and quantity of insect bodies are extracted to achieve a quantitative assessment of the degree of pest damage. Finally, a time series dataset is constructed, and the LSTM algorithm is used to predict the future spread position and speed of pests and diseases, quantifying the dynamic change trend of pests and diseases, improving the accuracy of pest and disease detection, and enabling agricultural managers to make control decisions more quickly.

[0124] Example 2 is as follows:

[0125] For further explanation of Example 1, Figure 3 is a schematic structural diagram of an agricultural crop pest and disease detection system based on image segmentation proposed by the present invention, as Figure 3 shown, an agricultural crop pest and disease detection system based on image segmentation is proposed, including:

[0126] Refer to Figure 3 in the image acquisition module for acquiring vertical-view and oblique-view images of the tea garden;

[0127] Refer to Figure 3 in the coordinate system alignment module for using the SIFT algorithm to extract image feature points of the vertical-view image and the oblique-view image; using the KNN algorithm to match the image feature points, and using the RANSAC algorithm to filter out incorrect matches to obtain the feature point registration result; performing perspective transformation on the oblique-view image according to the feature point registration result to generate an aligned reference image coordinate system;

[0128] Refer to Figure 3 in the foreground segmentation module for preprocessing the vertical-view image and the oblique-view image to generate a first dataset; using the U-Net algorithm to perform foreground segmentation on the first dataset to obtain a foreground area; performing post-processing on the foreground area to obtain the optimized foreground area;

[0129] Refer to Figure 3A pest and disease assessment module is used to locate disease areas in the foreground area using a color detection method, extract disease features from the disease areas, and identify the disease stage of the disease features using a CNN algorithm; use the YOLOv3 algorithm to locate insect body areas in the foreground area, extract pest features of the foreground area and the insect body areas, and evaluate the pest degree according to the pest features;

[0130] Reference Figure 3 A pest and disease prediction module is used to predict the spread direction and speed of pests and diseases using an LSTM algorithm according to the disease stage and the pest degree, and generate a spatial heat map according to the reference image coordinate system.

[0131] Furthermore, the process of obtaining the aligned reference image coordinate system includes:

[0132] Set an inclined view coordinate system for the inclined view image;

[0133] Calculate the homography matrix for the feature point registration result;

[0134] Use the homography matrix to perform perspective transformation on the inclined view image to obtain an image alignment result, and align the inclined view coordinate system to the reference image coordinate system;

[0135] Calculate the registration error between the image alignment result and the reference image. If the registration error is greater than the preset registration error, adjust the parameters of the RANSAC algorithm and perform registration again.

[0136] Furthermore, the implementation process of the post-processing includes:

[0137] Use connected component analysis to detect all connected regions in the foreground area, and calculate the area of each connected region; if the area of the region is less than the set area threshold, remove the connected region and update the foreground area to obtain a first foreground area;

[0138] Use morphological skeleton extraction technology to obtain the skeleton extraction result of the first foreground area. If the number of connected regions in the skeleton extraction result is greater than 2, use morphological closing operation to repair the first foreground area and generate an optimized foreground area.

[0139] Furthermore, the process of extracting the disease features includes:

[0140] The disease features include disease color features, disease texture features, and disease morphology features;

[0141] Calculate the average hue, average saturation, and average brightness of the disease area, and analyze the color distribution of the disease area in combination with the color histogram to obtain the disease color characteristics;

[0142] Convert the disease area into a grayscale image, calculate the gray-level co-occurrence matrix, and obtain the disease texture characteristics;

[0143] Use a binary mask to extract the disease spots in the disease area, calculate the area, perimeter, and roundness of the disease spots, and obtain the disease morphological characteristics.

[0144] Further, the calculation process of the pest degree includes:

[0145] The pest characteristics include insect body characteristics, insect hole characteristics, and bite mark characteristics;

[0146] Extract the insect type, insect size, and insect quantity of the insect body area to obtain the insect body characteristics;

[0147] Use connected component analysis to detect the insect hole contour in the foreground area, count the number of pixels of the insect hole contour, and calculate the degree of dispersion through the centroid position of the insect hole contour to obtain the insect hole characteristics;

[0148] Use an edge detection algorithm to extract the leaf edge contour of the foreground area, generate a smooth edge contour through polynomial fitting, compare the leaf edge contour with the smooth edge contour, and use the contour difference algorithm to determine the area of the concave area to obtain the bite mark characteristics;

[0149] Sum the insect body characteristics, the insect hole characteristics, and the bite mark characteristics with weights to obtain the pest degree.

[0150] An original system in a certain tea garden uses a handheld camera to take pictures of tea tree leaves, uses the SVM algorithm to classify disease types, and uses the HOG and SVM algorithms to classify insect bodies. Compare the original system with the results of the present invention. As shown in Table 4, compared with the original system, the present invention has obvious advantages in detection accuracy and diffusion prediction, etc., and can monitor tea garden pests and diseases more accurately.

[0151] Table 4 Comparison of results of different systems

[0152] Comparison item Original system This invention Disease detection accuracy 78.2％ 87.5％ Pest detection accuracy 80.1％ 86.7％ Diffusion prediction error No data 2.1m

[0153] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principle and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An agricultural crop pest and disease detection method based on image segmentation, characterized in that, Including: Set the flight path and shooting angle of the drone to obtain the vertical perspective image and the oblique perspective image of the tea garden; Set a reference image coordinate system for the vertical perspective image; use the SIFT algorithm to extract the image feature points of the reference image and the oblique perspective image; use the KNN algorithm to match the image feature points, and use the RANSAC algorithm to filter out the incorrect matches to obtain the feature point registration result; Perform perspective transformation on the oblique perspective image according to the feature point registration result to generate the aligned reference image coordinate system; Preprocess the vertical perspective image and the oblique perspective image to generate the first data set; Use the U-Net algorithm to perform foreground segmentation on the first data set to obtain the foreground area; Post-process the foreground area to obtain the optimized foreground area; Use a color detection method to extract the disease area from the foreground area, extract the disease characteristics of the disease area, use the CNN algorithm to identify the disease stage for the disease characteristics; use the YOLOv3 algorithm to locate the insect body area in the foreground area, extract the pest characteristics of the foreground area and the insect body area, and evaluate the pest degree according to the pest characteristics; According to the disease stage and the pest degree, use the LSTM algorithm to predict the spread direction and speed of the pests and diseases, and generate a spatial heat map according to the reference image coordinate system.

2. The agricultural crop pest and disease detection method based on image segmentation according to claim 1, wherein The process of obtaining the aligned reference image coordinate system includes: Set an oblique perspective coordinate system for the oblique perspective image; Calculate the homography matrix for the feature point registration result; Use the homography matrix to perform perspective transformation on the oblique perspective image to obtain the image alignment result, and align the oblique perspective coordinate system to the reference image coordinate system; Calculate the registration error between the image alignment result and the reference image. If the registration error is greater than the preset registration error, adjust the parameters of the RANSAC algorithm and perform registration again.

3. A method for detecting agricultural crop pests and diseases based on image segmentation according to claim 1, characterized in that, The implementation process of the post-processing includes: Use connected component analysis to detect all connected regions in the foreground area, and calculate the area of each connected region; if the area of the region is less than the set area threshold, remove the connected region and update the foreground area to obtain the first foreground area; Use morphological skeleton extraction technology to obtain the skeleton extraction result of the first foreground area. If the number of connected regions in the skeleton extraction result is greater than 2, use morphological closing operation to repair the first foreground area and generate the optimized foreground area.

4. A method for detecting agricultural crop pests and diseases based on image segmentation according to claim 1, characterized in that, The extraction process of the disease characteristics includes: The disease characteristics include disease color characteristics, disease texture characteristics, and disease morphology characteristics; Calculate the average hue, average saturation, and average brightness of the disease area, and analyze the color distribution of the disease area in combination with the color histogram to obtain the disease color characteristics; Convert the disease area to a grayscale image and calculate the gray-level co-occurrence matrix to obtain the disease texture characteristics; Use a binary mask to extract the disease spots in the disease area, and calculate the area, perimeter, and roundness of the disease spots to obtain the disease morphology characteristics.

5. A method for detecting agricultural crop pests and diseases based on image segmentation according to claim 1, characterized in that, The calculation process of the pest degree includes: The pest characteristics include insect body characteristics, wormhole characteristics, and bite mark characteristics; Extract the insect type, insect size, and insect quantity of the insect body area to obtain the insect body characteristics; Use connected component analysis to detect the wormhole contours in the foreground area, count the number of pixels of the wormhole contours, and calculate the degree of dispersion through the centroid position of the wormhole contours to obtain the wormhole characteristics; Use an edge detection algorithm to extract the leaf edge contours of the foreground area, generate smooth edge contours through polynomial fitting, compare the leaf edge contours with the smooth edge contours, and use a contour difference algorithm to determine the area of the concave area to obtain the bite mark characteristics; Sum the insect body characteristics, the wormhole characteristics, and the bite mark characteristics with weights to obtain the pest degree.

6. An agricultural crop pest and disease detection system based on image segmentation, characterized in that, It includes: An image acquisition module for acquiring a vertical view image and an inclined view image of a tea garden; A coordinate system alignment module for using the SIFT algorithm to extract image feature points of the vertical view image and the inclined view image; using the KNN algorithm to match the image feature points, and using the RANSAC algorithm to filter out incorrect matches to obtain a feature point registration result; Perform a perspective transformation on the inclined view image according to the feature point registration result to generate an aligned reference image coordinate system; A foreground segmentation module for preprocessing the vertical view image and the inclined view image to generate a first data set; Use the U-Net algorithm to perform foreground segmentation on the first data set to obtain a foreground area; perform post-processing on the foreground area to obtain the optimized foreground area; A pest and disease assessment module for using a color detection method to extract the disease area from the foreground area, extracting disease characteristics from the disease area, and using the CNN algorithm to identify the disease stage for the disease characteristics; using the YOLOv3 algorithm to locate the insect body area in the foreground area, extracting pest characteristics of the foreground area and the insect body area, and evaluating the pest degree according to the pest characteristics; A pest and disease prediction module for using the LSTM algorithm to predict the spread direction and spread speed of pests and diseases according to the disease stage and the pest degree, and generating a spatial heat map according to the reference image coordinate system.

7. An agricultural crop pest and disease detection system based on image segmentation according to claim 6, characterized in that, The acquisition process of the aligned reference image coordinate system includes: Set an inclined view coordinate system for the inclined view image; Calculate a homography matrix for the feature point registration result; Perform a perspective transformation on the inclined view image using the homography matrix to obtain an image alignment result, and align the inclined view coordinate system to the reference image coordinate system; Calculate the registration error between the image alignment result and the reference image. If the registration error is greater than a preset registration error, adjust the parameters of the RANSAC algorithm and perform registration again.

8. An agricultural crop pest and disease detection system based on image segmentation according to claim 6, characterized in that, The implementation process of the post-processing includes: Use connected component analysis to detect all connected regions in the foreground area, calculate the area of each connected region; if the area of the region is less than a set area threshold, remove the connected region and update the foreground area to obtain a first foreground area; The morphological skeleton extraction technique is used to obtain the skeleton extraction result of the first foreground region. If the number of connected regions of the skeleton extraction result is greater than 2, then morphological closing operation is used to repair the first foreground region, and the optimized foreground region is generated.

9. An agricultural crop pest and disease detection system based on image segmentation according to claim 6, characterized in that, The extraction process of the disease characteristics includes: The disease characteristics include disease color characteristics, disease texture characteristics, and disease morphological characteristics; Calculate the average hue, average saturation, and average brightness of the disease region, and analyze the color distribution of the disease region in combination with the color histogram to obtain the disease color characteristics; Convert the disease region into a grayscale image, and calculate the gray-level co-occurrence matrix to obtain the disease texture characteristics; Use a binary mask to extract the disease spots in the disease region, and calculate the area, perimeter, and roundness of the disease spots to obtain the disease morphological characteristics.

10. An agricultural crop pest and disease detection system based on image segmentation according to claim 6, characterized in that, The calculation process of the pest degree includes: The pest characteristics include pest body characteristics, pest hole characteristics, and bite mark characteristics; Extract the pest body type, pest body size, and pest body quantity in the pest body region to obtain the pest body characteristics; Use connected component analysis to detect the pest hole contour in the foreground region, count the number of pixels of the pest hole contour, and calculate the dispersion degree through the centroid position of the pest hole contour to obtain the pest hole characteristics; Use an edge detection algorithm to extract the leaf edge contour of the foreground region, generate a smooth edge contour through polynomial fitting, compare the leaf edge contour with the smooth edge contour, and use the contour difference algorithm to determine the area of the concave region to obtain the bite mark characteristics; Perform a weighted sum of the pest body characteristics, the pest hole characteristics, and the bite mark characteristics to obtain the pest degree.

Images