Method and system for detecting plant diseases and insect pests

Abstract

The invention discloses a plant disease and insect pest detection method and system, and belongs to the field of forestry production. In order to solve the problems of poor precision and time consumption of existing plant disease and insect pest detection, the invention provides a plant disease and insect pest detection method which comprises the following steps: acquiring a visible light image of a plant in a nursery garden and three-dimensional depth information of the plant to obtain an RGB-D image; dividing the RGB-D image into a plurality of equal-size grids, preprocessing the grids, inputting the grids into the deep learning model for detection, and finally obtaining a plant disease and insect pest detection result. According to the detection method, the three-dimensional depth information of the plant is introduced, so that the position and the range of the pest and disease damage can be more accurately judged; meanwhile, the robustness of the deep learning model is enhanced, and the model can still accurately detect diseases and insect pests under the condition that the illumination condition is poor; the whole detection method not only improves the detection accuracy, but also improves the detection efficiency, so that real-time and automatic disease and pest detection can be realized in a field environment, and the method is suitable for large-scale popularization and application. The system is simple in structural composition, real-time processing and analysis are carried out while data are collected, the complexity of installation and maintenance is reduced, and the real-time performance of the whole detection process is enhanced.

Description

Technical Field

[0001] The invention belongs to the technical field of forestry production, and more specifically, relates to a method and system for detecting plant diseases and insect pests. Background Art

[0002] In the field of forestry seedling cultivation, especially for fast-growing tree species such as eucalyptus seedlings, traditional pest and disease detection methods usually rely on manual sampling and laboratory testing, which are inefficient, inaccurate, and easily affected by human factors. These methods are not only time-consuming and labor-intensive, but also difficult to achieve large-scale, real-time monitoring. Most existing pest and disease detection systems are based on RGB images. Although they can provide visual information, in complex and changeable outdoor and nursery environments, it is difficult to accurately identify different types of pests and diseases by relying solely on RGB images, especially in the early stages of pests and diseases or when vegetation is dense. The detection effect is limited. In addition, RGB images cannot provide depth information of the scene, which limits the accurate judgment of the spatial distribution of pests and diseases.

[0003] Corresponding improvements have also been made to the above problems, such as China Patent Application No. CN202410792387.2, published on June 19, 2024. The patent discloses a method for detecting plant diseases and insect pests based on modal joint learning, which includes: obtaining a plant image to be detected; randomly cropping the plant image to be detected to obtain a number of cropped images, and determining the structural features of the plant image to be detected based on the cropped images; converting the plant image to be detected into a grayscale image, determining the texture value of each pixel based on the pixel value of each pixel in the grayscale image and each pixel in the neighborhood, and determining the texture features of the plant image to be detected based on the texture value of each pixel; determining the color features of the plant image to be detected based on the hue value, saturation value, brightness value and plant type of the plant image to be detected; determining the detection result of the plant image to be detected based on the structural features, texture features and color features of the plant image to be detected. The shortcomings of this patent are: although the accuracy of plant disease and insect pest detection is improved, the cost is high, and the source of the plant image to be detected is easily disturbed by the external environment.

[0004] Another example is Chinese patent application number CN202410362779.5, which was published on March 28, 2024. The patent discloses a method for rapid detection of pest and disease areas of garden plants, including: obtaining the target degree value of each suspected pest and disease pixel point and the target pixel point according to the number of suspected pest and disease pixels in the neighborhood range of each suspected pest and disease pixel point and the distance between the suspected pest and disease pixels; obtaining the decay probability of each target pixel point according to the grayscale distribution of the pixel points in the neighborhood range of each target pixel point and the texture value in the neighborhood range, the target degree value of each target pixel point, and the difference in the slope of adjacent edge pixels; obtaining the pest and disease area in the leaf image according to the decay probability of each target pixel point. The disadvantage of this patent is that although the influence of light on pest and disease detection is reduced, the calculation is complex and time-consuming, which increases the cost. Summary of the invention

[0005] 1. Problems to be solved

[0006] In view of the problem that existing plant pest and disease detection methods are of poor accuracy and time-consuming, the present invention provides a plant pest and disease detection method and system. The detection method of the present invention introduces the three-dimensional depth information of the plant, which helps to more accurately determine the location and range of pests and diseases; at the same time, it enhances the robustness of the deep learning model, which helps the model to accurately detect pests and diseases under poor lighting conditions; the entire detection method improves both the accuracy and efficiency of detection, so that real-time and automated pest and disease detection can be achieved in a field environment, which is suitable for large-scale promotion and use. The system structure of the present invention is simple, and real-time processing and analysis are performed while collecting data, which reduces the complexity of installation and maintenance and enhances the real-time nature of the entire detection process.

[0007] 2. Technical solution

[0008] To solve the above problems, the present invention adopts the following technical solutions.

[0009] A method for detecting plant diseases and insect pests comprises the following steps:

[0010] Image acquisition: Obtain visible light images of plants in the nursery and three-dimensional depth information of the plants to obtain RGB-D images;

[0011] Image gridding: Divide the RGB-D image into several grids of equal size and preprocess the grids;

[0012] Grid detection: Several grids are input into the deep learning model for detection to obtain the pest and disease detection results of several grids; the training process of the deep learning model includes an unsupervised learning stage and a supervised fine-tuning stage:

[0013] The SimCLR model is used in the unsupervised learning stage. The RGB-D image is input into the SimCLR model, and the SimCLR model extracts the image features and depth features in the RGB-D image.

[0014] The EfficientNet model is used in the supervised fine-tuning stage. The image features and deep features in the RGB-D images extracted by the SimCLR model are input into the EfficientNet model, and the EfficientNet model is used to identify the categories of pests and diseases. The EfficientNet model is adjusted using the RGB-D images with labeled pests and diseases.

[0015] Result summary output: The pest and disease detection results of several grids are summarized to obtain the plant pest and disease detection results.

[0016] By adopting the above technical solution, more details and features of the plants can be obtained by combining the three-dimensional depth information of the plants during image acquisition, and it is helpful to more accurately locate the position and range of pests and diseases, and to facilitate the identification of occluded diseased areas, thereby improving the detection accuracy; at the same time, the RGB-D image is gridded, and the gridding processing can improve the overall computing efficiency; secondly, it can further improve the detection accuracy, making the final plant pest and disease detection results more intuitive and vivid; finally, through the deep learning model combined with the unsupervised learning stage and the supervised fine-tuning stage, this model can learn rich features and classification boundaries, thereby realizing high-precision pest and disease detection; and it can adapt to changes in different lighting conditions, shooting angles and plant growth stages, improve the robustness of detection, reduce dependence on labeled data, and reduce labeling costs.

[0017] In general, the plant disease and pest detection method of the present application introduces three-dimensional depth information of plants. The three-dimensional depth information provides the spatial relationship between leaves, which helps to more accurately determine the location and range of pests and diseases. The three-dimensional depth information of plants also enhances the robustness of the deep learning model, because it is insensitive to changes in lighting conditions, which helps the model to accurately detect pests and diseases even in poor lighting conditions. The entire detection method improves both the accuracy and efficiency of detection, enabling real-time and automated pest and disease detection in field environments, which is suitable for large-scale promotion and use.

[0018] Furthermore, when performing image acquisition, the RGB-D camera is used to simultaneously obtain the visible light image of the plants in the nursery and the three-dimensional depth information of the plants; and the RGB-D camera is used to obtain several RGB-D images at different heights and / or different angles.

[0019] By adopting the above technical solution, the RGB-D camera can obtain color images and depth information at the same time, which is efficient and reduces the number of equipment, making installation more convenient; and RGB-D images at different heights and / or angles can obtain information on various parts and levels of the plant, and can provide plant information from different perspectives and levels, which helps to enhance the model's ability to identify pests and diseases; ultimately achieving the effect of improving detection accuracy.

[0020] Furthermore, when performing image acquisition, an RGB camera is used to collect visible light images of plants in the nursery; a depth camera is used to collect three-dimensional depth information of plants in the nursery; and both the RGB camera and the depth camera are placed at a distance of 1m to 1.4m from the plant's apical buds.

[0021] By adopting the above technical solution, different cameras are used to collect different data, so that the collection of two visible light images and three-dimensional depth information does not interfere with each other, which facilitates real-time adjustment of different camera positions; and the positions of the two cameras are limited so that the collection area maintains an appropriate number to avoid the collection area being too small affecting the overall detection efficiency, and the collection area being too large affecting the overall detection accuracy.

[0022] Furthermore, the specific training process of the unsupervised learning stage is as follows:

[0023] Data preparation: Collect several RGB-D images;

[0024] Data enhancement: Perform data enhancement operations on each RGB-D image;

[0025] SimCLR model training: The SimCLR framework is used for unsupervised learning to extract image features and depth features of RGB-D images from the trained SimCLR model, where the image features include the color and texture of the plants.

[0026] By adopting the above technical solution and using unsupervised learning for feature extraction, the SimCLR model can be trained without labeled data, which greatly reduces the cost and time of data labeling; and it can perform efficient feature extraction to more accurately identify pests and diseases.

[0027] Furthermore, the specific training process of the supervised fine-tuning stage is as follows:

[0028] Data preparation: Collect several RGB-D images with pests and diseases that are labeled; the images include pests and diseases of different types and severity, with corresponding labels.

[0029] Feature input: The image features and depth features of the RGB-D image extracted by the SimCLR model are input into the EfficientNet model, and the EfficientNet model identifies the pest category;

[0030] Model fine-tuning: Use several labeled RGB-D images with pests and diseases to adjust the EfficientNet model; specifically, the back propagation algorithm is used to optimize the model during the adjustment process, and some or all of the network layers of the EfficientNet model can be frozen for adjustment.

[0031] The above technical solution utilizes the parameters and structure of the pre-trained EfficientNet model to avoid training the model from scratch, thereby accelerating the model training process; and the EfficientNet model is an efficient convolutional neural network structure with fewer parameters and higher performance, so that the entire model can take into account both efficiency and accuracy.

[0032] Furthermore, the pest detection results of the plurality of grids include health status and disease severity, and the classification criteria are as follows:

[0033] Healthy: No signs of disease, leaves are intact, no diseases detected;

[0034] Mild disease: In a single grid, the infected area is between 0-20%, and the number of infected leaves is between 0-20%, which does not affect the overall health of the plant;

[0035] Moderate disease: In a single grid, the infected area is between 20-50%, or the number of infected leaves is between 20-50%, and the lesions are obvious;

[0036] Severe disease: In a single grid, the infected area exceeds 50%, and the number of infected leaves exceeds 50%, with large areas of leaves damaged.

[0037] By adopting the above technical solution, by determining the health status and disease degree of each grid, it is possible to achieve fine detection and classification of each plant, which is convenient for humans to quickly find the diseased area and intervene in time; and a comprehensive health report of the entire plant can be generated according to the status of each grid, which is convenient for a more intuitive understanding of the growth status of the plant and improves the timeliness and accuracy of the entire plant growth monitoring.

[0038] Furthermore, when summarizing the pest and disease detection results of several grids, it also includes counting the distribution of healthy grids, lightly diseased grids, moderately diseased grids and severely diseased grids to obtain healthy areas, lightly diseased areas, moderately diseased areas and severely diseased areas, and generating prevention and control recommendations based on the corresponding areas.

[0039] By adopting the above technical solution, the locations of disease-concentrated areas and healthy areas can be intuitively displayed, which is convenient for people to understand the spread trend of the disease and generate different prevention and control suggestions according to different areas. Human intervention can be implemented quickly and accurately without consuming a lot of time and intervention costs, thereby ensuring the stability of plant growth.

[0040] A system using any of the above plant disease and insect pest detection methods, comprising:

[0041] Perception module: used for data collection, obtaining visible light images of plants in the nursery and three-dimensional depth information of plants to obtain RGB-D images;

[0042] Computing module: used to grid and preprocess the RGB-D image, then enter the deep learning model for detection, and finally output the plant disease and pest detection results.

[0043] By adopting the above technical solution, the system introduces the three-dimensional depth information of plants. Compared with the traditional single spectral image detection method, it can capture the surface characteristics and structural information of plants more comprehensively, greatly reducing the impact of environmental factors on the detection results, thereby improving the accuracy of pest and disease detection and the robustness of the detection process; and the system has a simple structure and performs real-time processing and analysis while collecting data, which not only reduces the complexity of installation and maintenance, but also enhances the real-time nature of the entire detection process.

[0044] Furthermore, the perception module includes:

[0045] Camera unit: used to collect visible light images of plants in the nursery and three-dimensional depth information of the plants;

[0046] Heat dissipation unit: used to dissipate heat for the camera unit;

[0047] Power supply unit: used to supply power to the camera unit and the cooling unit.

[0048] By adopting the above technical solution, the setting of the heat dissipation unit enables the camera unit in the perception module to maintain stable performance under long-term working conditions, which will not affect the shooting accuracy, and can ensure that the entire perception module is in an environment with suitable temperature, thereby ensuring the normal operation of the unit components in the perception module.

[0049] Furthermore, the calculation module includes:

[0050] Jetson Xavier NX platform: used for gridding and preprocessing of RGB-D images, as well as plant pest and disease detection and analysis;

[0051] Display unit: used to output and display the plant pest and disease detection and analysis results in the Jetson Xavier NX platform;

[0052] Cooling unit: used to dissipate heat for the Jetson Xavier NX platform.

[0053] The above technical solution, using the powerful AI computing power and low power consumption characteristics of the Jetson Xavier NX platform, can process a large amount of data from the perception module in real time and accurately identify the types of plant diseases and pests, making the entire system more energy-efficient and environmentally friendly. The setting of the heat dissipation unit further ensures that the Jetson Xavier NX platform can still work stably under high temperature conditions, avoiding performance degradation or device failure due to overheating, and improving the safety of the computing module. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a schematic diagram of the overall process of the present invention;

[0055] Figure 2 Schematic diagram of the training process of the deep learning model in the present invention. DETAILED DESCRIPTION

[0056] The present invention is further described below in conjunction with specific embodiments and drawings.

[0057] like Figure 1 and Figure 2 As shown,

[0058] A method for detecting plant diseases and insect pests comprises the following steps:

[0059] Step 1: Image acquisition: Obtain visible light images of plants in the nursery and three-dimensional depth information of the plants to obtain RGB-D images;

[0060] In the traditional image acquisition process, the category of plant pests and diseases is determined only by acquiring visible light images, but visible light images are easily affected by lighting changes and environmental factors, resulting in low detection accuracy. In step 1, the present application introduces the three-dimensional depth information of the plant during image acquisition. The three-dimensional depth information of the plant can provide richer spatial structure and detail information, especially when the leaves of the plant are blocked or overlapped, the three-dimensional depth information can distinguish the foreground and background. More importantly, since the source of the image has more dimensional features, the generalization ability of the subsequent deep learning model is naturally improved, and it helps the deep learning model to learn more robust feature representations under limited data, thereby reducing the impact of data imbalance on model performance, thereby improving the detection accuracy of the deep learning model.

[0061] Therefore, in step 1, by combining the three-dimensional depth information of the plant with the visible light image of the plant, it is possible to better identify and locate field pests and diseases in the future, so as to achieve more efficient and accurate detection;

[0062] Specifically, in step 1, the collection of RGB-D images can be achieved by using an RGB-D camera to simultaneously obtain visible light images of plants in the nursery and three-dimensional depth information of the plants; and the RGB-D camera is used to obtain several RGB-D images at different heights and / or angles. The collection of two types of data can be achieved with only one RGB-D camera, which has high integration and high efficiency, reduces the number of devices used, and reduces maintenance costs and usage costs; and by collecting RGB-D images at different heights and / or angles, it is possible to provide plant information at different perspectives and spatial levels, thereby improving the comprehensive coverage of the data, providing more comprehensive data for subsequent detection, and improving detection accuracy.

[0063] Of course, the acquisition of RGB-D images can also be done by using an RGB camera to collect visible light images of plants in the nursery; using a depth camera to collect the three-dimensional depth information of plants in the nursery, and then fusing the visible light image with the three-dimensional depth information to obtain an RGB-D image; and both the RGB camera and the depth camera are placed 1m to 1.4m away from the top buds of the plants. Using different devices to collect different data makes the visible light image and the three-dimensional depth information of the plant not interfere with each other, and different cameras can be adjusted in real time to obtain the required data, which increases the operability and flexibility of the entire acquisition process. In addition, the positions of the two cameras are limited so that the two cameras can capture the appropriate area during a single acquisition, avoiding the problem of too small an area taking a long time to detect and too large an area with poor detection accuracy. Therefore, the acquisition of RGB-D images can choose a suitable method according to the actual situation.

[0064] Step 2: Image gridding: Divide the RGB-D image into several grids of equal size and preprocess the grids;

[0065] In step 2, the RGB-D image is divided into several grids for two reasons. The first is to divide the RGB-D image into several grids and analyze each grid, which greatly reduces the computing power of the subsequent model operation, reduces the computing pressure and configuration of the model, and improves the detection efficiency. The second is to analyze each grid, which greatly improves the detection accuracy, so as to achieve fine detection and classification of each plant.

[0066] Specifically, in step 2, several grids are preprocessed, including using filtering and morphological processing on each grid image to remove noise, eliminate background interference, and highlight the diseased area; then, adaptive threshold segmentation is performed, and the threshold is dynamically adjusted according to the local brightness to accurately segment the diseased area; the preprocessing operation after grid division not only improves the detection efficiency but also ensures the detection accuracy.

[0067] Step 3: Grid detection: Several grids are input into the deep learning model for detection to obtain the pest and disease detection results of several grids; the training process of the deep learning model includes an unsupervised learning stage and a supervised fine-tuning stage:

[0068] The SimCLR model is used in the unsupervised learning stage. The RGB-D image is input into the SimCLR model, and the SimCLR model extracts the image features and depth features in the RGB-D image.

[0069] The EfficientNet model is used in the supervised fine-tuning stage. The image features and deep features in the RGB-D images extracted by the SimCLR model are input into the EfficientNet model, and the EfficientNet model is used to identify the categories of pests and diseases. The EfficientNet model is adjusted using the RGB-D images with labeled pests and diseases.

[0070] In step three, the deep learning model combines the unsupervised learning stage and the supervised fine-tuning stage. The unsupervised learning stage provides powerful feature extraction capabilities and can process unlabeled data sets, saving the time and cost of labeling data. The supervised fine-tuning stage is then used to provide specific task data for specific training and adjustment, so that the model has high accuracy and efficiency in the task of plant pest and disease detection. At the same time, the combination of unsupervised and supervised methods enables the model to better adapt to different pest and disease detection scenarios and data sets, increasing the applicability of the model.

[0071] Step 4: Result summary output: Summarize the pest and disease detection results of several grids to obtain the plant pest and disease detection results.

[0072] It should be noted here that the collection of plant three-dimensional depth information is relatively complicated, and the equipment is expensive, requiring professionals to operate, and the amount of plant three-dimensional depth information data is very large, and the model needs to consume a lot of computing resources and time when processing. This also leads to the fact that when performing plant disease and insect pest detection, the three-dimensional depth information of plants is usually not collected for cost considerations. The reason why this application combines plant three-dimensional depth information and visible light images of plants to ensure detection accuracy while not bringing huge computing power requirements and computing pressure to the model is that the deep learning model of this application combines unsupervised learning and supervised fine-tuning, and uses unsupervised learning to process a large number of unlabeled data sets, avoiding the problem of time-consuming and costly labeling of large-scale and diverse data.

[0073] This application introduces the three-dimensional depth information of plants in the detection, integrates it with the visible light image of the plants, provides more details about the plant morphology and spatial distribution, can better identify and locate field pests and diseases, and achieve more efficient and accurate detection; and grid processing is performed on RGB-D images. Grid processing can improve the overall computing efficiency; secondly, it can further improve the detection accuracy, making the final plant pest and disease detection results more intuitive and vivid; finally, through the deep learning model combined with the unsupervised learning stage and the supervised fine-tuning stage, this model can learn rich features and classification boundaries, thereby achieving high-precision pest and disease detection; and can adapt to changes in different lighting conditions, shooting angles and plant growth stages, and improve the robustness of detection. The entire detection method not only improves the accuracy of detection but also improves the efficiency of detection, so that real-time and automated pest and disease detection can be achieved in field environments, thus having broad prospects in large-scale forestry nursery management and precision forestry applications.

[0074] In a specific implementation, the specific training process of the unsupervised learning stage is as follows:

[0075] Data preparation: Collect several RGB-D images. The RGB-D images here are not labeled, that is, it is unknown whether the images contain pests and diseases. Through the subsequent SimCLR framework learning, useful features are extracted from these images.

[0076] Data augmentation: Perform data augmentation operations on each RGB-D image, including cropping, rotating, and color jittering of the RGB-D image to generate different image views that are visually different but contain the same semantic information, i.e., the characteristics of plant diseases and pests;

[0077] SimCLR model training: The SimCLR framework is used for unsupervised learning to extract image features and deep features of RGB-D images from the trained SimCLR model. The image features include the color and texture of plants. In the specific implementation, EfficientNet-B3 is used as the backbone network, and the features are mapped to a low-dimensional space through a nonlinear projection head. The NT-Xent loss function is used for contrastive learning to maximize the feature similarity of different views of the same image. Large batches, cosine annealing learning rate schedulers, and mixed precision training are used during training to improve efficiency.

[0078] In a specific implementation, the specific training process of the supervised fine-tuning stage is as follows:

[0079] Data preparation: Collect several RGB-D images with pests and diseases that are labeled; the images include pests and diseases of different types and severity, with corresponding labels.

[0080] Feature input: The image features and depth features of the RGB-D image extracted by the SimCLR model are input into the EfficientNet model, and the EfficientNet model identifies the pest category;

[0081] Model fine-tuning: Use several annotated RGB-D images with pests and diseases to adjust the EfficientNet model; specifically, use the SimCLR pre-trained EfficientNet-B3 model as the initial weight, and modify the last fully connected layer to adapt to the number of target categories. Use the cross entropy loss function in the fine-tuning process, and apply data enhancement techniques such as random cropping and flipping. Use a weighted sampler to deal with the problem of class imbalance, and use a cosine annealing learning rate strategy. Use the back-propagation algorithm for optimization in the fine-tuning process, and selectively freeze some network layers for fine-tuning.

[0082] In a specific implementation, the pest detection results of the plurality of grids include health status and disease severity, and the classification criteria are as follows:

[0083] Healthy: No signs of disease, leaves are intact, no diseases detected;

[0084] Mild disease: In a single grid, the infected area is between 0-20%, and the number of infected leaves is between 0-20%, which does not affect the overall health of the plant;

[0085] Moderate disease: In a single grid, the infected area is between 20-50%, or the number of infected leaves is between 20-50%, and the lesions are obvious;

[0086] Severe disease: In a single grid, the infected area exceeds 50%, and the number of infected leaves exceeds 50%, with large areas of leaves damaged.

[0087] In this embodiment, by determining the health or disease degree of each grid, it is possible to perform more refined detection, capture early signs of plant diseases and insect pests, and facilitate timely intervention; at the same time, because each grid is analyzed separately, the scope and degree of the disease can be determined more accurately, avoiding misjudging healthy areas as diseased areas, or missing diseased areas; more importantly, determining each grid can quickly locate the specific location of the disease, which is convenient for staff to promptly treat the symptoms; and the status of each grid can be summarized and analyzed to generate a map or heat map of the plant health status, which helps to intuitively understand the overall health status of the plant, and can also be used to predict the development trend of the disease, providing a scientific basis for future disease and insect pest control work.

[0088] In a specific implementation, when summarizing the pest detection results of several grids, the distribution of healthy grids, mildly diseased grids, moderately diseased grids and severely diseased grids is also counted to obtain healthy areas, mildly diseased areas, moderately diseased areas and severely diseased areas, and prevention and control suggestions are generated according to the corresponding areas. Specifically, the prevention and control suggestions can be referred to as follows:

[0089] Areas with mild disease: It is recommended to monitor disease progression and take preventive measures;

[0090] Moderately diseased areas: It is recommended to strengthen monitoring and local treatment, and take appropriate pesticide application or other intervention measures;

[0091] Severely diseased areas: It is recommended to immediately apply pesticides or take other prevention and control measures to prevent further spread of the disease.

[0092] In this embodiment, an overall matrix can be generated by summarizing the detection results of several grids, and each grid in the matrix corresponds to a corresponding classification result (healthy, mild disease, moderate disease, severe disease); the role of the matrix is ​​to intuitively display the health status of each grid and provide a basis for global disease analysis. And by summarizing the number of grids of each category in the entire area, the distribution of healthy grids, mild disease grids, moderate disease grids and severe disease grids is counted, the proportion of diseased grids (number of diseased grids / total number of grids) is calculated, and the proportions are calculated according to mild, moderate and severe diseases, respectively, to obtain a spatial distribution map, intuitively displaying the location of disease concentration areas and healthy areas, so that users can understand the spread trend of the disease and help determine whether local intervention or full-area treatment is needed.

[0093] A system using the plant disease and insect pest detection method as described above comprises:

[0094] Perception module: used for data collection, obtaining visible light images of plants in the nursery and three-dimensional depth information of plants to obtain RGB-D images;

[0095] Computing module: used to grid and preprocess the RGB-D image, then enter the deep learning model for detection, and finally output the plant disease and pest detection results.

[0096] The system introduces three-dimensional depth information of plants. Compared with the traditional single spectral image detection method, it can capture the surface characteristics and structural information of plants more comprehensively, greatly reducing the impact of environmental factors on the detection results, thereby improving the accuracy of pest and disease detection and the robustness of the detection process; and the system has a simple structure and performs real-time processing and analysis while collecting data, which not only reduces the complexity of installation and maintenance, but also enhances the real-time nature of the entire detection process.

[0097] In a specific embodiment, the perception module includes:

[0098] Camera unit: used to collect visible light images of plants in the nursery and three-dimensional depth information of the plants;

[0099] Heat dissipation unit: used to dissipate heat for the camera unit;

[0100] Power supply unit: used to supply power to the camera unit and the cooling unit.

[0101] The setting of the heat dissipation unit enables the camera unit in the perception module to maintain stable performance under long-term working conditions, which will not affect the shooting accuracy, and can ensure that the entire perception module is in an environment with suitable temperature, ensuring the normal operation of the unit components in the perception module.

[0102] In a specific embodiment, the calculation module includes:

[0103] Jetson Xavier NX platform: used for gridding and preprocessing of RGB-D images, as well as plant pest and disease detection and analysis;

[0104] Display unit: used to output and display the plant pest and disease detection and analysis results in the Jetson Xavier NX platform;

[0105] Cooling unit: used to dissipate heat for the Jetson Xavier NX platform.

[0106] By leveraging the powerful AI computing capabilities and low power consumption of the Jetson Xavier NX platform, it is able to process large amounts of data from the perception module in real time and accurately identify the types of plant diseases and pests, making the entire system more energy-efficient and environmentally friendly. The setting of the heat dissipation unit further ensures that the Jetson Xavier NX platform can still work stably under high temperature conditions, avoiding performance degradation or device failure due to overheating, and improving the safety of the computing module.

[0107] Example 1

[0108] In order to further facilitate the examiner to understand the present application, the following example is given: the plant disease and insect pest detection method described in the present application is applied in a certain nursery, which specifically includes the following steps:

[0109] Step 1: Image acquisition: Use an RGB-D camera to collect RGB-D images of multiple plants in a nursery area. The color and texture information of the plants are obtained through RGB images, and the three-dimensional structure of the plants is recorded through depth images. Especially when the leaves of the plants are blocked or overlapped, the depth information is used to distinguish the foreground and background.

[0110] Step 2: Image gridding and preprocessing: Divide the RGB-D image of each plant into multiple grid areas of equal size, and perform independent pest and disease detection on each grid. That is, the subsequent operation analyzes the disease type, health status and severity of each grid separately.

[0111] Perform noise removal and morphological processing on each grid to eliminate background interference in the collected grid image, such as soil or other vegetation; at the same time, adaptive threshold segmentation is used to accurately extract the pest and disease area;

[0112] Step 3: Grid detection: Each grid is input into the deep learning model for detection, and SimCLR is used for representation learning on unlabeled data to extract the color, texture, and depth features of the plant. Then, the EfficientNet model is fine-tuned in a supervised manner through the labeled data for pest and disease classification;

[0113] Step 4: Result summary output: Summarize the results of grid detection, generate a spatial distribution map of the disease, and accurately locate the pest area; and when summarizing the results, remove duplicate detection frames through the non-maximum suppression algorithm, and finally generate a health status report for the entire plant, including the type, proportion and health status (mild, moderate, severe) of each pest.

[0114] The examples described in the present invention are merely descriptions of the preferred implementation modes of the present invention, and are not intended to limit the concept and scope of the present invention. Without departing from the design concept of the present invention, various modifications and improvements made to the technical solutions of the present invention by engineers and technicians in this field should all fall within the protection scope of the present invention.

Claims

1. A method for detecting plant diseases and insect pests, characterized in that: The following steps are involved: Image acquisition: Obtain visible light images of plants in the nursery and three-dimensional depth information of the plants to obtain RGB-D images; Image gridding: Divide the RGB-D image into several grids of equal size and preprocess the grids; Grid detection: Several grids are input into the deep learning model for detection to obtain the pest and disease detection results of several grids; the training process of the deep learning model includes an unsupervised learning stage and a supervised fine-tuning stage: The SimCLR model is used in the unsupervised learning stage. The RGB-D image is input into the SimCLR model, and the SimCLR model extracts the image features and depth features in the RGB-D image. The EfficientNet model is used in the supervised fine-tuning stage. The image features and deep features in the RGB-D images extracted by the SimCLR model are input into the EfficientNet model, and the EfficientNet model is used to identify the categories of pests and diseases. The EfficientNet model is adjusted using the RGB-D images with labeled pests and diseases. Result summary output: The pest and disease detection results of several grids are summarized to obtain the plant pest and disease detection results.

2. A method for detecting plant diseases and insect pests according to claim 1, characterized in that: When performing image acquisition, the RGB-D camera is used to simultaneously obtain the visible light image of the plants in the nursery and the three-dimensional depth information of the plants; and the RGB-D camera is used to obtain several RGB-D images at different heights and / or different angles.

3. A method for detecting plant diseases and insect pests according to claim 1, characterized in that: When performing image acquisition, an RGB camera is used to collect visible light images of plants in the nursery; a depth camera is used to collect three-dimensional depth information of plants in the nursery; and both the RGB camera and the depth camera are placed 1m to 1.4m away from the plant's apical buds.

4. A method for detecting plant diseases and insect pests according to claim 1, characterized in that: The specific training process of the unsupervised learning stage is as follows: Data preparation: Collect several RGB-D images; Data enhancement: Perform data enhancement operations on each RGB-D image; SimCLR model training: The SimCLR framework is used for unsupervised learning to extract image features and depth features of RGB-D images from the trained SimCLR model, where the image features include the color and texture of the plants.

5. A method for detecting plant diseases and insect pests according to claim 1 or 4, characterized in that: The specific training process of the supervised fine-tuning stage is as follows: Data preparation: Collect several RGB-D images with pests and diseases that are labeled; the images include pests and diseases of different types and severity, with corresponding labels. Feature input: The image features and depth features of the RGB-D image extracted by the SimCLR model are input into the EfficientNet model, and the EfficientNet model identifies the pest category; Model fine-tuning: Use several labeled RGB-D images with pests and diseases to adjust the EfficientNet model; specifically, the back propagation algorithm is used to optimize the model during the adjustment process, and some or all of the network layers of the EfficientNet model can be frozen for adjustment.

6. A method for detecting plant diseases and insect pests according to claim 1, characterized in that: The pest and disease detection results of the plurality of grids include health status and disease severity, and the classification criteria are as follows: Healthy: No signs of disease, leaves are intact, no diseases detected; Mild disease: In a single grid, the infected area is between 0-20%, and the number of infected leaves is between 0-20%, which does not affect the overall health of the plant; Moderate disease: In a single grid, the infected area is between 20-50%, or the number of infected leaves is between 20-50%, and the lesions are obvious; Severe disease: In a single grid, the infected area exceeds 50%, and the number of infected leaves exceeds 50%, with large areas of leaves damaged.

7. A method for detecting plant diseases and insect pests according to claim 6, characterized in that: When summarizing the pest and disease detection results of several grids, it also includes counting the distribution of healthy grids, lightly diseased grids, moderately diseased grids and severely diseased grids, and then obtaining healthy areas, lightly diseased areas, moderately diseased areas and severely diseased areas, and generating prevention and control recommendations based on the corresponding areas.

8. A system using the plant disease and insect pest detection method according to any one of claims 1 to 7, characterized in that: include: Perception module: used for data collection, obtaining visible light images of plants in the nursery and three-dimensional depth information of plants to obtain RGB-D images; Computing module: used to grid and preprocess the RGB-D image, then enter the deep learning model for detection, and finally output the plant disease and pest detection results.

9. A plant disease and insect pest detection system according to claim 8, characterized in that: The perception module comprises: Camera unit: used to collect visible light images of plants in the nursery and three-dimensional depth information of the plants; Heat dissipation unit: used to dissipate heat for the camera unit; Power supply unit: used to supply power to the camera unit and the cooling unit.

10. A plant disease and insect pest detection system according to claim 8, characterized in that: The calculation module comprises: Jetson Xavier NX platform: used for gridding and preprocessing of RGB-D images, as well as plant pest and disease detection and analysis; Display unit: used to output and display the plant pest and disease detection and analysis results in the Jetson Xavier NX platform; Cooling unit: used to dissipate heat for the Jetson Xavier NX platform.

Images