System and method for detecting plant diseases and insect pests of nursery plants

Abstract

The invention discloses a nursery plant disease and insect pest detection system and method, and belongs to the field of forestry production. In order to solve the problems that existing plant disease and insect pest detection is poor in precision and long in consumed time, the nursery plant disease and insect pest detection system comprises a sensing unit used for collecting data and a calculation unit used for processing the data, and the sensing unit comprises a camera assembly. The camera assembly is used for collecting visible light images of nursery plants and three-dimensional depth information of the nursery plants; the calculation unit comprises a calculation assembly, and the calculation assembly is used for carrying out the image feature extraction and depth feature extraction of the visible light image of the nursery plant and the three-dimensional depth information of the plant in an unsupervised learning mode. According to the image features and the depth features, disease and pest categories of nursery plants are detected in a supervised fine adjustment mode. The whole detection system not only improves the detection accuracy, but also improves the detection efficiency, so that real-time and automatic disease and pest detection can be realized in a field environment, and the system is suitable for large-scale popularization and application; the whole detection method is helpful to more accurately judge positions and ranges of diseases and insect pests, and can still realize accurate detection of the diseases and insect pests under more complex conditions.

Description

Technical Field

[0001] The invention belongs to the technical field of forestry production, and more specifically, relates to a nursery plant disease and insect pest detection system and method. Background Art

[0002] In modern forestry production, the health management of nursery plants is an important part of improving crop yield and quality. Existing pest and disease detection technologies usually rely on RGB cameras or a single type of sensor to identify plant pests and diseases through image processing and machine learning algorithms. However, these technical solutions have obvious shortcomings. First, existing devices usually rely only on image data of the visible spectrum, which is easily affected by changes in light and environmental factors, resulting in low detection accuracy. Secondly, these devices have limited coverage and can usually only detect single plants or local areas, making it difficult to achieve large-scale monitoring in a vast nursery environment. In addition, the real-time performance of traditional pest and disease detection devices is poor, and data usually needs to be uploaded to the server for centralized processing. This processing method will cause time delays and affect the timely response to pests and diseases. Finally, most of the existing devices use traditional machine learning algorithms, which are prone to misjudgment or missed judgment when facing complex environments and changeable pest and disease forms. Therefore, the existing technology has significant deficiencies in detection accuracy, coverage, real-time performance, and algorithm performance, and is in urgent need of improvement.

[0003] Corresponding improvements have also been made to the above problems, such as China Patent Application No. CN202211114174.1, published on November 29, 2022. The patent discloses an automatic detection device for plant diseases and insect pests and an automatic detection method for plant diseases and insect pests. The automatic detection device for plant diseases and insect pests includes a shell, an imaging component, an environmental detection component and a control device compartment. The imaging component includes a camera and a servo pan / tilt, the servo pan / tilt is fixedly connected to the top of the shell, the camera is connected to the servo pan / tilt, the environmental detection component includes a meteorological detector and a soil rapid detector, the meteorological detector is fixedly connected to the top of the shell, the soil rapid detector includes a housing and a pin for inserting into the soil to collect soil parameters, the housing is vertically slidably connected to the side wall of the housing, the pin is fixedly connected to the housing, the control device compartment is fixedly connected to the housing, and a program controller for controlling the working state of the automatic detection device for plant diseases and insect pests is installed in the control device compartment. The shortcomings of this patent are: although the accuracy of plant disease and insect pest detection is improved, the setting of the environmental monitoring component increases the cost, and the components are prone to mutual interference and affect the detection effect.

[0004] Another example is Chinese patent application number CN202410362779.5, which was published on March 28, 2024. The patent discloses a method for rapid detection of pest and disease areas of garden plants, including: obtaining the target degree value of each suspected pest and disease pixel point and the target pixel point according to the number of suspected pest and disease pixels in the neighborhood range of each suspected pest and disease pixel point and the distance between the suspected pest and disease pixels; obtaining the decay probability of each target pixel point according to the grayscale distribution of the pixel points in the neighborhood range of each target pixel point and the texture value in the neighborhood range, the target degree value of each target pixel point, and the difference in the slope of adjacent edge pixels; obtaining the pest and disease area in the leaf image according to the decay probability of each target pixel point. The disadvantage of this patent is that although the influence of light on pest and disease detection is reduced, the calculation is complex and time-consuming, which increases the cost. Summary of the invention

[0005] 1. Problems to be solved

[0006] In view of the problem that the existing plant pest and disease detection has poor accuracy and is time-consuming, the present invention provides a nursery plant pest and disease detection system and method. The entire detection system improves both the accuracy and efficiency of detection, enabling real-time and automated pest and disease detection in a field environment, which is suitable for large-scale promotion and use; the entire detection method helps to more accurately determine the location and range of pests and diseases, and can still accurately detect pests and diseases under relatively complex conditions.

[0007] 2. Technical solution

[0008] To solve the above problems, the present invention adopts the following technical solutions.

[0009] A nursery plant pest and disease detection system includes a perception unit for collecting data and a computing unit for processing the data, wherein the perception unit includes a camera component for collecting visible light images of nursery plants and three-dimensional depth information of the nursery plants; the computing unit includes a computing component for first using an unsupervised learning method to extract image features and depth features from the visible light images of nursery plants and the three-dimensional depth information of the plants, and then using a supervised fine-tuning method to detect the pest and disease categories of the nursery plants based on the image features and the depth features.

[0010] By adopting the above technical solution, by introducing the three-dimensional depth information of the plant into the data acquisition of the detection system, compared with the traditional single spectral image detection method, the three-dimensional depth information of the plant can more comprehensively capture the surface characteristics and structural information of the plant, greatly reducing the impact of environmental factors on the detection results, thereby improving the accuracy of pest and disease detection and the robustness of the detection process; and the calculation method of the computing component is to extract features from the data first and then detect the features, which greatly improves the computing efficiency and improves the calculation accuracy.

[0011] In general, the nursery plant pest and disease detection system of the present application achieves high-precision detection of pests and diseases by collecting multi-dimensional data and performing data fusion; and the system has a simple structure, and the sensing unit and the computing unit are electrically connected to each other, which also increases the real-time nature of the entire detection process. The data collected by the sensing unit can be quickly entered into the computing unit for calculation, ensuring the timeliness of the entire process, saving costs while ensuring accuracy and efficiency, and enabling real-time and automated pest and disease detection in field environments, which is suitable for large-scale promotion and use.

[0012] Furthermore, the camera assembly includes an RGB camera and a depth camera, the RGB camera collects visible light images of the nursery plants; the depth camera collects three-dimensional depth information of the nursery plants; and both the RGB camera and the depth camera are placed 1m to 1.4m away from the plant's apical buds.

[0013] By adopting the above technical solution, different cameras are used to collect different data, so that the collection of two visible light images and three-dimensional depth information does not interfere with each other, which facilitates real-time adjustment of different camera positions; and the positions of the two cameras are limited so that the collection area maintains an appropriate number to avoid the collection area being too small affecting the overall detection efficiency, and the collection area being too large affecting the overall detection accuracy.

[0014] Furthermore, the camera assembly is arranged on an adjustable bracket, and the adjustable bracket can change the height of the camera assembly from the ground and change the shooting angle of the camera assembly.

[0015] By adopting the above technical solution, the setting of the adjustable bracket makes the position of the camera component adjustable, so that information of various parts and levels of the plant can be obtained, and plant information of different perspectives and levels can be provided, which increases the comprehensiveness of the data source and further improves the accuracy of subsequent detection results.

[0016] Furthermore, the sensing unit further includes:

[0017] Heat dissipation component: used to dissipate heat from camera components;

[0018] Power supply assembly: used to supply power to the camera assembly and the heat dissipation assembly.

[0019] By adopting the above technical solution, the setting of the heat dissipation component enables the camera component in the perception unit to maintain stable performance under long-term working conditions, which will not affect the shooting accuracy, and can ensure that the entire perception unit is in an environment with suitable temperature, thereby ensuring the normal operation of the unit components in the perception unit.

[0020] Furthermore, the computing component is a deep learning model, and the training process of the deep learning model includes an unsupervised learning phase and a supervised fine-tuning phase:

[0021] The SimCLR model is used in the unsupervised learning stage. The visible light images and three-dimensional depth information of the nursery plants are input into the SimCLR model, and the SimCLR model extracts the image features and depth features of the nursery plants.

[0022] The EfficientNet model is used in the supervised fine-tuning stage. The image features and depth features extracted by the SimCLR model are input into the EfficientNet model, and the EfficientNet model is used to identify the types of pests and diseases. The EfficientNet model is adjusted using the visible light images and three-dimensional depth information of the labeled pests and diseases.

[0023] By adopting the above technical solution, the deep learning model combines the unsupervised learning stage and the supervised fine-tuning stage. The unsupervised learning stage provides powerful feature extraction capabilities and can process unlabeled data sets, saving the time and cost of labeling data. The supervised fine-tuning stage then provides specific task data for specific training and adjustment, so that the model has high accuracy and efficiency in the task of plant disease and pest detection. At the same time, the combination of unsupervised and supervised methods enables the model to better adapt to different disease and pest detection scenarios and data sets, increasing the applicability of the model.

[0024] Furthermore, the computing unit comprises:

[0025] Jetson Xavier NX computing platform: used for performing pest and disease detection on nursery plants, and the Jetson XavierNX computing platform includes the deep learning model;

[0026] Display component: used to display the test results;

[0027] Heat dissipation component: used to dissipate heat for the Jetson Xavier NX computing platform.

[0028] The above technical solution, using the powerful AI computing capabilities and low power consumption characteristics of the Jetson Xavier NX computing platform, can process large amounts of data from the perception unit in real time and accurately identify the types of plant diseases and pests, making the entire system more energy-efficient and environmentally friendly. The setting of the heat dissipation component further ensures that the Jetson Xavier NX computing platform can still work stably under high temperature conditions, avoiding performance degradation or device failure due to overheating, and improving the safety of the computing unit.

[0029] Furthermore, the computing unit also includes a plurality of reserved interfaces, and the plurality of reserved interfaces are used to connect with the sensing unit for data and power.

[0030] By adopting the above technical solution, the setting of several reserved interfaces makes the connection between the computing unit and the perception unit highly flexible and operable, which is convenient for adjustment in response to different actual usage situations and increases the applicability of the entire system.

[0031] A detection method of a nursery plant pest detection system as described in any one of the above items comprises the following steps:

[0032] The perception unit collects visible light images of the nursery plants and three-dimensional depth information of the nursery plants;

[0033] The computing unit performs computational analysis on the collected visible light images and three-dimensional depth information of the nursery plants to obtain the disease and insect detection results; the computing process is as follows: first, image features and depth features are extracted from the visible light images and three-dimensional depth information of the nursery plants; then, the extracted image features and depth features are calculated to obtain the disease and insect detection results.

[0034] By adopting the above technical scheme, the three-dimensional depth information of the plant is introduced into the nursery plant pest and disease detection method. The three-dimensional depth information provides the spatial relationship between leaves, which helps to more accurately determine the location and range of pests and diseases. The three-dimensional depth information of the plant also enhances the robustness of the computing unit, because it is insensitive to changes in lighting conditions, which helps the computing unit to accurately detect pests and diseases even in poor lighting conditions. The entire detection method not only improves the accuracy of detection but also improves the efficiency of detection, and thus has broad prospects in large-scale forestry nursery management and precision forestry applications.

[0035] Furthermore, before the computing unit performs calculations, it also includes fusing the visible light image of the nursery plants with the three-dimensional depth information of the nursery plants to obtain an RGB-D image; then dividing the RGB-D image into a number of grids of equal size, and performing preprocessing on the several grids before calculating each grid.

[0036] By adopting the above technical solution, the visible light image and the three-dimensional depth information are fused to obtain an RGB-D image, which is convenient for subsequent operations; and the grid processing can improve the overall computing efficiency; secondly, it can further improve the detection accuracy, making the final plant disease and pest detection results more intuitive and vivid, which is convenient for timely response and adjustment. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a schematic diagram of the overall structure of the sensing unit of the present invention;

[0038] Figure 2 It is a schematic diagram of the overall structure of the computing unit of the present invention;

[0039] Figure 3 It is a front structural schematic diagram of the sensing unit of the present invention;

[0040] Figure 4 is a schematic diagram of the internal structure of the sensing unit of the present invention;

[0041] Figure 5 It is a structural schematic diagram of the heat dissipation assembly of the present invention;

[0042] Figure 6 It is a front structural schematic diagram of the computing unit of the present invention;

[0043] Figure 7 is a schematic diagram of the internal structure of the computing unit of the present invention;

[0044] Figure 8 It is a structural schematic diagram of a controller of a computing unit of the present invention;

[0045] Fig. 9 It is a schematic diagram of the system flow of the present invention.

[0046] In the figure: 1. First heat dissipation component; 11. Cooling fan; 12. Fixing piece; 2. Internal fixing plate; 3. Switch button; 4. Battery component; 5. Heat dissipation plate; 6. POE interface; 7. Camera component 1; 8. Camera fixing plate; 9. Camera component 2;

[0047] 21. Heat dissipation hole; 22. First power interface; 23. First USB interface; 24. First network cable interface; 25. Fixing interface; 26. Bottom plate; 27. Second heat dissipation component; 28. Jetson Xavier NX development board; 281. CSI interface; 282. Second USB interface; 283. HDMI interface; 284. Second power interface; 285. USB power interface; 286. Second network cable interface; 287. 20x2 pin strip; 288. Fan power cable; 289. Cooling fan; 29. ​​Display component. DETAILED DESCRIPTION

[0048] The present invention is further described below in conjunction with specific embodiments and drawings.

[0049] like Figure 1 , Figure 2 and Fig. 9 As shown,

[0050] A nursery plant pest and disease detection system includes a perception unit for collecting data and a computing unit for processing the data, wherein the perception unit includes a camera component for collecting visible light images of nursery plants and three-dimensional depth information of the nursery plants; the computing unit includes a computing component for first using an unsupervised learning method to extract image features and depth features from the visible light images of nursery plants and the three-dimensional depth information of the plants, and then using a supervised fine-tuning method to detect the pest and disease categories of the nursery plants based on the image features and the depth features.

[0051] Specifically, one of the feasible ways is: the connection between the perception unit and the computing unit is carried out through POE, the power of the perception unit is provided by a built-in battery, and the power of the computing unit is provided by an external power supply. After the perception unit works normally and collects data, the data is transmitted to the computing unit through the POE interface, and the computing unit outputs and displays the results after processing the data.

[0052] It should be noted here that, given that the collection operation of plant three-dimensional depth information is relatively complicated and the equipment is expensive, it requires professional personnel to operate, and the amount of plant three-dimensional depth information data is very large, the model needs to consume a lot of computing resources and time when processing, which leads to the fact that when performing plant disease and insect pest detection, the three-dimensional depth information of plants is usually not collected for cost considerations. The reason why this application combines plant three-dimensional depth information and visible light images of plants to ensure detection accuracy while not bringing huge computing power requirements and computing pressure to the model is that the computing unit of this application combines unsupervised learning and supervised fine-tuning for calculation, and uses unsupervised learning to process a large number of unlabeled data sets, avoiding the problem of time-consuming and costly labeling of large-scale and diverse data.

[0053] In general, in the nursery plant pest and disease detection system of the present application, by introducing the three-dimensional depth information of the plant in data collection, compared with the traditional single spectral image detection method, the three-dimensional depth information of the plant can more comprehensively capture the surface characteristics and structural information of the plant, greatly reducing the impact of environmental factors on the detection results; and the system has a simple structure, and the perception unit and the computing unit are electrically connected to each other, which also increases the real-time performance of the entire detection process. The data collected by the perception unit can be quickly entered into the computing unit for calculation, ensuring the timeliness of the entire process, saving costs while ensuring accuracy and efficiency, and enabling real-time and automated pest and disease detection in field environments, which is suitable for large-scale promotion and use.

[0054] In one embodiment, the camera assembly includes an RGB camera and a depth camera, the RGB camera collects visible light images of nursery plants; the depth camera collects three-dimensional depth information of nursery plants, and then fuses the visible light images of nursery plants with the three-dimensional depth information to obtain an RGB-D image; and the RGB camera and the depth camera are both placed 1m to 1.4m away from the plant's terminal bud. In this embodiment, different devices are used to collect different data, so that the visible light image and the three-dimensional depth information of the plant do not interfere with each other, and different cameras can be adjusted in real time to obtain the required data, which increases the operability and flexibility of the entire collection process. In addition, the positions of the two cameras are limited to ensure that when the two cameras work together, 2-3 plug trays are covered in a single detection (plug trays refer to containers containing nursery plants, and each plug tray usually holds 20-30 plants), and a larger area of ​​plant areas can be monitored in a single scan, so that the two cameras can collect the appropriate area when performing a single collection, avoiding the problem of too small area detection taking a long time and too large area detection accuracy being poor, and being suitable for wide application in large-scale nursery environments.

[0055] In one embodiment, the camera assembly is arranged on an adjustable bracket, and the adjustable bracket can change the height of the camera assembly from the ground and change the shooting angle of the camera assembly. Specifically, in this embodiment, the adjustable bracket can be arranged in the structure of a telescopic rod, and the camera assembly is rotatably arranged on the top of the telescopic rod, and the telescopic rod itself can be extended and retracted to change the height of the camera assembly from the ground; and because it is rotatably arranged on the top of the telescopic rod, the camera assembly can rotate around the telescopic rod to achieve the adjustable shooting angle of the camera assembly.

[0056] The setting of the adjustable bracket makes the entire data collection process highly flexible and operational, and can be adjusted in real time according to the detection requirements, thereby optimizing the detection coverage. And by changing the height and shooting angle of the camera component, information on various parts and levels of the plant can be obtained, and plant information at different perspectives and levels can be provided, which increases the comprehensiveness of the data source and thereby improves the accuracy of subsequent detection results.

[0057] In one embodiment, the sensing unit further includes:

[0058] Heat dissipation component: used to dissipate heat from camera components;

[0059] Power supply assembly: used to supply power to the camera assembly and the heat dissipation assembly; the input voltage of the power supply assembly is 220V, and the output voltage is adjusted according to the equipment requirements. The working voltage of the camera assembly is 12V, while the working voltage of the heat dissipation assembly is 5V. The power supply assembly is reasonably designed to ensure the reliable operation of the perception unit in different working environments.

[0060] Specifically, in this embodiment, in order for the examiner to further understand the perception unit of the present application, the perception unit is explained as follows: Figure 3 , Figure 4 and Figure 5 As shown, the perception unit includes a shell, on which a camera assembly 1 7 and a camera assembly 2 9 are arranged. The camera assembly 1 7 is used to collect visible light images of plants, and the camera assembly 2 9 is used to collect three-dimensional depth information of plants; the camera assembly 1 7 and the camera assembly 2 9 are fixed to the shell by a camera fixing plate 8 and an inner fixing plate 2, respectively, to increase the stability of the camera assembly; a first heat dissipation assembly 1 and a battery assembly 4 are arranged inside the shell, and the battery assembly 4 supplies power to the first heat dissipation assembly 1 and the two camera assemblies to ensure their normal operation; the first heat dissipation assembly 1 specifically includes a heat dissipation fan 11, and the heat dissipation fan 11 is fixed inside the shell by a fixing member 12. A heat dissipation plate 5 and a POE interface 6 are also arranged on the shell, and the heat dissipation plate 5 further improves the heat dissipation effect of the entire perception unit; the POE interface 6 is used for connection with the computing unit. At the same time, in order to facilitate the operability of the camera assembly and the heat dissipation assembly, a switch button 3 is also arranged on the surface of the shell, and the switch button 3 is used to control the operation or stop of the camera assembly and the heat dissipation assembly.

[0061] The camera components will generate a lot of heat energy when working for a long time, which may cause the performance of the camera components to decline and even affect the detection accuracy. In order to prevent the accumulation of heat energy, the perception unit is equipped with a dedicated heat dissipation component. The heat dissipation component effectively reduces the operating temperature of the camera components through the cooling fan, ensuring that the camera components operate in an efficient and stable state.

[0062] In one embodiment, the computing component is a deep learning model, and the training process of the deep learning model includes an unsupervised learning phase and a supervised fine-tuning phase:

[0063] The SimCLR model is used in the unsupervised learning stage. The visible light images and three-dimensional depth information of the nursery plants are input into the SimCLR model, and the SimCLR model extracts the image features and depth features of the nursery plants.

[0064] The EfficientNet model is used in the supervised fine-tuning stage. The image features and depth features extracted by the SimCLR model are input into the EfficientNet model, and the EfficientNet model is used to identify the types of pests and diseases. The EfficientNet model is adjusted using the visible light images and three-dimensional depth information of the labeled pests and diseases.

[0065] In one embodiment, the computing unit comprises:

[0066] Jetson Xavier NX computing platform: used for performing pest and disease detection on nursery plants, and the Jetson XavierNX computing platform includes the deep learning model;

[0067] Display component: used to display the test results;

[0068] Heat dissipation component: used to dissipate heat for the Jetson Xavier NX computing platform.

[0069] like Figure 6 , Figure 7 and Figure 8As shown, in this embodiment, in order for the examiner to further understand the computing unit of the present application, the computing unit is explained as follows: the computing unit includes a base plate 26, and a fixing part interface 25 is provided on the base plate 26, so that the fixing part passes through the fixing part interface 25 to fix the base plate 26 on an external object; a rectangular protective shell is provided on the base plate 26, and a Jetson Xavier NX development board 28 is provided inside the protective shell. The Jetson Xavier NX development board 28 is used as a computing component to detect the types of pests and diseases of nursery plants; a second heat dissipation component 27 is also provided inside the protective shell, and the second heat dissipation component 27 is used to heat the Jetson Xavier The NX development board 28 dissipates heat, so that it can still maintain high performance under long-term working conditions; at least one side of the protective shell is provided with a heat dissipation hole 21, and the heat dissipation hole 21 further improves the heat dissipation effect of the entire computing unit; the side of the protective shell is also provided with a first power interface 22, a first USB interface 23 and a first network cable interface 24, the first power interface 22, the first USB interface 23 can be used to connect with the perception unit to realize data transmission and power connection between the perception unit; the first network cable interface 24 is used to connect with the external network to facilitate communication between the computing unit and the outside world; the top surface of the protective shell is provided with a display component 29 for displaying the detection results of the Jetson Xavier NX development board 28, and the display component 29 adopts a high-brightness, high-resolution LCD or LED screen to ensure that it can still be clearly displayed under strong light. The screen size is moderate, which not only meets the needs of information display, but also does not take up too much space. The display component 29 may also be equipped with a touch function or a key operation interface to facilitate the operator to set up the device and adjust the parameters. In this way, the user can adjust the working mode of the device or check the device status according to the specific situation on site.

[0070] More specifically, the Jetson Xavier NX development board 28 is provided with a CSI interface 281, a second USB interface 282, an HDMI interface 283, a second power interface 284, a USB power interface 285, a second network cable interface 286 and a 20x2 pin strip 287, as well as a fan power cable 288 and a cooling fan 289; each interface corresponds to a different function. The specifications of the Jetson Xavier NX development board 28 are as follows: Jetson Xavier NX 8GB is equipped with a 384-core NVIDIA Volta GPU and 48 Tensor cores, supports 21TOPS of AI performance, is equipped with a 6-core NVIDIA Carmel CPU (1.9GHz), and 8GB LPDDR4x memory. These configurations enable the computing unit to achieve high-performance embedded AI computing at low power consumption, which is suitable for real-time processing of pest and disease detection tasks.

[0071] The computing unit in this application adopts an embedded Jetson computing platform. Its low-power design is more energy-saving and environmentally friendly than traditional large-scale server processing solutions. The efficient processing capability of the Jetson computing platform ensures that the device can output the detection results in a short time, realizing real-time monitoring and early warning of pests and diseases. This high real-time performance helps to take quick measures in the early stages of pests and diseases to avoid the spread of diseases and crop losses.

[0072] In one embodiment, the computing unit also includes a plurality of reserved interfaces, which are used to connect data and power to the sensing unit. The setting of a plurality of reserved interfaces makes the connection between the computing unit and the sensing unit highly flexible and operable, and is easy to adjust in the face of different actual usage situations, thereby increasing the applicability of the entire system. Specifically, the power supply of the Jetson Xavier NX computing platform can be provided by the power supply component of the sensing unit, which simplifies the power configuration of the system; or the camera component can be powered by the USB interface of the Jetson Xavier NX computing platform to ensure the stability of the power supply.

[0073] A detection method of a nursery plant pest detection system as described in any one of the above items comprises the following steps:

[0074] The perception unit collects visible light images of the nursery plants and three-dimensional depth information of the nursery plants;

[0075] The computing unit performs computational analysis on the collected visible light images and three-dimensional depth information of the nursery plants to obtain the results of disease and insect detection; the computational process is as follows: first, an unsupervised learning method is used to extract image features and depth features from the visible light images and three-dimensional depth information of the nursery plants; then a supervised fine-tuning method is used to calculate the extracted image features and depth features to obtain the results of disease and insect detection, which include the types of disease and insect pests.

[0076] Specifically, the computing unit uses a deep learning model to detect plant diseases and pests when performing computational analysis. The training process of the deep learning model includes an unsupervised learning stage and a supervised fine-tuning stage: the SimCLR model is used in the unsupervised learning stage, and the RGB-D image is input into the SimCLR model, and the SimCLR model extracts image features and depth features in the RGB-D image; the EfficientNet model is used in the supervised fine-tuning stage, and the image features and depth features in the RGB-D image extracted by the SimCLR model are input into the EfficientNet model, and the EfficientNet model identifies the category of diseases and pests; and the EfficientNet model is adjusted using the RGB-D images marked with diseases and pests.

[0077] The specific training process of the unsupervised learning stage is as follows:

[0078] Data preparation: Collect several RGB-D images. The RGB-D images here are not labeled, that is, it is unknown whether the images contain pests and diseases. Through the subsequent SimCLR framework learning, useful features are extracted from these images.

[0079] Data augmentation: Perform data augmentation operations on each RGB-D image, including cropping, rotating, and color jittering of the RGB-D image to generate different image views that are visually different but contain the same semantic information, i.e., the characteristics of plant diseases and pests;

[0080] SimCLR model training: The SimCLR framework is used for unsupervised learning to extract image features and deep features of RGB-D images from the trained SimCLR model. The image features include the color and texture of plants. In the specific implementation, EfficientNet-B3 is used as the backbone network, and the features are mapped to a low-dimensional space through a nonlinear projection head. The NT-Xent loss function is used for contrastive learning to maximize the feature similarity of different views of the same image. Large batches, cosine annealing learning rate schedulers, and mixed precision training are used during training to improve efficiency.

[0081] The specific training process of the supervised fine-tuning stage is as follows:

[0082] Data preparation: Collect several RGB-D images with pests and diseases that are labeled; the images include pests and diseases of different types and severity, with corresponding labels.

[0083] Feature input: The image features and depth features of the RGB-D image extracted by the SimCLR model are input into the EfficientNet model, and the EfficientNet model identifies the pest category;

[0084] Model fine-tuning: Use several annotated RGB-D images with pests and diseases to adjust the EfficientNet model; specifically, use the SimCLR pre-trained EfficientNet-B3 model as the initial weight, and modify the last fully connected layer to adapt to the number of target categories. Use the cross entropy loss function in the fine-tuning process, and apply data enhancement techniques such as random cropping and flipping. Use a weighted sampler to deal with the problem of class imbalance, and use a cosine annealing learning rate strategy. Use the back-propagation algorithm for optimization in the fine-tuning process, and selectively freeze some network layers for fine-tuning.

[0085] This application introduces the three-dimensional depth information of plants in the detection and fuses it with the visible light image of the plants to provide more details about the plant morphology and spatial distribution, which can better identify and locate field pests and diseases and achieve more efficient and accurate detection; through the deep learning model combined with the unsupervised learning stage and the supervised fine-tuning stage, this model can learn rich features and classification boundaries, thereby achieving high-precision pest and disease detection; and can adapt to changes in different lighting conditions, shooting angles and plant growth stages to improve the robustness of detection. The entire detection method not only improves the accuracy of detection but also improves the efficiency of detection, making it possible to achieve real-time and automated pest and disease detection in a field environment, thus having broad prospects in large-scale forestry nursery management and precision forestry applications.

[0086] In one embodiment, before the computing unit performs calculations, it also includes fusing the visible light image of the nursery plants and the three-dimensional depth information of the nursery plants to obtain an RGB-D image; then dividing the RGB-D image into a number of grids of equal size, and performing calculations on each grid after preprocessing the several grids.

[0087] Specifically, in this embodiment, the pest detection result includes the pest type, health status and disease degree of the plant; the classification criteria of the plant health status and disease degree are as follows:

[0088] Healthy: No signs of disease, leaves are intact, no diseases detected;

[0089] Mild disease: In a single grid, the infected area is between 0-20%, and the number of infected leaves is between 0-20%, which does not affect the overall health of the plant;

[0090] Moderate disease: In a single grid, the infected area is between 20-50%, or the number of infected leaves is between 20-50%, and the lesions are obvious;

[0091] Severe disease: In a single grid, the infected area exceeds 50%, and the number of infected leaves exceeds 50%, with large areas of leaves damaged.

[0092] By determining the health or disease level of each grid, more refined detection can be performed and computational efficiency can be improved, so that early signs of plant diseases and pests can be captured, facilitating timely intervention. At the same time, because each grid is analyzed separately, the scope and extent of the disease can be determined more accurately, avoiding misjudging healthy areas as diseased areas or missing diseased areas. More importantly, determining each grid can quickly locate the specific location of the disease, facilitating timely symptomatic treatment by staff. Moreover, the status of each grid can be summarized and analyzed to generate a map or heat map of the plant health status, which helps to intuitively understand the overall health status of the plant. It can also be used to predict the development trend of the disease and provide a scientific basis for future disease and pest control work.

[0093] Example 1

[0094] In order to help the examiner better understand this application, the following example is given: install the nursery plant pest and disease detection system of this application in a nursery environment, fix the perception unit at a height of 1.4 meters from the ground, and adjust the angles of the RGB camera and the depth camera to cover the range of 3 plug trays. After the system is started, the perception unit begins to collect image data and depth information, and transmits the data to the computing unit wirelessly. The Jetson platform in the computing unit processes the data, detects the presence of leaf spot disease in certain plants, and sends the test results to the central control device in real time, prompting the management personnel to handle it.

[0095] The system structure of the present application is highly integrated, and the perception unit and the computing unit are connected through a standard interface, which is convenient for the installation, maintenance and expansion of the system. The system also supports the access of other sensor modules, such as multi-spectral cameras, thermal infrared cameras, etc., to further enhance the detection capability and scope of application. It is particularly worth noting that for different areas in large nurseries, the system of the present application can be deployed at multiple points. By adjusting the installation height and camera angle of the perception unit, the system can cover the detection range of multiple areas at the same time. The deep learning model in the Jetson platform in the computing unit will be adaptively adjusted according to the environmental characteristics of different areas to ensure that high-precision detection results can be obtained under various environmental conditions.

[0096] The examples described in the present invention are merely descriptions of the preferred implementation modes of the present invention, and are not intended to limit the concept and scope of the present invention. Without departing from the design concept of the present invention, various modifications and improvements made to the technical solutions of the present invention by engineers and technicians in this field should all fall within the protection scope of the present invention.

Claims

1. A nursery plant pest detection system, comprising a sensing unit for collecting data and a computing unit for processing the data, characterized in that: The perception unit includes a camera component, which is used to collect visible light images of nursery plants and three-dimensional depth information of nursery plants; the computing unit includes a computing component, which is used to first use an unsupervised learning method to extract image features and depth features of the visible light images of nursery plants and the three-dimensional depth information of plants, and then use a supervised fine-tuning method to detect the types of diseases and pests of nursery plants based on the image features and depth features.

2. A nursery plant pest detection system according to claim 1, characterized in that: The camera assembly includes an RGB camera and a depth camera. The RGB camera collects visible light images of nursery plants; the depth camera collects three-dimensional depth information of nursery plants; and both the RGB camera and the depth camera are placed 1m to 1.4m away from the plant's apical buds.

3. A nursery plant pest detection system according to claim 2, characterized in that: The camera assembly is arranged on an adjustable bracket, and the adjustable bracket can change the height of the camera assembly from the ground and change the shooting angle of the camera assembly.

4. A nursery plant pest detection system according to claim 1 or 2, characterized in that: The sensing unit also includes: Heat dissipation component: used to dissipate heat from camera components; Power supply assembly: used to supply power to the camera assembly and the heat dissipation assembly.

5. A nursery plant pest detection system according to claim 1, characterized in that: The computing component is a deep learning model, and the training process of the deep learning model includes an unsupervised learning phase and a supervised fine-tuning phase: The SimCLR model is used in the unsupervised learning stage. The visible light images and three-dimensional depth information of the nursery plants are input into the SimCLR model, and the SimCLR model extracts the image features and depth features of the nursery plants. The EfficientNet model is used in the supervised fine-tuning stage. The image features and depth features extracted by the SimCLR model are input into the EfficientNet model, and the EfficientNet model is used to identify the types of pests and diseases. The EfficientNet model is adjusted using the visible light images and three-dimensional depth information of the labeled pests and diseases.

6. A nursery plant pest detection system according to claim 5, characterized in that: The computing unit comprises: Jetson Xavier NX computing platform: used for performing pest and disease detection on nursery plants, and the Jetson Xavier NX computing platform includes the deep learning model; Display component: used to display the test results; Heat dissipation component: used to dissipate heat for the Jetson Xavier NX computing platform.

7. A nursery plant pest detection system according to claim 6, characterized in that: The computing unit also includes a plurality of reserved interfaces, and the plurality of reserved interfaces are used to connect with the sensing unit for data and power.

8. A detection method of a nursery plant pest detection system according to any one of claims 1 to 7, characterized in that: The steps include: The perception unit collects visible light images of the nursery plants and three-dimensional depth information of the nursery plants; The computing unit performs computational analysis on the collected visible light images and three-dimensional depth information of the nursery plants to obtain the disease and insect detection results; the computing process is as follows: first, image features and depth features are extracted from the visible light images and three-dimensional depth information of the nursery plants; then, the extracted image features and depth features are calculated to obtain the disease and insect detection results.

9. A method for detecting plant diseases and insect pests in a nursery according to claim 8, characterized in that: Before the calculation unit performs calculation, it also includes fusing the visible light image of the nursery plants and the three-dimensional depth information of the nursery plants to obtain an RGB-D image; then dividing the RGB-D image into a number of grids of equal size, and performing preprocessing on the several grids before calculating each grid.

Images