Deep learning-based tomato trialeurodes vaporariorum disease automatic identification method and system
By constructing a targeted augmented dataset and optimizing the network structure, combined with knowledge transfer methods and adaptive processing, the problem of insufficient accuracy in tomato whitefly identification was solved, achieving high-precision and robust automatic identification, suitable for mobile terminals and field monitoring equipment.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-25
- Publication Date
- 2026-03-31
AI Technical Summary
Existing technologies for the automatic identification of whiteflies on tomatoes suffer from problems such as easy loss of small target features, significant interference in complex environments, and insufficient identification accuracy, making it difficult to meet the needs of precise prevention and control.
A lightweight object detection network is constructed, and targeted structural optimization is performed. The model is trained by combining knowledge transfer methods, and adaptive post-processing is implemented, including data augmentation, adaptive spatial attention mechanism and dynamic adjustment of confidence, to improve the recognition accuracy and robustness of the model in complex environments.
It significantly improves the accuracy and stability of whitefly identification, ensuring the efficient application of the model in real field environments and providing technical support for precision plant protection of tomato crops.
Smart Images
Figure CN121768041A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of smart agriculture technology, and in particular to an automatic identification method and system for tomato whitefly disease based on deep learning. Background Technology
[0002] Tomato whiteflies are a major pest affecting tomato production. Due to their tiny size, field monitoring relies heavily on manual labor, resulting in low efficiency, poor timeliness, and high costs. While automatic identification using computer vision technology is a developing trend, existing general pest identification models suffer from drawbacks when dealing with whiteflies, such as easily losing small target features, significant interference from complex environments, and insufficient identification accuracy, making them unsuitable for the practical needs of precise control. Therefore, there is an urgent need for an automatic identification scheme specifically optimized for the characteristics of whiteflies, possessing both high accuracy and robustness. Summary of the Invention
[0003] The purpose of this invention is to provide a high-precision automatic identification method and system specifically for tomato whiteflies, in order to solve the problem of poor identification effect in existing technologies.
[0004] To achieve the above objectives, the present invention adopts the following technical solution: A deep learning-based method for automatic identification of tomato whitefly disease includes the following steps: S1. Collect and label images containing tomato whiteflies, and perform targeted data augmentation on the labeled dataset; S2. Construct a lightweight object detection network and perform targeted structural optimization on it; using the enhanced dataset, train the optimized network using the knowledge transfer method to obtain a whitefly-specific identification model; S3. Input the image to be detected into the dedicated recognition model to obtain preliminary results, and perform adaptive optimization processing based on image environment perception on the results to output the final recognition result; S4. Based on the final identification results, count the number of whiteflies and trigger an early warning based on the statistical results.
[0005] Preferably, in step S1, the targeted data augmentation processing includes: small target sample proliferation enhancement, which involves transforming and copying the labeled whitefly target individuals and superimposing them onto the background area of other training images; and complex background fusion enhancement, which involves synthesizing the leaf foreground area containing whiteflies with a randomly selected complex background image. Through these two enhancement methods, the diversity and complexity of the training data are effectively increased, and the model's sensitivity to small targets and robustness in complex environments are improved.
[0006] Preferably, in step S2, the targeted structural optimization includes: introducing a dense connection structure into the feature extraction part of the lightweight target detection network to enhance the reuse and transmission of shallow, small features; embedding an adaptive spatial attention mechanism on the feature layer used for detecting small targets in the feature fusion part of the network to enhance attention to potential target regions; and optimizing and adjusting the preset candidate box size of the network according to the actual size distribution of whitefly targets.
[0007] Preferably, the workflow of the adaptive spatial attention mechanism is as follows: global pooling is performed on the input feature map, the pooling result is processed by a convolutional layer to generate a spatial weight map, and then the weight map is multiplied with the original feature map, so that the network can autonomously focus on the region in the image that is more likely to contain the target.
[0008] Preferably, in step S2, the knowledge transfer method specifically involves: training a teacher model using a large-scale general dataset; extracting the deep feature representations generated by the teacher model for whitefly data as transfer knowledge; and, when training the specialized recognition model, aligning its internal feature representations with the transfer knowledge by designing a loss function. This method can transfer the rich and robust feature representation capabilities learned by the large model to a lightweight specialized model.
[0009] Preferably, in step S3, the adaptive optimization processing based on image environment perception includes: dynamic confidence adjustment, which dynamically adjusts the confidence threshold used to filter recognition results according to the brightness level of the input image; and redundant box merging, which merges multiple recognition boxes that are highly overlapping in spatial location and removes duplicate boxes. This process significantly improves the adaptability of the system under different lighting conditions and the cleanliness of the output results.
[0010] Preferably, the dynamic adjustment of confidence level specifically involves: setting a baseline confidence level threshold and calculating the average brightness of the image; adjusting the baseline threshold inversely based on the average brightness value, where the lower the brightness, the lower the threshold, to ensure a high target detection rate even in low-light environments.
[0011] Preferably, the redundant box merging specifically involves: calculating the distance between the center points of all preliminary identification boxes, determining boxes with a distance less than a preset threshold as duplicate identifications of the same target, and retaining only the one with the highest confidence, thereby eliminating multiple redundant responses of the model to a single target.
[0012] Preferably, in step S4, the threshold condition for triggering the early warning can be flexibly set according to different growth stages of tomatoes or actual prevention and control needs, so that the system can adapt to the precise management needs under different application scenarios.
[0013] This invention also proposes an automatic identification system for tomato whitefly disease to implement the above method, comprising: Image input unit, used to acquire images of tomato plants to be detected; The data processing unit is used for image preprocessing and data enhancement. The model calculation unit has a built-in whitefly-specific identification model for performing image recognition; The result optimization unit is used to perform adaptive optimization processing on the recognition results; The early warning output unit is used to count the number of pests, visualize the results, and issue early warning information.
[0014] The system can be integrated into a mobile smart terminal as a software application, or deployed in fixed field monitoring equipment or mobile patrol equipment, and includes a unit that communicates with a remote management platform, enabling flexible deployment and application.
[0015] The beneficial effects of this invention are: This invention forms a complete and highly specialized whitefly identification technology solution by constructing a targeted augmented dataset, performing specialized structural optimization on a lightweight network, training it using a knowledge transfer method, and implementing adaptive post-processing. This method significantly improves the identification accuracy and stability of tiny whitefly targets in real and complex field environments, while ensuring the efficiency of the model, making it easy to deploy and apply in actual agricultural production, and providing reliable technical support for precision plant protection and intelligent management of tomato crops.
[0016] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing embodiments of this application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings. Attached Figure Description
[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0018] Figure 1 This is a schematic diagram of the automatic identification method for tomato whitefly disease based on deep learning according to the present invention. Figure 2 This is a block diagram of the automatic identification system for tomato whitefly disease based on deep learning according to the present invention; Figure 3 This is a diagram showing the multi-category collaborative training effect of the whitefly identification model of this invention; Figure 4This is a visual comparison of the detection performance of the tomato whitefly identification model of this invention in complex field environments. Detailed Implementation
[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.
[0020] Example 1 like Figure 1 The flowchart shown is a deep learning-based automatic identification method for tomato whitefly disease. The implementation steps of the method of this invention are as follows: S1. Collect and label images containing tomato whiteflies, and perform targeted data augmentation on the labeled dataset, as follows: At multiple tomato growing bases, at different times and under different weather conditions, leaves infested with whiteflies were photographed using smartphones. Relevant images were also selected from publicly available datasets, resulting in approximately 8,500 raw images. Annotation tools were used to create fine bounding boxes for adult and nymph whiteflies, yielding 7,503 effectively annotated images containing approximately 120,000 targets. Perform targeted data augmentation on the labeled dataset: Small target sample proliferation enhancement: Randomly select a batch of images, crop out the whitefly annotation boxes, and after random scaling of 0.8-1.2 times and 90% transparency processing, randomly paste them into the leaf background area of other images in the same batch to generate about 3000 enhanced images; Complex background fusion enhancement: Approximately 2,000 leaf areas containing whiteflies were cropped out and alpha-blended with background images of soil, weeds, etc., taken from the field, to generate approximately 2,000 new images with higher background complexity.
[0021] The final dataset used for training consisted of approximately 12,500 images.
[0022] S2. Construct a lightweight object detection network and perform targeted structural optimization on it; using the augmented dataset, train the optimized network using a knowledge transfer method to obtain a whitefly-specific identification model, as follows: A lightweight object detection network based on the YOLOv5s architecture is constructed, and the following targeted structural optimizations are performed on it: In the shallow layers of the backbone network, a dense connection block consisting of three convolutional layers is inserted to enhance the reuse of small features in the shallow network. An adaptive spatial attention module is added to the feature map (P3) path in the feature pyramid network responsible for detecting small targets. This module performs global average pooling and global max pooling on the input feature map, concatenates the results and passes them through a convolutional layer to generate a spatial attention weight map, which is then multiplied with the original feature map. Clustering algorithms were used to analyze the width and height of all whitefly annotation boxes in the dataset, resulting in 9 sets of smaller preset anchor boxes that are more suitable for the whitefly's body size, which replaced the original preset anchor boxes in the network. The optimized network is trained using a knowledge transfer method, including: Pre-trained teacher model: A large model (such as YOLOv5x) is pre-trained using a large-scale general plant disease and pest dataset as the teacher model; Extracting transferable knowledge: Input the original whitefly data (approximately 7503 images) from S1 into the teacher model, and extract the feature maps output from the end of its backbone network as the deep feature representation to be transferred; Training the student model: Using the optimized network described above as the student model, the total loss function during training is set as: L_total = L_det + λ * L_distill L_det is the detection loss of the student model itself, L_distill is the mean squared error loss (feature transfer loss) between the corresponding layer feature map of the student model and the soft target feature map of the teacher model, and λ is the weight coefficient, initially 0.5, which gradually decays to 0.1 during training. A total of 300 training rounds were conducted to obtain a whitefly-specific recognition model.
[0023] S3. Input the image to be detected into the dedicated recognition model to obtain preliminary results, and perform adaptive optimization processing based on image environment perception on the results to output the final recognition result, as follows: Deploy the trained dedicated recognition model; during recognition: The tomato leaf image to be detected was scaled up to 640x640 and input into the model to obtain preliminary identification results including the whitefly bounding box and confidence score. Adaptive optimization processing is performed on the preliminary results: Dynamic confidence adjustment: Calculate the average gray value V of the input image (normalized to 0-1). Set the dynamic confidence threshold T = 0.4 * [1 + 0.2 * (0.5 - V)]. When the image is dark (V<0.5), the T value decreases to improve the detection rate; when the image is bright, the T value increases accordingly. Use this dynamic threshold to filter the initial results. Redundant box merging: For the filtered detection boxes, calculate the Euclidean distance between the center points of any two boxes. If the distance is less than 10 pixels, it is determined to be a duplicate recognition of the same insect. Only the box with the highest confidence is retained, and the rest are deleted. This is the final recognition result.
[0024] S4. Based on the final identification result, count the number of whiteflies and trigger an alert based on the statistical result, as follows: The system draws the final identified bounding box on the original image in the application interface (such as a mobile APP), displays the category and confidence level, and counts the total number of whiteflies in the current image. The system allows users to set different warning thresholds based on different stages such as tomato seedling, flowering, and fruiting stages. When the average number of insects counted in 10 consecutive frames exceeds the set threshold, the system triggers an alert and reminds the user through pop-up prompts on the interface, phone vibration, etc.
[0025] Example 2: like Figure 2 The diagram shown is an automatic identification system for tomato whitefly disease based on deep learning. This system can be deployed on various hardware platforms.
[0026] One implementation is an application integrated into a smartphone: Image input unit: Use the phone's camera to take a real-time picture or select an image of a tomato leaf from the album.
[0027] Data processing unit: performs data augmentation as described in S1 in training mode; performs normalized scaling on the input image in recognition mode.
[0028] Model calculation unit: integrates and solidifies the weights of a dedicated recognition model obtained through S2 training.
[0029] Result optimization unit: Executes the confidence dynamic adjustment and redundant box merging algorithm described in S3.
[0030] Early warning output unit: Displays images with marked boxes and the number of insects on the mobile phone screen in real time, and triggers audiovisual warnings based on set thresholds.
[0031] Another implementation method is to deploy intelligent monitoring equipment in the field: The hardware includes a high-definition camera, an edge computing box (such as Jetson Nano), a solar power panel, and a 4G communication module. The edge computing box embeds the software for each unit of the above system. The device automatically rotates and photographs the surrounding tomato plants at regular intervals (e.g., hourly). The edge computing box performs image recognition, result optimization, and quantity statistics, and uploads the recognition results (including labeled thumbnails, quantity, location, and time) to a cloud-based agricultural IoT platform via a 4G network. The platform aggregates data from various monitoring points and performs comprehensive analysis. When the number of insects at a certain monitoring point exceeds a preset threshold multiple times consecutively, the platform automatically generates an early warning work order, which is pushed to the manager responsible for the area via SMS or APP to guide them in accurate inspection and prevention.
[0032] Performance testing The performance of the dedicated recognition model trained by the method of this invention and two contrasting models was evaluated on an independent test set containing 1,500 real field images and approximately 20,000 whitefly targets: Comparison Model 1: The standard YOLOv5s model, directly trained using the original dataset of this project (without targeted enhancements or structural optimizations).
[0033] Comparison Model 2: A publicly available, universal tomato pest and disease identification model (capable of identifying multiple targets, including whiteflies).
[0034] The tests were conducted on the same server configured with an NVIDIA GTX 1080Ti to control hardware variables. Commonly used metrics in object detection were employed: mean precision (mAP, with an IoU threshold of 0.5), recall, and precision. The model's inference speed and model file size on the Jetson Xavier NX edge device were also evaluated. The test results are shown in the table below.
[0035] Results analysis: Identification accuracy: The dedicated identification model significantly outperforms the two comparative models in all three core metrics: mAP, recall, and precision. In particular, the recall rate reaches 92.7%, indicating that the method of this invention can effectively detect the vast majority of whiteflies and greatly reduce the false negative rate, which is crucial for early warning. The mAP@0.5 reaches 95.3%, demonstrating its excellent overall performance.
[0036] Model efficiency: While maintaining a similar model size (14.6MB vs 14.4MB) and edge inference speed as the comparison model 1, the dedicated recognition model achieved a leap in accuracy (mAP improved by 23.9 percentage points). Compared with the comparison model 2, the dedicated recognition model completely outperformed the latter in accuracy, while its model size was only one-third of the latter and its inference speed was about three times faster, fully demonstrating the balance between lightweight and high performance.
[0037] Actual results: 100 challenging samples (such as those with strong light reflection, severe shadows, extremely cluttered backgrounds, and extremely dense insects) were randomly selected from the test set images and manually reviewed by experts; the false positives and false negatives of the dedicated recognition model were significantly less than those of the two comparison models, proving its strong robustness in complex real-world scenarios.
[0038] Visualization of model detection results: To intuitively demonstrate the training process and actual recognition effect of this invention, this invention combines the visualization analysis of the model training stage with the recognition results of the final test set, and demonstrates the effect from two dimensions: dynamic training and static testing.
[0039] I. Visual Analysis of the Overall Training Process During the model training phase, we continuously monitored the learning progress of each category, demonstrating the overall training performance of the YOLOv5 model on all six types of pests and diseases (early blight, late blight, bacterial leaf spot, anthracnose, stem rot, and whitefly) during iterative training; please refer to [link / reference]. Figure 3 , Figure 3 This is a graph showing the multi-class collaborative training effect of the whitefly identification model of this invention; the graph reflects the balance of sample distribution of each class and the progress of feature learning during the training process. As can be seen from the graph: The early and late disease categories showed stable performance during training, and the model's ability to identify lesions of various morphologies gradually improved. Due to the small size of the whitefly and the complex background, the initial identification of whiteflies fluctuated greatly, but with training iterations and the application of data augmentation strategies, the detection accuracy gradually improved. For data-scarce categories (such as anthracnose and stem rot), the graph shows training fluctuations caused by insufficient samples.
[0040] II. Final Model Detection Results After the model was trained, we conducted comprehensive testing on an independent test set containing 1500 real field images, and selected typical scenes for visualization. Please refer to [link / reference needed]. Figure 4 , Figure 4This is a visual comparison of the detection performance of the tomato whitefly identification model of the present invention in complex field environments. The figure shows the recognition performance of the whitefly identification model after training on real field test images. It includes examples of multiple scenarios (different lighting, background complexity, and insect density) and comparative diagrams of difficult sample recognition, comprehensively demonstrating the model's high accuracy, high robustness, and practical value in real-world applications.
[0041] In summary, this invention, through targeted and innovative design across the entire process from data, network structure, training methods to post-processing, successfully constructs a high-precision, high-efficiency, and highly robust solution specifically for identifying tomato whiteflies. This solution can be deployed in a lightweight form on mobile terminals for individual farmers, or it can be systematically built into an automated monitoring network to serve large farms, demonstrating good practical value and broad prospects for promotion.
[0042] Finally, it should be noted that the above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for automatic identification of tomato whitefly disease based on deep learning, characterized in that, The method comprises the following steps: Collect and label images containing tomato whitefly, and perform targeted data enhancement processing on the labeled data set; A lightweight target detection network is constructed, and the network is optimized in structure. The optimized network is trained using the enhanced data set and a knowledge transfer method to obtain a whitefly-specific identification model; The preliminary result is obtained by inputting the image to be detected into the specific identification model, and the result is adaptively optimized based on image environment perception to output the final identification result; The number of whiteflies is counted based on the final identification result, and a warning is triggered based on the counting result.
2. The deep learning-based tomato whitefly disease automatic identification method according to claim 1, characterized in that, The targeted data enhancement processing comprises: Copy and superimpose the labeled whitefly target individuals after transformation to the background area of other training images; Synthesize the leaf foreground area containing whiteflies with a randomly selected complex background image. 3.The deep learning-based tomato whitefly disease automatic identification method according to claim 1, characterized in that, The targeted structure optimization in step S2 comprises: Introduce a dense connection structure in the feature extraction part of the lightweight target detection network to strengthen the reuse and transmission of shallow micro features; Embed an adaptive spatial attention mechanism in the feature fusion part of the network on the feature layer for detecting micro targets to enhance the attention to potential target areas; Optimize and adjust the preset candidate box size of the network according to the actual size distribution of the whitefly target.
4. The method according to claim 3, wherein the method further comprises: The working process of the adaptive spatial attention mechanism is: performing global pooling operation on the input feature map, generating a spatial weight map after convolution layer processing of the pooling result, and multiplying the weight map with the original feature map. 5.The deep learning-based tomato whitefly disease automatic identification method according to claim 1, characterized in that, The knowledge transfer method comprises: Training a teacher model using a large-scale general data set; Extracting deep feature representation generated by the teacher model for whitefly data as transfer knowledge; Aligning the internal feature representation of the specific identification model to the transfer knowledge by designing a loss function during training of the specific identification model. 6.The deep learning-based tomato whitefly disease automatic identification method according to claim 1, wherein, The adaptive optimization processing based on image environment perception in step S3 comprises: Dynamically adjusting the confidence threshold for screening the identification result according to the brightness level of the input image; Merging multiple recognition boxes with highly overlapping spatial positions to remove duplicate boxes.
7. The deep learning-based tomato whitefly disease automatic identification method according to claim 6, characterized in that, The confidence dynamic adjustment specifically comprises: setting a reference confidence threshold and calculating the average brightness of the image; inversely linearly adjusting the reference threshold according to the average brightness value, and the lower the brightness, the lower the threshold.
8. The deep learning-based tomato whitefly disease automatic identification method according to claim 6, characterized in that, The redundant box merging specifically comprises: calculating the center point distance of all preliminary identification boxes, and determining that the boxes with a distance less than a preset threshold are duplicate identifications of the same target, and only keeping the one with the highest confidence. 9.The deep learning-based tomato whitefly disease automatic identification method according to claim 1, wherein, The warning threshold is set based on the number of whiteflies per unit leaf area or the growth trend of the number of whiteflies detected per unit time.
10. A tomato whitefly disease automatic identification system for implementing the method of any one of claims 1-9, characterized in that, It comprises: An image input unit for obtaining tomato plant images to be detected; A data processing unit for pre-processing and data enhancement of the images; A model calculation unit in which the whitefly-specific identification model is built, for performing image recognition; A result optimization unit for adaptively optimizing the identification result; A warning output unit for counting the number of pests, visualizing the result, and issuing a warning message.