Tea tree black mold detection and diagnosis method

Abstract

The invention discloses a diagnosis method for tea tree black mold detection, and the method comprises the following steps: S1, obtaining image data of tea tree leaves, and constructing a data set containing high-resolution images of healthy and infected tea tree leaves; s2, analyzing the image data by using a deep learning model to identify the existence and severity of black mold; s3, enhancing the understanding of the model on images and texts by combining a visual question and answer technology and utilizing question and answer pairs corresponding to the image data, so as to improve the recognition accuracy; and S4, realizing real-time detection and diagnosis of the tea tree black mold by using the trained model through a mobile network and an unmanned aerial vehicle platform. Compared with a manual observation method, the method has the advantages that the accuracy of tea tree black mold detection is improved by more than 20%, the accuracy of tea tree black mold detection is improved by more than 5% compared with an existing image recognition method, light weight of the model can be realized, the average reasoning time at a mobile terminal and an unmanned aerial vehicle terminal is 0.2 second per image, and on-site real-time monitoring is facilitated.

Description

Technical Field

[0001] The present invention relates to the technical fields of smart agriculture and black mold disease detection and diagnosis, and particularly relates to a method for detecting and diagnosing black mold disease of tea trees. Background Art

[0002] Black mold disease of tea trees is a common disease of tea trees caused by various fungi. It mainly infects tea tree leaves, tender shoots and fruits. In severe cases, it can cause a large number of leaves to fall off, new shoots to wither, and even affect the growth and yield of tea trees. The typical symptom of this disease is the formation of a black mold layer on the surface of infected leaves, usually round or irregular in shape, and may be accompanied by phenomena such as leaf yellowing and wilting. Black mold disease is extremely likely to occur and spread in high-temperature and high-humidity environments, causing huge economic losses to the tea industry. Therefore, timely, accurate detection and prevention and control of black mold disease of tea trees are crucial.

[0003] Traditional methods for detecting black mold disease of tea trees mainly rely on manual observation and laboratory analysis. Manual observation refers to experienced tea farmers or technicians patrolling in the tea garden and judging whether there are symptoms of black mold disease on the leaves by the naked eye. Although this method is intuitive, it has obvious limitations: First, the efficiency of manual patrol is low, it is difficult to cover large areas of tea gardens, and it is impossible to achieve comprehensive monitoring of diseases; Second, the results of manual observation are easily affected by subjective factors, and the judgment criteria of different personnel may vary, resulting in low accuracy and consistency of detection results; Third, this method is difficult to effectively identify the early stage of the disease, and it is easy to delay the prevention and control time, resulting in the spread of the disease. Laboratory analysis requires bringing the collected leaf samples back to the laboratory and diagnosing them through methods such as microscopic observation and culture isolation. Although this method has high accuracy, it takes a long time, has high costs, and cannot achieve real-time dynamic monitoring of the disease, and it is difficult to meet the requirements of modern agriculture for precise monitoring and rapid response.

[0004] With the development of technology, some modern image recognition techniques have begun to be applied to the detection of tea tree diseases. These methods usually use camera devices to collect images of tea tree leaves, and then perform disease recognition through image processing and machine learning algorithms. Compared with traditional methods, these methods have improved the detection efficiency to a certain extent, reduced manual labor, and can perform a certain degree of automated monitoring. However, the existing disease detection methods based on image recognition still have some deficiencies: First, in a complex tea garden environment, such as under different lighting conditions, different shooting angles, and the presence of leaf occlusion, the recognition accuracy will significantly decrease, making it difficult to meet the requirements of practical applications; Second, these methods usually only focus on the visual information of images and lack the ability to understand text information, such as understanding information about the type and severity of diseases, making it difficult to perform accurate and comprehensive disease diagnosis; Third, these methods usually need to run on powerful computing devices and are difficult to be deployed in real time on devices such as mobile phones or drones, which limits their application in actual production.

[0005] To overcome the limitations of the above traditional and modern methods, there is an urgent need to develop new, more efficient, accurate, and practical detection methods for tea tree black mold disease. This new detection method should have the following characteristics: It can achieve non-invasive detection; it can adapt to complex tea garden environments; it can quickly and accurately identify different types of black mold disease symptoms; it can provide quantitative disease information; it can be easily deployed on devices such as mobile phones or drones to achieve large-area and real-time disease monitoring. Based on this, the present invention proposes a detection and diagnosis method for tea tree black mold disease based on the combination of deep learning and visual question answering technology, aiming to provide tea farmers with a more effective and reliable disease monitoring means and ensure the healthy growth of tea trees and the improvement of tea quality. Summary of the Invention

[0006] The present invention aims to solve the above problems and provides a diagnostic method for detecting tea tree black mold disease. This method includes the following steps: The flowchart is as Figure 1 shown:

[0007] S1: Obtain image data of tea tree leaves and construct a dataset containing high-resolution images of healthy and black mold-infected tea tree leaves;

[0008] S2: Use a deep learning model to analyze the image data to identify the presence and severity of black mold disease;

[0009] S3: Combine visual question answering technology and use question-and-answer pairs corresponding to the image data to enhance the model's understanding of images and text, thereby improving the recognition accuracy;

[0010] S4: Using the trained model, through the mobile network and the drone platform, real-time detection and diagnosis of tea tree black mold disease are achieved.

[0011] Further, the image dataset in S1 can be an original dataset and / or a custom dataset.

[0012] Further, the deep learning model in S2 is a ResNet model. Preferably, the ResNet50 model is initialized with the weights pre-trained on the ImageNet dataset.

[0013] Based on the above technical solution, the training strategy of the ResNet model is federated learning. The specific steps of federated learning are as follows:

[0014] A: Local training: The model is trained on the local data of each tea garden, and the models of each tea garden are independently trained for a number of epochs.

[0015] B: Parameter aggregation: After training, the model parameters of each tea garden are sent to the central server for aggregation to generate new global model parameters.

[0016] C: Model update: The central server sends the updated global model parameters back to each tea garden, and each tea garden retrains the local model with the new model parameters.

[0017] D: Iteration: Repeat the above steps until the model converges.

[0018] Further, the method further includes applying the BLIP method to enable the model to understand complex inputs of images and texts.

[0019] Further, the pre-training steps of the BLIP model of the BLIP method are as follows:

[0020] A: Data input: Input the image and the corresponding text description into the BLIP model.

[0021] B: Feature extraction: The BLIP model uses its own encoder to extract features from the image and the text, respectively obtaining an image feature vector and a text feature vector.

[0022] C: Cross-modal fusion: The image and text feature vectors are fused, and the association between the two is established through the self-attention mechanism.

[0023] D: Loss calculation: Calculate the loss between the answer output by the model and the true answer.

[0024] E: Parameter optimization: Using the gradient descent algorithm, optimize the model parameters so that the model can correctly answer questions about the image content.

[0025] Furthermore, the method further includes using a dual attention mechanism to enable the model to simultaneously focus on relevant image regions and problem parts.

[0026] More preferably, the dual attention mechanism includes spatial attention and channel attention.

[0027] Specifically, the implementation steps of the spatial attention are as follows: Feature map input: Use the feature map of the intermediate layer of the ResNet50 network as the input of the spatial attention module; Calculate attention weights: Use convolutional operations and the Sigmoid function to calculate the attention weights at each position in the feature map. The range of these weight values is between 0 and 1, and the larger the value, the more important the position; Weighted output: Multiply the original feature map by the calculated attention weights to obtain the weighted feature map, thereby achieving the weighting of the feature map.

[0028] Specifically, the implementation steps of the channel attention are as follows: Feature map input: Use the feature map of the intermediate layer of the ResNet50 network as the input of the channel attention module; Global average pooling: Perform global average pooling on the feature map to compress each channel of the feature map into a single value; Channel attention weight calculation: Use two fully connected layers and the Sigmoid function to calculate the attention weights for each channel; Weighted output: Multiply the original feature map by the calculated channel attention weights to obtain the weighted feature map, thereby achieving the weighting of the feature channels.

[0029] The specific implementation method of the dual attention mechanism is as follows:

[0030] Module embedding: We embed the spatial attention module and the channel attention module into multiple intermediate layers of the ResNet50 network respectively. The embedding method is in parallel, that is, first calculate the spatial attention on the feature map, and then calculate the channel attention on the feature map after the spatial attention calculation. The two attention mechanisms work together to improve the feature extraction ability of the model;

[0031] Attention mechanism fusion: After extracting the spatial attention and the channel attention, fuse the attention mechanisms by multiplying the feature maps extracted by the two attention mechanisms;

[0032] Model training: During the training process, the attention module is trained and optimized together with the ResNet50 network, enabling the model to learn how to adaptively select important regions and channels.

[0033] Furthermore, the creation of the custom dataset includes image enhancement operations and the construction of question-answer pairs.

[0034] Specifically, the image enhancement operations include:

[0035] Random cropping and scaling: Randomly select local regions of the image for cropping and scale the image to its original size, enabling the model to learn lesion features at different scales.

[0036] Adding Gaussian noise: Add Gaussian noise to the image to simulate the image noise that may exist in the real environment and improve the model's noise resistance.

[0037] Adjusting image brightness and contrast: Randomly adjust the brightness and contrast of the image to simulate image differences under different lighting conditions and improve the model's adaptability to lighting changes.

[0038] Using blur and sharpen filters: Apply blur and sharpen filters to the image to simulate changes in image sharpness under different shooting conditions and improve the model's robustness.

[0039] Performing affine transformation: Perform random affine transformations such as rotation, translation, scaling, and shearing on the image to simulate changes in shooting angles and positions and improve the model's adaptability to angle and position changes.

[0040] Specifically, the strategy for constructing question-answer pairs follows the following principles:

[0041] Diversity: The question-answer pairs cover different types of questions, such as whether there is a disease, the severity of the disease, the distribution location of the disease, etc.

[0042] Accuracy: Ensure that the answer to each question is consistent with the image content.

[0043] Clarity: Ensure that both the questions and answers are concise and easy to understand.

[0044] Examples of question-answer pairs:

[0045] Question: "Does this leaf have black mold disease?" Answer: "Yes."

[0046] Question: "What stage is the black mold disease in?" Answer: "Mid-stage."

[0047] Question: "Which area is the black mold disease mainly distributed in?" Answer: "Bottom leaves."

[0048] Question: "What is the state of this leaf?" Answer: "Healthy."

[0049] Question: "What is the severity of the black mold disease on this leaf?" Answer: "Mild."

[0050] More question-answer pairs: According to the actual situation, more question-answer pairs of different types and angles can be constructed to ensure that the model can better understand image and text information.

[0051] The present invention also provides the application of a diagnostic method for detecting tea tree black mold disease as described above in the detection of tea tree black mold disease in tea plantations.

[0052] The present invention has the following beneficial effects:

[0053] The present invention relates to a diagnostic method for detecting tea tree black mold disease based on deep learning and Visual Question Answering (VQA) technology. This method combines a deep learning model and visual question answering technology to achieve high-precision and rapid detection of black mold disease on tea tree leaves. The system includes a dataset containing high-resolution images of healthy and black mold-infected tea tree leaves, as well as corresponding question-and-answer pairs. By fine-tuning a deep learning model such as ResNet and combining the BLIP (Bootstrapping Language-Image Pre-training) method, the model can understand complex visual and text inputs, improving the accuracy and relevance of detection. In addition, the method also includes the creation of a custom dataset to enhance the generalization ability of the model. This diagnostic method is lightweight and can be integrated into mobile networks and drones, providing an effective technical means for real-time monitoring of tea tree black mold disease.

[0054] Experiments have shown that the method of the present invention has improved the accuracy of tea tree black mold disease detection by more than 20% compared with the manual observation method and by more than 5% compared with existing image recognition methods. Moreover, it can achieve the lightweight of the model, with an average inference time of 0.2 seconds per image on the mobile and drone sides, facilitating on-site real-time monitoring. In addition, the creation of the custom dataset has increased the average detection accuracy of the model by more than 3% under different tea tree varieties and environmental conditions (such as sunny days, cloudy days, and different shooting angles). BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only one embodiment of the present invention. For those of ordinary skill in the art, without creative efforts, other implementation drawings can be obtained according to the provided drawings.

[0056] Figure 1 : Flowchart of the present invention;

[0057] Figure 2 : Example diagram of the dataset of the present invention, including example diagrams of tea tree leaves with healthy and black mold diseases of different infection degrees, where (Ⅰ) original image; (Ⅱ) principle framework diagram; (Ⅲ) schematic diagram after image processing;

[0058] Figure 3 : Flowchart of the model training of the present invention;

[0059] Figure 4 : Flow chart of model deployment of the present invention; Figure 5 : Example image of original data in Example 2 of the present invention. Detailed implementation manners

[0060] The present invention will be further described below with reference to the accompanying drawings and examples:

[0061] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts belong to the protection scope of the present invention. In the present invention, unless otherwise specified, the devices and raw materials used can be purchased from the market or are commonly used in the art. The methods in the following embodiments are conventional methods in the art unless otherwise specified.

[0062] Example 1: Construction of the original dataset (Tea Coal Disease, TCDNet)

[0063] 1.1 Dataset description

[0064] An original dataset containing 6377 high-resolution tea tree leaf images obtained by an RGB camera is constructed, among which 1200 are healthy leaf images and 5177 are leaf images infected with black mold disease. The image resolution is 2560x1920 pixels. All images are labeled by professional plant pathologists, and the labeling information includes:

[0065] A: Whether black mold disease exists (binary classification: yes / no); B: Types of black mold disease (multi-classification: early stage, middle stage, late stage); C: Severity of black mold disease (quantitative: mild, moderate, severe); D: Bounding box annotation of key parts of the leaf.

[0066] 1.2 Image sources

[0067] The dataset images mainly come from three different tea gardens, namely Chunxi Tea Garden in Feixian, Linyi, Shangshantang Tea Garden in Weihai, and Kaiyue Tea Farm in Jufeng, Rizhao, to ensure the diversity of the dataset and improve the generalization ability of the model. Different lighting conditions and angles are used for shooting to simulate the actual environment.

[0068] 1.3 Data preprocessing

[0069] The following preprocessing is performed on the original images: Resize the images to 512x512 pixels; Normalize the pixel values to between 0 and 1; Perform data augmentation operations such as random rotation, flipping, scaling, and color enhancement.

[0070] 1.4 Data annotation tool

[0071] Use the LabelImg annotation tool to annotate images and save the annotation information as an XML file.

[0072] An example diagram of the dataset is as Figure 2 shown: It includes example diagrams of tea tree leaves with healthy and different degrees of infected black mold. Among them, (Ⅰ) the original image; (Ⅱ) the principle framework diagram; (Ⅲ) the schematic diagram after image processing.

[0073] Example 2: Model training

[0074] The model training process of the present invention aims to construct a tea tree black mold detection model with high precision and high generalization ability. This process mainly includes the following steps: The model training flowchart is as Figure 3 shown.

[0075] 2.1 Selection and initialization of the basic model:

[0076] (1) Basic model: In order to make full use of the feature extraction ability of the pre-trained model and combine the deep structure and excellent performance of the ResNet network, we select ResNet50 as the basic deep learning model. ResNet50 is a classic convolutional neural network with 50 layers of depth and can show good performance in image recognition tasks.

[0077] (2) Initialization of pre-trained weights: We initialize the ResNet50 model with the weights pre-trained on the ImageNet dataset. ImageNet is a large dataset containing millions of images. The pre-trained weights can enable the model to have good feature extraction ability in the initial stage, accelerate model convergence, and improve the final performance of the model. In this way, we avoid training a deep neural network from scratch, greatly saving computing resources and time.

[0078] 2.2 Training strategy: Federated learning

[0079] (1) Necessity of federated learning: In order to improve the generalization ability of the model in different environments, we adopt the training strategy of federated learning. Due to the differences in tea tree planting environments, there may be significant differences in lighting conditions, soil conditions, and planting varieties in different tea gardens. A model trained directly with the data of one tea garden often has difficulty achieving good results in other tea gardens. Federated learning allows the model to be trained on the local data of multiple participating parties (here referring to three different tea gardens sampled in this application) without sharing the original data, thus protecting data privacy.

[0080] (2) Training process: Under the federated learning framework, the model training process is as follows:

[0081] Local training: The model is trained on the local data of each tea garden, and the models of each tea garden are independently trained for a number of epochs.

[0082] Parameter aggregation: After training, the model parameters of each tea garden are sent to the central server for aggregation to generate new global model parameters.

[0083] Model update: The central server sends the updated global model parameters back to each tea garden, and each tea garden retrains the local model with the new model parameters.

[0084] Iteration: Repeat the above steps until the model converges.

[0085] (3) Advantages of federated learning: Federated learning can effectively utilize the data of multiple tea gardens, improve the generalization ability of the model in different environments, and achieve the improvement of model performance while protecting data privacy.

[0086] 2.3 Selection of optimizer and adjustment of learning rate:

[0087] (1) Optimizer: To quickly optimize the model parameters, we use the Adam optimizer. Adam is an adaptive optimization algorithm that can dynamically adjust the learning rate according to the historical gradients of different parameters, thus achieving a faster convergence speed and better model performance.

[0088] (2) Initial learning rate: We set the initial learning rate to 0.001.

[0089] (3) Cosine annealing strategy: To avoid the model falling into a local optimal solution, we adopt the cosine annealing strategy to dynamically adjust the learning rate. The cosine annealing strategy can make the learning rate gradually decrease first and then gradually increase as the training progresses, thus helping the model explore a better parameter space.

[0090] 2.4 Selection of loss function

[0091] (1) Classification tasks: For classification tasks such as the presence or absence of black mold disease and disease types, we choose the cross-entropy loss function as the loss function. The cross-entropy loss function can effectively measure the difference between the probability distribution predicted by the model and the true labels, enabling the model to accurately classify.

[0092] (2) Bounding Box Regression: For the bounding box regression task of black mold lesions, we choose SmoothL1Loss as the loss function. Smooth L1 Loss can effectively solve the gradient explosion problem in the regression task, enabling the model to accurately predict the position of the bounding box.

[0093] 2.5 Training Parameters

[0094] (1) Batch Size: We set the Batch Size to 32, that is, 32 images are used for each training.

[0095] (2) Epochs: We set the Epochs to 100, that is, all training data is iterated 100 times.

[0096] (3) Learning Rate Decay: We adopt the learning rate decay strategy, and the learning rate is halved every 20 Epochs.

[0097] 2.6 Specific Implementation of Federated Learning

[0098] (1) Number of Local Training Rounds: On the local data of each tea garden, we perform model training for 0 Epochs.

[0099] (2) Parameter Aggregation: After the local model of each tea garden is trained for 10 Epochs, the model parameters are sent to the central server, and the federated averaging algorithm is used for parameter aggregation. The federated averaging algorithm averages the parameters of all local models to obtain new global model parameters.

[0100] (3) Model Update: The updated global model parameters are sent back to each tea garden, and the models in each tea garden are retrained using the global model parameters.

[0101] (4) Data Description:

[0102] Source of Tea Garden Data: The data comes from three different tea gardens, namely Chunxi Tea Garden in Feixian County, Linyi, Shangshantang Tea Garden in Weihai, and Kaiyue Tea Farm in Jufeng, Rizhao, representing different climates and planting conditions.

[0103] Data Volume: The data volume of each tea garden accounts for about one-third of the total dataset, that is, each tea garden has about 2125 images (6377 / 3 = 2125).

[0104] Data Annotation: All data has been annotated by professional plant pathologists. The annotation information includes the presence or absence of black mold, the type of black mold (early, mid-stage, late stage), and the bounding box annotation of the key parts of the leaves.

[0105] 2.7 Experimental Data Analysis

[0106] (1) Model training accuracy: After 100 Epochs of training, the accuracy of the model on the validation set reached 95.2%. This indicates that the model has good classification ability and can accurately identify whether tea tree leaves are infected with black mold disease.

[0107] (2) Example of original data: Image files such as Figure 5 shown

[0108] Image file name Whether diseased Disease type Disease degree image_001.jpg No None None image_002.jpg Yes Early stage Mild image_003.jpg Yes Middle stage Moderate image_004.jpg Yes Late stage Severe

[0109] (3) Data analysis:

[0110] We used a confusion matrix to analyze the prediction results of the model on the validation set and found that the model performed well in the identification of early and mid-stage black mold disease, but there was a slight deviation in the identification of late-stage black mold disease. This may be due to the complex symptoms of late-stage black mold disease, with various manifestations such as leaf wilting and drying. Therefore, it is necessary to increase the training data for late-stage symptoms in subsequent research to further improve the identification accuracy.

[0111] We also statistically analyzed the performance of the model on three local tea garden datasets and found that the performance of the model in the three tea gardens was relatively close, verifying the improvement effect of the federated learning strategy on the generalization ability of the model.

[0112] Example 3: Application of the BLIP method

[0113] This example details how to apply the BLIP (Bootstrapping Language-Image Pre-training) method to improve the model's understanding ability of image and text information, thereby enhancing the accuracy of tea tree black mold disease detection.

[0114] 3.1 BLIP model pre-training:

[0115] (1) Purpose: The core purpose of BLIP model pre-training is to enable the model to simultaneously understand complex visual information and text information and establish the association between the two. The pre-trained BLIP model can better handle visual question answering tasks, that is, generate correct answers based on the given image and question.

[0116] (2) Pre-training process:

[0117] Data input: Input the image and the corresponding text description into the BLIP model.

[0118] Feature extraction: The BLIP model uses its own encoder to extract features from the image and text, obtaining the image feature vector and the text feature vector respectively.

[0119] Cross-modal fusion: Fuse the image and text feature vectors and establish the correlation between the two through the self-attention mechanism.

[0120] Loss calculation: Calculate the loss between the answer output by the model and the true answer.

[0121] Parameter optimization: Use the gradient descent algorithm to optimize the model parameters so that the model can correctly answer questions about the image content.

[0122] (3) Data source: We use the Visual Genome dataset as the pre-training data for the BLIP model. The Visual Genome dataset is a large-scale scene understanding dataset containing approximately 108,000 images, each of which is accompanied by detailed region descriptions, attribute information, relationship information, and question-answer pairs. By using the Visual Genome dataset for pre-training, the BLIP model can learn general visual and text feature representations, laying a solid foundation for fine-tuning on the tea tree black mold dataset.

[0123] (4) Explanation of pre-training data:

[0124] ① Dataset name: Visual Genome

[0125] ② Dataset scale: It contains approximately 108,000 images, and each image is accompanied by detailed region descriptions, attribute information, relationship information, and question-answer pairs.

[0126] ③ Data format: Each image data is accompanied by a JSON file containing information such as object labels, attributes, relationships in the image, and also contains a large number of question-answer pairs based on the image content.

[0127] 3.2 Parameter settings:

[0128] (1) BLIP model parameters: We adopt the pre-training parameters of the BLIP model on the Visual Genome dataset and keep the parameters unchanged. The advantage of doing this is that it can avoid training the BLIP model from scratch, reduce the training difficulty and computational cost, and the pre-trained parameters have good generality.

[0129] (2) Joint training: In the fine-tuning stage, we jointly train the pre-trained BLIP model and the ResNet50 model. Specifically, we use the ResNet50 model as the visual encoder to extract the features of the image, then input the extracted features into the text encoder of the BLIP model, and calculate the loss to jointly optimize the parameters of the two models.

[0130] (3) Fine-tuning:

[0131] Fine-tuning purpose: On the tea tree black mold disease dataset, jointly fine-tune the pre-trained BLIP model and ResNet50 model so that the model can better understand image and text information related to tea tree black mold disease. For example, the model can answer questions such as "Is this leaf infected with black mold disease?" and "What stage is this black mold disease in?".

[0132] Fine-tuning method: Input the images and their corresponding Q&A pairs in the tea tree black mold disease dataset into the joint model. Use the cross-entropy loss function as the classification loss and use the Adam optimizer for fine-tuning.

[0133] 3.3 Experimental data analysis

[0134] (1) Accuracy improvement: To verify the improvement effect of the BLIP model on the classification accuracy of tea tree black mold disease, we conducted an ablation experiment. An ablation experiment means that while controlling other variables unchanged, gradually remove some modules in the model, and analyze the role of each module by comparing the changes in model performance. Specifically, we compared the classification accuracies of using the ResNet50 model (without using the BLIP model) and using the ResNet50 model combined with the BLIP model (pre-trained first and then fine-tuned). The results show that after pre-training with the BLIP model, the classification accuracy of the model on the validation set has increased significantly.

[0135] (2) List of accuracy improvements:

[0136] Model configuration Classification accuracy rate (%) ResNet50 (without using BLIP) 91.7 ResNet50 + BLIP (pre-training + fine-tuning) 95.2 Accuracy improvement 3.5

[0137] (3) Data analysis: As can be seen from the above table, after pre-training with the BLIP model, the classification accuracy of the model has increased by 3.5%, which indicates that the BLIP model can effectively utilize text information and improve the classification performance of the model. It shows that through the BLIP method, the model can better understand the text information hidden in the image, better fuse the image information and text information, thereby improving the accuracy of the model.

[0138] Example 4: Dual attention mechanism

[0139] This example details how to embed a dual attention mechanism in the ResNet50 network to enhance the model's ability to extract image features, thereby improving the accuracy of tea tree black mold disease detection. The dual attention mechanism includes spatial attention and channel attention, which work together to enable the model to pay more attention to important regions and feature channels in the image.

[0140] 4.1 Attention mechanism structure:

[0141] (1) Spatial Attention:

[0142] Function: The spatial attention mechanism aims to enable the model to pay more attention to relevant regions in the image, such as the areas of black mold lesions. By assigning different weights to different spatial positions, the model can focus more on the regions valuable for disease identification and suppress the influence of irrelevant regions.

[0143] Implementation steps:

[0144] Feature map input: Use the feature map from the middle layer of the ResNet50 network as the input to the spatial attention module.

[0145] Calculate attention weights: Use convolutional operations and the Sigmoid function to calculate the attention weights for each position in the feature map. The value range of this weight is between 0 and 1, and the larger the value, the more important the position.

[0146] Weighted output: Multiply the original feature map by the calculated attention weights to obtain the weighted feature map, thus achieving the weighting of the feature map.

[0147] (2) Channel Attention:

[0148] Function: The channel attention mechanism aims to enable the model to pay more attention to the importance of different feature channels. Different feature channels may extract different feature information in the image. The channel attention mechanism can enable the model to focus more on those feature channels that are more important for identifying black mold, thereby improving the expression ability of features.

[0149] Implementation steps:

[0150] Feature map input: Use the feature map from the middle layer of the ResNet50 network as the input to the channel attention module.

[0151] Global average pooling: Perform global average pooling on the feature map to compress each channel of the feature map into a single value.

[0152] Calculate channel attention weights: Use two fully connected layers and the Sigmoid function to calculate the attention weights for each channel.

[0153] Weighted output: Multiply the original feature map by the calculated channel attention weights to obtain the weighted feature map, thus achieving the weighting of the feature channels.

[0154] 4.2 Specific implementation:

[0155] The core idea of this embodiment is to embed spatial attention and channel attention modules in the ResNet50 network and improve the feature extraction ability through joint training.

[0156] (1) Feature extraction

[0157] Algorithm: Use the ResNet50 network for feature extraction.

[0158] Formula: Since ResNet50 itself is a combination of a series of convolutions, poolings, and activations, the specific mathematical expressions are not written out here, but it can be simply expressed as:

[0159] F = ResNet50(X), where X is the input image and F is the feature map extracted by ResNet50.

[0160] Explanation: Assume that the feature map extracted by ResNet50 is F ∈ R^(C×H×W), where C is the number of channels, H is the height, and W is the width.

[0161] (2) Attention embedding

[0162] Algorithm: Insert spatial attention module and channel attention module in the middle layer of the ResNet50 network. Here, a parallel connection method is adopted, with spatial attention first and then channel attention.

[0163] (3) Attention calculation

[0164] (3.1) Spatial attention module

[0165] Algorithm:

[0166] a. Perform average pooling and max pooling on the feature map F in the channel dimension respectively to obtain two 1×H×W feature maps.

[0167] b. Concatenate these two feature maps in the channel dimension to obtain a 2×H×W feature map.

[0168] c. Perform a convolution operation on the concatenated feature map to obtain a 1×H×W feature map.

[0169] d. Apply the sigmoid activation function to this feature map to obtain the spatial attention weight Ms.

[0170] Formula:

[0171] F_avg = AvgPool(F) ∈ R^(1×H×W)

[0172] F_max = MaxPool(F) ∈ R^(1×H×W)

[0173] F_concat = Concat(F_avg, F_max) ∈ ℝ^(2×H×W)

[0174] F_conv = Conv(F_concat) ∈ ℝ^(1×H×W)

[0175] Ms = Sigmoid(F_conv) ∈ ℝ^(1×H×W)

[0176] Where:

[0177] a. AvgPool and MaxPool represent average pooling and max pooling operations respectively on the channel dimension; b. Concat represents the concatenation operation on the channel dimension; c. Conv represents the convolution operation; d. Sigmoid represents the sigmoid activation function.

[0178] (3.2) Channel Attention Module

[0179] Algorithm:

[0180] a. Perform average pooling and max pooling on the feature map F in the spatial dimension respectively to obtain two C×1×1 feature maps.

[0181] b. Pass these two feature maps through a shared MLP respectively to obtain two C×1×1 feature maps.

[0182] c. Add these two feature maps, and pass through the sigmoid activation function to obtain the channel attention weight Mc.

[0183] Formula:

[0184] F_avg = AvgPool(F) ∈ ℝ^(C×1×1)

[0185] F_max = MaxPool(F) ∈ ℝ^(C×1×1)

[0186] F_MLP_avg = MLP(F_avg) ∈ ℝ^(C×1×1)

[0187] F_MLP_max = MLP(F_max) ∈ ℝ^(C×1×1)

[0188] F_sum = F_MLP_avg + F_MLP_max ∈ ℝ^(C×1×1)

[0189] Mc = Sigmoid(F_sum) ∈ ℝ^(C×1×1)

[0190] Where:

[0191] a. AvgPool and MaxPool represent average pooling and max pooling operations in the spatial dimension respectively. b. MLP represents a multi-layer perceptron. c. Sigmoid represents the sigmoid activation function.

[0192] (4) Attention fusion

[0193] Algorithm:

[0194] 1. Multiply the spatial attention weight Ms with the feature map F to obtain the spatially attention-weighted feature map F-s.

[0195] 2. Multiply the channel attention weight Mc with F-s to obtain the feature map F-attn that has been weighted by both spatial and channel attention.

[0196] Formula:

[0197] F_s = Ms * F

[0198] F_attn = Mc * F_s

[0199] where * represents element-wise multiplication.

[0200] (5) Model training

[0201] Algorithm: Use the ResNet50 network with an attention mechanism for end-to-end training.

[0202] Formula:

[0203] Loss = LossFunction(Output, GroundTruth)

[0204] Output = ResNet50_with_Attention(X)

[0205] Update model parameters:

[0206] Explanation:

[0207] Define an appropriate loss function (LossFunction), such as the cross-entropy loss function. Through the backpropagation algorithm, calculate the gradients of the loss function with respect to all model parameters (including the parameters of ResNet50 and the attention module). Use an optimizer (such as SGD, Adam) to update the model parameters. Repeat the above steps until the model converges.

[0208] Summary

[0209] Through the above steps, we can embed the spatial attention and channel attention modules into the ResNet50 network and perform joint training, thereby enhancing the model's feature extraction ability. The key points are:

[0210] Location selection: Reasonably select the insertion layer of ResNet50.

[0211] Parallel structure: Spatial attention and channel attention act on the feature map in parallel.

[0212] Attention fusion: Effectively fuse the two attention mechanisms.

[0213] End-to-end training: Simultaneously optimize the parameters of ResNet50 and the attention module.

[0214] 4.3 Experimental data analysis:

[0215] (1) Accuracy improvement: To verify the improvement effect of the dual attention mechanism on the model accuracy, we conducted ablation experiments and compared the classification accuracies of using the original ResNet50 model and the ResNet50 model embedded with the dual attention mechanism. The results show that after embedding the dual attention mechanism, the classification accuracy of the model on the validation set has been significantly improved.

[0216] (2) Original data list:

[0217] To more clearly demonstrate the effectiveness of the dual attention mechanism, we list a set of original data and compare the performances of using the original ResNet50 model and the model after embedding the dual attention mechanism on the validation set, as shown in the following table:

[0218]

[0219] (3) Accuracy improvement list: To more intuitively demonstrate the accuracy improvement effect, we give an accuracy improvement list.

[0220] Model configuration Classification accuracy rate (%) ResNet50 (without attention mechanism) 91.7 ResNet50 + dual attention mechanism 93.7 Accuracy improvement 2.0

[0221] (4) Data analysis: It can be seen from the above data and tables that the overall accuracy of the model after embedding the dual attention mechanism has increased by 2% compared with the original model, indicating that this method can effectively improve the model performance. In terms of sample details, as shown in sample_2 and sample_5, the model after embedding the dual attention mechanism corrected the mispredictions of the original model, indicating that the model can capture the characteristics of diseases more accurately.

[0222] (5) Analysis of reasons:

[0223] Function of spatial attention: The spatial attention mechanism enables the model to pay more attention to the areas of black mold disease spots in the image, reduces the interference of irrelevant areas such as the background, and thus improves the model's recognition ability of disease spots.

[0224] Channel attention effect: The channel attention mechanism enables the model to pay more attention to the feature channels that are more important for identifying black mold disease, enhancing the model's feature extraction ability for different types of black mold disease.

[0225] Fusion method: The two attention mechanisms are fused, enabling the model to simultaneously focus on the regions and channels of significance in the image, allowing the model to extract useful information from different perspectives, thereby performing disease detection more accurately.

[0226] Example 5: Creation of a custom dataset

[0227] This example details how to create a custom dataset through image enhancement and Q&A pair construction to improve the model's generalization ability in different environments and scenarios, thereby enhancing the accuracy of tea tree black mold disease detection.

[0228] 5.1 Image enhancement method:

[0229] (1) Purpose: The purpose of image enhancement is to expand the dataset, generate more diverse image samples, enable the model to better adapt to environmental changes such as different lighting conditions, angles, occlusions, etc., and improve the model's robustness and generalization ability.

[0230] (2) Enhancement operations: We performed the following multiple enhancement operations on each image in the original dataset:

[0231] Random cropping and scaling: Randomly select a local area of the image for cropping and scale the image to the original size, enabling the model to learn lesion features at different scales.

[0232] Add Gaussian noise: Add Gaussian noise to the image to simulate the image noise that may exist in the real environment and improve the model's anti-noise ability.

[0233] Adjust image brightness and contrast: Randomly adjust the brightness and contrast of the image to simulate image differences under different lighting conditions and improve the model's adaptability to lighting changes.

[0234] Use blur and sharpen filters: Apply blur and sharpen filters to the image to simulate the clarity changes of the image under different shooting conditions and improve the model's robustness.

[0235] Perform affine transformation: Perform random rotation, translation, scaling, and shearing affine transformations on the image to simulate shooting angle and position changes and improve the model's adaptability to angle and position changes.

[0236] (3) Enhancement parameters:

[0237] Random cropping: The size of the cropped area is 0.8 - 1.0 times the size of the original image

[0238] Gaussian noise: The standard deviation of the noise is 0 to 0.05

[0239] Brightness adjustment: The brightness adjustment range is 0.8 to 1.2 times

[0240] Contrast adjustment: The contrast adjustment range is 0.8 to 1.2 times

[0241] Rotation angle: -10 to +10 degrees

[0242] Scaling factor: 0.8 to 1.2 times

[0243] Translation distance: -10% to +10%

[0244] 5.2 Q&A pair construction:

[0245] (1) Purpose: The purpose of the Q&A pairs is to enable the model to connect image information and text information, so that the model can not only identify the disease spots, but also understand information such as the type of disease, the severity of the disease, and the location of the disease spots.

[0246] (2) Construction strategy: We have manually constructed multiple Q&A pairs for each enhanced image. The construction of the Q&A pairs follows the following principles:

[0247] Diversity: The Q&A pairs cover different types of questions, such as whether there is a disease, the severity of the disease, the distribution location of the disease, etc.

[0248] Accuracy: Ensure that the answer to each question is consistent with the image content.

[0249] Clarity: Ensure that both the questions and answers are concise and easy to understand.

[0250] Examples of Q&A pairs:

[0251] Question: "Does this leaf have black mold?" Answer: "Yes."

[0252] Question: "What stage is the black mold in?" Answer: "Mid-stage."

[0253] Question: "Which area is the black mold mainly distributed in?" Answer: "Bottom leaves."

[0254] Question: "What is the state of this leaf?" Answer: "Healthy."

[0255] Question: "What is the severity of the black mold on this leaf?" Answer: "Mild."

[0256] More Q&A pairs: According to the actual situation, more Q&A pairs of different types and angles can be constructed to ensure that the model can better understand the image and text information.

[0257] 5.3 Dataset Scale and Distribution:

[0258] (1) Dataset Scale: Create a custom dataset containing 2000 enhanced images and their corresponding Q&A pairs.

[0259] (2) Data Distribution: To ensure that the model can learn different degrees of black mold disease, the custom dataset includes normal leaves, mildly infected, moderately infected, and severely infected leaves, ensuring a balanced distribution of black mold disease infection in the dataset.

[0260] 5.4 Experimental Data Analysis:

[0261] (1) Accuracy Improvement:

[0262] To verify the improvement effect of the custom dataset on the model's generalization ability, we conducted a comparative experiment to compare the detection accuracies of the model trained with the original dataset and the model trained with the original dataset + custom dataset under different environments. Different environments refer to detections under different lighting conditions and different shooting angles. The experimental results show that the detection accuracy of the model trained with the custom dataset has been significantly improved under different environments.

[0263] (2) List of Data Sources for Accuracy Improvement:

[0264] To more clearly demonstrate the effectiveness of the custom dataset, we listed a set of original data and compared the performances of the model trained with the original dataset and the model trained with the original dataset + custom dataset under different environments. For the sake of simplicity, we selected 5 representative test environments for comparison.

[0265] As shown in the following table:

[0266]

[0267] List of Accuracy Improvements:

[0268] Model configuration Classification accuracy rate (%) Training the model only using the original dataset 91.7 Training the model using the original dataset + custom dataset 93.8 Accuracy improvement 2.1

[0269] (3) Data Analysis:

[0270] As can be seen from the table, the accuracy of the model trained with the custom dataset is higher than that of the model trained only with the original dataset in each environment, indicating that the custom dataset can effectively improve the generalization ability of the model in different environments.

[0271] Calculated from the data in the table, after training with the custom dataset, the average accuracy of the model has increased by 2.1%, which also indicates that the custom dataset can effectively improve the generalization ability of the model in different environments.

[0272] Especially for some complex scenarios, such as side light, backlight, and leaf occlusion, the improvement of the model after training with the custom dataset is more significant. This indicates that the custom dataset effectively enhances the model's adaptability to complex environments through various augmentation operations.

[0273] Through the above detailed descriptions, experimental data, list displays, and data analyses, this embodiment clearly elaborates the creation process of the custom dataset and its effectiveness in improving model performance, providing strong support for understanding the technical solution of the present invention.

[0274] Embodiment 6: Model Deployment

[0275] This embodiment details how to lightweight the trained model and deploy it to mobile devices and drones to achieve real-time monitoring of tea tree black mold disease. The flowchart is as Figure 4 shown:

[0276] 6.1 Lightweight Model Construction:

[0277] (1) Purpose: To enable the model to run efficiently on resource-constrained mobile devices and drones, we need to lightweight the trained model. While ensuring model performance, reduce the model size and computational complexity.

[0278] (2) Model Compression Methods: We adopted a variety of model compression techniques, including pruning, quantization, and knowledge distillation, to achieve model lightweighting.

[0279] ① Pruning:

[0280] Principle: Pruning techniques reduce the number of model parameters and computational complexity by removing unimportant connections and neurons in the model.

[0281] Specific Steps:

[0282] Weight Importance Evaluation: Calculate the importance scores of each connection and neuron in the model. Common methods include based on the absolute value of the weight, based on the magnitude of the gradient, etc.

[0283] Connection Pruning: Delete connections and neurons with importance scores below a certain threshold.

[0284] Fine-tuning: Fine-tune the pruned model to restore model performance.

[0285] ② Quantization:

[0286] Principle: Quantization techniques reduce the storage space and computational complexity of a model by converting the model's weights and activation values from high-precision floating-point numbers (e.g., 32-bit) to low-precision integers (e.g., 8-bit).

[0287] Specific steps:

[0288] Post-training quantization: Quantize the weights and activation values in the trained model. Usually, linear quantization methods are used to map floating-point numbers to the integer range.

[0289] Fine-tuning after quantization: Fine-tune the quantized model to minimize the loss of model accuracy.

[0290] ③ Knowledge Distillation:

[0291] Principle: Knowledge distillation techniques improve the performance of a small model (referred to as the student model) by transferring the knowledge of a large model (referred to as the teacher model).

[0292] Specific steps:

[0293] Teacher model: Use the trained original ResNet50 model as the teacher model.

[0294] Student model: Use a smaller model as the student model, such as MobileNet or ShuffleNet.

[0295] Knowledge transfer: Use the prediction results of the teacher model as the target for training the student model.

[0296] Data description: The data for training the student model is the original training dataset, and it is trained using the same training method as the teacher model.

[0297] (3) Parameter description of model compression:

[0298] Pruning ratio: During the pruning process, we retain the top 75% of the parameters ranked by the absolute value of the weights in the model and discard the remaining 25% of the parameters.

[0299] Quantization bits: We use the post-training quantization method to quantize the model weights and activation values from 32-bit floating-point numbers to 8-bit integers.

[0300] Knowledge distillation: We use the MobileNetV2 model with approximately 1 / 4 of the number of parameters of the original ResNet50 model as the student model.

[0301] Lightweight model volume: After compression, the volume of the model is reduced from 100MB to approximately 15MB.

[0302] 6.2 Mobile deployment:

[0303] (1) Deployment platform: We deploy the lightweight model to Android mobile devices and develop an Android application for real-time detection of tea tree black mold disease.

[0304] (2) Deployment steps:

[0305] Model conversion: Convert the model trained with Pytorch to a model format supported by the Android platform, such as TFLite.

[0306] Model loading: Load the converted model into the Android application.

[0307] Image acquisition: Acquire images of tea tree leaves through the mobile phone camera.

[0308] Image preprocessing: Preprocess the acquired images, such as scaling, normalization, etc.

[0309] Model inference: Input the preprocessed images into the model for inference to obtain the disease detection results.

[0310] Result display: Display the detection results (such as disease type, location, severity, etc.) on the mobile phone screen.

[0311] UI design: In terms of UI design, we adopt a simple and clear design style to enable users to quickly get started with the application.

[0312] 6.3 Drone deployment:

[0313] (1) Deployment platform: We integrate the lightweight model into a drone equipped with a camera and use the drone to cruise and collect tea garden images for real-time disease detection.

[0314] (2) Deployment steps:

[0315] Model integration: Integrate the model into the embedded system of the drone.

[0316] Image acquisition: The drone acquires tea garden images through the equipped camera.

[0317] Image preprocessing: Preprocess the acquired images.

[0318] Model inference: Input the preprocessed images into the model for inference.

[0319] Result transmission: Transmit the detection results (such as disease type, location, severity, etc.) to the ground control station via wireless network.

[0320] Result visualization: Display the detection results on the ground control station.

[0321] Drone model: We selected the DJI Mavic 2 series drones for testing and deployed the model on the Jetson Nano development board carried by the drones.

[0322] 6.4 Experimental data analysis:

[0323] Inference speed on the mobile side: The average inference speed of the model on Android mobile devices is 0.2 seconds per image.

[0324] Detection accuracy on the mobile side: The detection accuracy of the model on the mobile side is 92%.

[0325] Comparison with other methods:

[0326] Method Accuracy Inference speed Lightweight Traditional method (manual observation) 80％ 3 minutes / leaf None Existing image recognition method 88％ 0.8 seconds / image No The method of the present invention (mobile device) 92％ 0.2 seconds / image Yes The method of the present invention (drone device) 92％ 0.2 seconds / image Yes

[0327] Example 7: Implementation of federated learning

[0328] This example details how to apply federated learning in the present invention to improve the generalization ability of the model in different tea garden environments and protect the data privacy of each tea garden. We selected the Federated Averaging algorithm (FedAvg) as the aggregation strategy for federated learning and elaborated on its implementation steps.

[0329] 7.1 Federated learning strategy: Federated Averaging algorithm (FedAvg)

[0330] (1) Reason for selection: FedAvg is a commonly used federated learning algorithm. Its advantages lie in its simplicity, ease of implementation, and high computational efficiency, making it suitable for aggregating and updating model parameters. The core idea of the FedAvg algorithm is to train the model in parallel on the local data of each participating party (in the present invention, referring to three different tea gardens), and then average the trained model parameters to obtain the global model.

[0331] (2) Algorithm process:

[0332] Local training: Each participating party independently trains the model on its local data.

[0333] Parameter upload: Each participating party sends the trained model parameters to the central server.

[0334] Parameter aggregation: The central server averages the received model parameters to obtain new global model parameters.

[0335] Model distribution: The central server sends the global model parameters back to each participating party for the next round of local training.

[0336] Iteration: Repeat the above steps until the model converges or reaches a predetermined number of iterations.

[0337] 7.2 Data Preparation:

[0338] (1) Dataset Partitioning: To simulate data for different tea garden scenarios, we divide the original dataset into three subsets according to the tea garden distribution. Each subset contains data from one tea garden. The amount of data in each subset is approximately 1 / 3 of the original dataset, that is, each tea garden has about 2125 images.

[0339] (2) Data Distribution: Try to ensure that within each subset, the infection degree of black mold and the distribution of leaf types are uniform, so as to ensure that the models trained in each tea garden have good performance.

[0340] (3) Data Privacy: After the dataset is partitioned, each tea garden can only access local data and cannot access the data of other tea gardens, thus ensuring data privacy.

[0341] 7.3 Initial Model:

[0342] (1) Initial Model Parameters: Use the pre-trained ResNet50 model parameters as the initial model parameters for federated learning. These pre-trained parameters can provide better initial weights for the model and help the model converge faster.

[0343] (2) BLIP Model: To ensure the consistency of the experiment, we choose to keep the pre-trained BLIP model parameters unchanged.

[0344] 7.4 Local Training:

[0345] (1) Number of Training Epochs: Conduct 5 epochs of local training on the data subsets of each tea garden respectively.

[0346] (2) Optimizer: Use the Adam optimizer and use the same parameter settings as in the global model training.

[0347] (3) Local Model: The local model parameters of each tea garden are optimized independently and only interact with the models of other tea gardens when the model parameters are aggregated.

[0348] 7.5 Parameter Aggregation:

[0349] (1) Federated Averaging: After local training is completed, use the federated averaging algorithm to aggregate the local model parameters of each tea garden to generate global model parameters.

[0350] (2) Parameter Calculation Formula: Assume that the model parameters of the kth tea garden are wk, and there are K tea gardens in total. The calculation formula for the global model parameters w_global is as follows:

[0351] w_global = (1 / K) * sum(wk), where k = 1 to K

[0352] That is: The global model parameters are the average of the local model parameters of each tea garden.

[0353] (3) Aggregation Server: The parameter aggregation operation is completed on the central server.

[0354] 7.6 Model Update:

[0355] (1) Parameter Distribution: Distribute the calculated global model parameters to each tea garden for the next round of local model training.

[0356] (2) Iterative Loop: Each tea garden uses the updated model parameters for a new round of local training, and then repeats the parameter aggregation step until the model converges or reaches the predetermined number of iterations.

[0357] (3) Convergence Judgment: When the training loss of the model no longer decreases significantly or the performance on the validation set no longer improves, it is determined that the model has converged.

[0358] 7.7 Experimental Data Analysis:

[0359] (1) Accuracy Improvement: To verify the effectiveness of the federated learning method, we compared the accuracy of the model trained using the federated learning method and the model trained using the data of a single tea garden.

[0360] (2) Original Data List: To more clearly demonstrate the effectiveness of federated learning, we listed the original accuracies of the models trained using federated learning and the data of a single tea garden on the validation set. For simplicity, we selected three tea gardens for comparison:

[0361]

[0362] (3) List of Average Accuracy Improvements:

[0363] Model configuration Average classification accuracy rate (%) Training using a single tea garden's data 90.06 Training using federated learning 93.97 Accuracy improvement 3.91

[0364] (4) Data Analysis:

[0365] As can be seen from the above table, the accuracy of the federated learning model on the validation sets of the three tea gardens is significantly higher than that of the model trained using the data of a single tea garden, verifying the effectiveness of federated learning.

[0366] The average accuracy after training the federated learning model is 93.97%, while the average accuracy of the model trained with single tea garden data is 90.06%, and the overall improvement of the federated learning model is 3.91%. The performance of the federated learning model on the three tea gardens is close, which indicates the generalization ability of the federated learning, so that there is no obvious performance degradation between different tea gardens. Federated learning enables the model to be trained with the data of each tea garden, avoiding overfitting of the model and improving the generalization performance of the model.

[0367] The foregoing has shown and described the basic principles and main features of the present invention. For those skilled in the art, it is obvious that the present invention is not limited to the details of the above-described exemplary embodiments. Therefore, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, it is intended to embrace all changes falling within the meaning and scope of the equivalent elements of the claims in the present invention.

[0368] In addition, it should be understood that although this specification is described in accordance with the embodiments, not every embodiment only contains an independent technical solution. This narrative manner of the specification is only for clarity. Those skilled in the art should regard the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A diagnostic method for detecting black mold of tea trees, characterized in that: The method comprises the following steps: S1: Obtain image data of tea leaves and construct a dataset containing high-resolution images of healthy and black mold infected tea leaves; S2: Analyzing the image data using a deep learning model to identify the presence and severity of black mold; S3: Combining visual question answering technology, using question-answer pairs corresponding to the image data, to enhance the model's understanding of images and texts, thereby improving recognition accuracy; S4: Using the trained model, real-time detection and diagnosis of tea tree black mold disease can be achieved through mobile networks and drone platforms.

2. The diagnostic method for detecting black mold of tea tree according to claim 1, characterized in that: The image dataset in S1 may be an original dataset and / or a custom dataset.

3. The diagnostic method for detecting black mold of tea tree according to claim 1, characterized in that: The deep learning model in S2 is a ResNet model, and the ResNet50 model is initialized using weights pre-trained on the ImageNet dataset.

4. A diagnostic method for detecting black mold of tea trees according to claim 3, characterized in that: The training strategy of the ResNet model is federated learning. The specific steps of federated learning are as follows: A: Local training: The model is trained on the local data of each tea garden, and the model of each tea garden is trained independently for several epochs; B: Parameter aggregation: After training, the model parameters of each tea garden are sent to the central server for aggregation to generate new global model parameters; C: Model update: The central server sends the updated global model parameters back to each tea garden, and each tea garden retrains the local model with the new model parameters; D: Iteration: Repeat the above steps until the model converges.

5. The diagnostic method for detecting black mold of tea tree according to claim 1, characterized in that: The method also includes applying the BLIP method to enable the model to understand complex inputs of images and text.

6. A diagnostic method for detecting black mold of tea trees according to claim 5, characterized in that: The BLIP model pre-training steps of the BLIP method are as follows: A: Data input: images and corresponding text descriptions are input into the BLIP model; B: Feature extraction: The BLIP model uses its own encoder to extract features from images and texts, and obtains image feature vectors and text feature vectors respectively; C: Cross-modal fusion: fuse the image and text feature vectors and establish a connection between the two through the self-attention mechanism; D: Loss calculation: Calculate the loss between the answer output by the model and the true answer; E: Parameter optimization: Use the gradient descent algorithm to optimize the model parameters so that the model can correctly answer questions about the image content.

7. The diagnostic method for detecting black mold of tea trees according to claim 1, characterized in that: The method also includes using a dual attention mechanism that enables the model to focus on both relevant image regions and question parts simultaneously.

8. A diagnostic method for detecting black mold of tea trees according to claim 7, characterized in that: The dual attention mechanism includes spatial attention and channel attention.

9. The diagnostic method for detecting black mold of tea trees according to claim 2, characterized in that: The creation of the custom dataset includes image enhancement operations and the construction of question-answer pairs.

10. Use of the diagnostic method for detecting tea tree black mold disease as claimed in any one of claims 1 to 9 in detecting tea tree black mold disease in a tea garden.

Images