Method and System for Disease Grading of Staple Crops Based on Prompt-Driven Large Language Models

Abstract

The present invention provides a method and system for grading diseases of staple food crops based on a prompt-driven large model. The method includes the following steps: collecting and preprocessing staple food crop images to obtain a staple food crop dataset; training a ResNet-18 network based on the staple food crop dataset for initial disease grading to obtain disease characterization prompt information; using the disease characterization prompt information to drive the prompt-driven large model to segment the lesion area and leaf area of the staple food crop image to be graded; calculating the ratio of the lesion area to the leaf area to obtain the final fine-grained disease grading result of the staple food crop image to be graded, and completing the grading of diseases of staple food crops based on the prompt-driven large model. The present invention realizes the automatic detection and grading of crop diseases by integrating deep learning and interactive segmentation technology, not only improves the accuracy of disease detection and classification, but also provides the possibility for optimizing the use of pesticides, which helps to reduce environmental pollution and the risk of crop drug resistance.

Description

Technical Field

[0001] The present invention belongs to the technical field of agricultural vision, and specifically relates to a method and system for grading major grain crop diseases based on a prompt-driven large model. Background Art

[0002] In recent years, with the development of agricultural automation technology, especially the application of deep learning technology, the recognition and classification of crop diseases have been significantly improved. These technological advancements have improved the accuracy of disease detection and helped to take control measures earlier. However, existing methods still have certain limitations. Especially in the precise segmentation of disease areas, the complex background and the irregularity of lesion morphology will lead to a decrease in segmentation accuracy, which in turn affects the accuracy of disease classification. In addition, even if the disease is correctly identified, how to scientifically and reasonably apply pesticides remains a problem. Improper use of pesticides not only causes an environmental burden but also may cause the crop to develop resistance, leading to more serious ecological and health problems. Summary of the Invention

[0003] Aiming at the deficiencies of the prior art, the present invention provides a method and system for grading major grain crop diseases based on a prompt-driven large model to improve the accuracy and efficiency of disease detection and grading.

[0004] To achieve the above object, the present invention provides the following solutions:

[0005] A method for grading major grain crop diseases based on a prompt-driven large model, comprising the following steps:

[0006] S1: Collect and preprocess major grain crop images to obtain a major grain crop dataset;

[0007] S2: Train a ResNet-18 network based on the major grain crop dataset, and perform initial disease grading on the major grain crop to be graded based on the trained ResNet-18 network to obtain disease characterization prompt information;

[0008] S3: Use the disease characterization prompt information to drive a prompt-driven large model to segment the lesion area and leaf area of the major grain crop image to be graded;

[0009] S4: Calculate the ratio of the lesion area to the leaf area to obtain the final fine-grained disease grading result of the major grain crop image to be graded, and complete the grading of major grain crop diseases based on a prompt-driven large model.

[0010] Preferably, in step S1, the major grain crop dataset includes a classification dataset, a target detection fine-tuning dataset, and a lesion segmentation dataset;

[0011] The classification dataset includes a disease category dataset, a crop category dataset, a disease characterization category dataset, and a disease initial degree category dataset;

[0012] The object detection fine-tuning dataset uses Labelme to generate object bounding boxes for the leaf and lesion regions, which are used to fine-tune the Grounding DINO model;

[0013] The lesion segmentation dataset uses Labelme to annotate the masks of the lesion regions, which are used to evaluate the lesion segmentation accuracy.

[0014] Preferably, in step S2, the method for obtaining the disease characterization prompt information is as follows:

[0015] Train the first ResNet-18 network using the disease initial degree category dataset, and use the trained first ResNet-18 network to perform initial disease grading on the main food crop image to be graded. Based on the preset disease grading standard, obtain the healthy disease level, the severe disease level, and the moderate disease level;

[0016] Train the second ResNet-18 network based on the disease characterization category dataset, and use the trained second ResNet-18 network to extract the disease characterization prompt information of the main food crop image to be graded at the moderate disease level; where the disease characterization includes spots, fusiform, perforations, and irregularities.

[0017] Preferably, in step S2, it further includes:

[0018] Train the third ResNet-18 network based on the crop category dataset, and use the trained third ResNet-18 network to classify the crop category of the main food crop image to be graded;

[0019] Train the fourth ResNet-18 network based on the disease category dataset, and use the trained fourth ResNet-18 network to classify the disease category of the main food crop image to be graded.

[0020] Preferably, in step S3, the method for obtaining the lesion region and the leaf region of the main food crop image to be graded is as follows:

[0021] Fine-tune the Grounding DINO model based on the object detection fine-tuning dataset, and constrain the fine-tuning process based on the constructed total loss function; where the total loss function includes classification loss, regression loss, and IoU loss;

[0022] Drive the fine-tuned Grounding DINO model based on the disease characterization prompt information to obtain the object bounding box of the lesion region of the main food crop image to be graded;

[0023] Use the target box of the lesion area as the lesion location prompt information to drive the SAM model to segment the lesion area of the main food crop image to be graded;

[0024] Based on the leaf prompt information, drive the un-finetuned Grounding DINO model to detect the target box of the leaf area of the main food crop image to be graded;

[0025] Use the target box of the leaf area as the leaf position prompt information to drive the SAM model to segment the leaf area of the main food crop image to be graded.

[0026] Preferably, the disease characterization prompt information includes the disease image of the main food crop to be graded and the text description of the disease characterization; the method for obtaining the target box of the lesion area of the main food crop image to be graded is:

[0027] Based on the disease image of the main food crop to be graded and the text description of the disease characterization, drive the finetuned Grounding DINO model. The finetuned Grounding DINO model uses the encoder to extract the visual features and text features of the disease image of the main food crop to be graded;

[0028] Refine the visual features based on the language-guided query, and send the text features and the refined visual features into the decoder for object detection to obtain the boundary box of the lesion area.

[0029] The present invention also provides a main food crop disease grading system based on a prompt-driven large model for implementing the method, including:

[0030] A dataset construction module for collecting and preprocessing the main food crop images to obtain a main food crop dataset;

[0031] An initial disease grading module for training the ResNet-18 network based on the main food crop dataset and performing initial disease grading on the main food crop to be graded based on the trained ResNet-18 network to obtain the disease characterization prompt information;

[0032] A detection module for using the disease characterization prompt information to drive the prompt-driven large model to segment the lesion area and the leaf area of the main food crop image to be graded;

[0033] A final disease grading module for calculating the ratio of the lesion area to the leaf area to obtain the final fine-grained disease grading result of the main food crop image to be graded, and completing the main food crop disease grading based on the prompt-driven large model.

[0034] Preferably, it further includes an accuracy evaluation module for evaluating the segmentation accuracy of the lesion area and the leaf area.

[0035] Compared with the prior art, the beneficial effects of the present invention are as follows: By driving the large model, the present invention significantly improves the accuracy of disease identification. Especially when facing disease data containing subtle, different-sized and scattered discriminative features, the present invention shows strong pertinence. The technical solution consists of multiple stages, including crop classification, initial disease grading, prompt information classification, lesion detection, and lesion ratio calculation, etc. Each stage is carefully designed and optimized to ensure the efficiency and accuracy of the overall system. In addition, the present invention also introduces finely tuned GroundingDINO and SAM components, which work together to achieve precise positioning and segmentation of disease sites. Such a design enables the system to effectively detect and analyze plant diseases in the actual agricultural production environment, reduces the disease management cost, and improves the quality and yield of crops. Generally speaking, the present invention provides a more intelligent and efficient solution for modern agriculture and is expected to become an important tool for future disease monitoring and control. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0037] Figure 1 It is a framework diagram of the method for grading staple crop diseases based on a prompt-driven large model in an embodiment of the present invention;

[0038] Figure 2 It is a flowchart of the ResNet-18 network in an embodiment of the present invention;

[0039] Figure 3 It is a flowchart of detecting the location of Grounding DINO lesions in an embodiment of the present invention;

[0040] Figure 4 It is a flowchart of segmenting lesions by prompt information-driven SAM in an embodiment of the present invention;

[0041] Figure 5 It is a flowchart of the disease severity assessment test in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0042] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0043] To make the above objects, features, and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0044] Example 1

[0045] The present invention has developed the SCropDSE system, which is a comprehensive framework for evaluating the severity of diseases (disease grading) in staple crops, as Figure 1 shown. The system includes three main components: image information processing, lesion detection, and disease severity assessment. Image information processing first classifies the severity of diseases in staple crop images (such as severe, general, and healthy). This step addresses the capacity limitation of SAM when segmenting images with overly severe diseases. Based on this preliminary classification, a prompt information model is trained to generate accurate lesion prompts to guide GroundingDINO to locate the lesion areas. In addition, image information processing is also used to train a disease recognition and classification model using the staple crop image database. Lesion detection fine-tunes GroundingDINO (grounding detector) to obtain lesion prompts and staple crop disease images, thereby obtaining boxes for guiding SAM (Segment Anything Model) to accurately segment leaves and lesions. Finally, the disease severity evaluator calculates the proportion of the lesion area based on the leaf and lesion masks obtained by SAM, thereby achieving accurate disease severity assessment.

[0046] A method for grading diseases in staple crops based on a prompt-driven large model includes the following steps:

[0047] S1: Collect and preprocess staple crop images to obtain a staple crop dataset; specifically, in this embodiment, the published dataset resources such as PlantVillage, Corn LeafInfection, etc. are integrated; the collected pictures are preprocessed such as denoising, cropping, adjusting brightness and contrast, etc. to improve the image quality; agricultural experts and professional software (such as labelme) are invited to annotate the lesions in the images, marking information such as the location, size, and type of the lesions.

[0048] A further implementation manner is that in step S1, the staple crop dataset includes a classification dataset, a target detection fine-tuning dataset, and a lesion segmentation dataset;

[0049] The classification dataset includes a disease category dataset, a crop category dataset, a disease characterization category dataset, and a disease initial degree category dataset; among them, the disease category dataset contains 10,000 images under 79 disease categories, which are used to test the performance of the SCropDSE system. The crop category dataset covers seven staple crops, such as rice, sorghum, corn, sweet potato, wheat, soybean, and potato. The disease characterization category dataset is used to train the model that provides GroundingDINO prompt information, and contains 647 images in four categories: spot lesions, fusiform lesions, perforation lesions, and irregular lesions; the disease initial degree category dataset contains 28,968 images for the rough screening of disease severity, which are divided into three categories: healthy, normal, and severe.

[0050] The object detection fine-tuning dataset uses Labelme to generate object bounding boxes for the leaf and lesion areas, which are used to fine-tune the Grounding DINO model; it contains 58 images.

[0051] The lesion segmentation dataset uses Labelme to annotate the masks of the lesion areas, which are used to evaluate the lesion segmentation accuracy. It contains 772 pictures.

[0052] S2: Train the ResNet-18 network based on the staple food crop dataset, and perform initial disease grading on the staple food crops to be graded based on the trained ResNet-18 network to obtain disease characterization prompt information; before performing more fine-grained disease grading in the present invention, first train a ResNet-18 network on the staple food crop dataset to classify the initial severity level of each image. This classification divides the conditions into three categories: healthy, severe, and moderate. Based on the preset severity level division threshold, the images identified as healthy are classified into the severity level 0, while the images with severe symptoms are classified into level 9. Those images belonging to the moderate category go through further processing.

[0053] A further implementation manner is that in step S2, the method for obtaining the disease characterization prompt information is:

[0054] Train the first ResNet-18 network using the disease initial degree category dataset, and perform initial disease grading on the staple food crop images to be graded using the trained first ResNet-18 network. Based on the preset disease grading standard, obtain the healthy disease level, the severe disease level, and the moderate disease level;

[0055] Specifically, input the picture I with a size of 224×224 into the ResNet-18 network. First, pass through a 7×7 convolution, then pass through a max pooling layer, and then continuously pass through multiple such as Figure 2The residual structure shown is finally passed through an average pooling layer, and the final classification result can be output through a fully connected layer. The above process is defined as follows:

[0056] C = FC(AvgPool(F))

[0057] F = ResBlocks(MaxPool(Conv(I))).

[0058] Among them, C is the final classification result, F is the visual feature after convolution and pooling, Conv(·) represents the convolutional layer, MaxPool(·) is the max pooling layer, ResBlocks(·) is the residual block, AvgPool(·) is the average pooling layer, and FC(·) is the fully connected layer.

[0059] Train the second ResNet-18 network based on the disease characterization category dataset, and use the trained second ResNet-18 network to extract the disease characterization hint information of the main food crop images to be classified at the moderate disease level; among them, the disease characterizations include fusiform, perforated, irregular, and spot.

[0060] A further implementation is that in step S2, it further includes:

[0061] Train the third ResNet-18 network based on the crop category dataset, and use the trained third ResNet-18 network to classify the crop categories of the main food crop images to be classified; specifically, in the crop information processing part of the present invention, it also focuses on identifying the specific staple crop types shown in each image. This research mainly focuses on the classification of seven staple crops: corn, potato, rice, sorghum, soybean, sweet potato, and wheat. The trained ResNet-18 network is also used.

[0062] Train the fourth ResNet-18 network based on the disease category dataset, and use the trained fourth ResNet-18 network to classify the disease categories of the main food crops to be classified. Classify different disease categories for each identified crop. This includes early blight, late blight, rust, bacterial infection, and other diseases.

[0063] S3: Use the disease characterization hint information to drive the prompt-driven large model to segment the lesion area and leaf area of the main food crop images to be classified;

[0064] In a further embodiment, in step S3, the method for obtaining the lesion area and leaf area of the main food crop image to be graded is as follows:

[0065] Fine-tune the Grounding DINO model based on the target detection fine-tuning dataset to make the model sensitive to several lesion characterization prompt words (fusiform, perforated, irregular, spot); and constrain the fine-tuning process based on the constructed total loss function; wherein, the total loss function includes classification loss, regression loss, and IoU loss; specifically, the classification loss is responsible for ensuring that the model correctly classifies the lesion type, the IoU loss is used to optimize the position of the bounding box, and the regression loss helps the model learn a more accurate bounding box regression. The definitions of the total loss and the three losses are as follows:

[0066]

[0067] where δ is a parameter used to balance the loss terms, is the i-th predicted bounding box, is the j-th true bounding box, and N is the number of predicted bounding boxes.

[0068] Based on the disease characterization prompt information, drive the fine-tuned Grounding DINO model to obtain the target box of the lesion area of the main food crop image to be graded. In a further embodiment, the disease characterization prompt information includes the disease image of the main food crop to be graded and the text description of the disease characterization. The method for obtaining the target box of the lesion area of the main food crop image to be graded is as follows:

[0069] Based on the disease image of the main food crop to be graded and the text description of the disease characterization, drive the fine-tuned Grounding DINO model. The fine-tuned Grounding DINO model uses the encoder to extract the visual features and text features of the disease image of the main food crop to be graded;

[0070] Refine the visual features based on the language-guided query, and send the text features and the refined visual features into the decoder for object detection to obtain the lesion area bounding box. Specifically, refer to Figure 3 , Grounding DINO receives the plant disease image and its text description as input, enabling the model to locate and identify the disease area in the image based on the text prompt. For example, if the text prompt is "fusiform lesion", the model loads the fine-tuned model weights to extract visual and text features, and refines these visual features through the language-guided selection query. Assuming the input image is I and the text prompt is t, the above process can be defined as follows:

[0071] V = φ(I; θ v )

[0072] Among them, φ(·) represents the visual feature extractor, V represents the extracted visual features, and θ v represents the weight parameters of the model.

[0073] T = ψ(t; θ t )

[0074] Among them, ψ(·) represents the text feature extractor, T represents the extracted visual features, and θ t represents the weight parameters of the model.

[0075] Q = χ(V, T; θ q )

[0076] Among them, χ(·) represents the language-guided query module, V is the above-mentioned extracted visual features, T is the extracted text features, and θ q represents the weight parameters of the model.

[0077] Finally, the refined visual query vector Q and text vector T are fed into the decoder to generate the final bounding box. It is defined as follows:

[0078] Boxes = Decoder(Q, T; θ r )

[0079] Among them, Decoder(·) represents the decoder, and θ r represents the weight parameters of the model.

[0080] Use the target box of the lesion area as the lesion position prompt information to drive the SAM model to segment the lesion area of the main food crop image to be graded; specifically, refer to Figure 4 , SAM processes the input image and the lesion bounding box generated by GroundingDINO through the image encoder and the prompt encoder, and extracts the corresponding visual features and bounding box features. For the input image I, SAM uses the image encoder (such as ViT) to extract the visual feature vector F, which is defined as follows:

[0081] F = ImageEncoder(I)

[0082] The lesion bounding box is represented as a set of coordinates [x0, y0, x1, y1], where (x0, y0) is the upper left corner coordinate of the box, and (x1, y1) is the lower right corner coordinate of the box. The SAM predictor will convert the input box coordinates into proportional coordinates relative to the image size, because the SAM model expects the input box coordinates to be normalized coordinates based on the image size. The bounding box conversion process is defined as follows:

[0083]

[0084] where W and H are the width and height of the image.

[0085] Subsequently, the converted lesion bounding box coordinates [x'0, y'0, x'1, y'1] are input into Figure 4 the prompt encoder shown in the figure, and converted into vectors in the feature space. It is defined as follows:

[0086] P lesions = PromptEncoder([x'0, y'0, x'1, y'1])

[0087] These feature vectors are processed simultaneously by the mask decoder to output the finally segmented lesion area. It is defined as follows:

[0088] Mask lesions = MaskDecoder(F, P lesions )

[0089] Based on the leaf prompt information (leaf), the un-finetuned Grounding DINO model is driven to detect the target bounding boxes of the leaf regions in the main food crop images to be graded;

[0090] The processing of the leaf bounding boxes is the same as above, and the vector P leaf of the leaf boxes can be obtained. It is input into the decoder together to guide the SAM model to segment the leaf regions in the main food crop images to be graded. See Figure 4 , and it is defined as follows:

[0091] Mask leaf = MaskDecoder(F, P leaf )

[0092] S4: Calculate the ratio of the lesion area to the leaf area, obtain the final fine-grained disease grading result of the main food crop images to be graded, complete the disease grading of the main food crops based on the prompt-driven large model, obtain the specific disease levels, and combine the two classification results obtained in step S2: the crop category and the disease category to form a complete disease diagnosis result (including specific crop category information, disease category information, and disease severity level).

[0093] In this embodiment, the disease grading of levels 0 and 9 is obtained in step S2;

[0094] Using the lesion segmentation mask (lesion area) and the leaf mask (leaf area) to calculate the area ratio, obtain the ratio of the lesion to the leaf, and according to the professional ratio grading table, obtain the finer-grained disease grading result of each picture.

[0095] Specifically, based on the areas of the lesion mask and the leaf mask, calculate the ratio of the lesion to obtain, and thus evaluate its disease severity. It is defined as follows:

[0096]

[0097] where mask i,j represents an element in the mask matrix, i and j are pixel indices, and lesions and leaf are the prompt words for detecting lesions and leaves respectively.

[0098] This embodiment further includes S5: Model evaluation.

[0099] Step S51 Lesion location detection evaluation. To constrain the fine-tuning process of GroundingDINO, we constructed a minimized loss function, which consists of three parts: classification loss, regression loss, and IoU;

[0100] Step S52 Segmentation model evaluation. The present invention uses the segmented lesion area and leaf area to calculate the ratio to grade the disease severity, so the disease segmentation accuracy is evaluated.

[0101] Specifically, we adopted relatively common segmentation evaluation metrics. One is the Precision (PA), and the other is the Intersection over Union (IoU), which are defined as follows:

[0102]

[0103] where TP, FP, TN, and FN are defined as follows:

[0104] True Positive (TP) refers to the number of correctly identified positive samples, indicating the situation where the model accurately predicts positive examples.

[0105] False Positive (FP) represents the number of negative samples that are misclassified as positive examples by the model.

[0106] True Negative (TN) corresponds to the number of correctly identified negative samples, where the model accurately predicts negative examples.

[0107] False Negative (FN) reflects the number of positive samples that are misclassified as negative examples by the model.

[0108] Step S6: Testing.

[0109] There are approximately 5000 test images, with a total of 6 levels, namely level 0, level 1, level 3, level 5, level 7, and level 9. The disease severity of each image is evaluated, its evaluation result is compared with the true label, and the evaluation accuracy of each level is calculated.

[0110] Specifically, see Figure 5, during the test, it is divided into five stages. In the first stage, the staple food crops are classified and their disease categories are classified; in the second stage, their initial severity is classified. Images of the "severe" level are defined as level 9, "healthy" images are defined as level 0, and the remaining "medium" images enter the next stage; in the third stage, for images with "medium" severity, lesion characterization prompt information is generated; in the fourth stage, the fine-tuned Grounding DINO processes the leaves and lesion prompts to generate target boxes for the leaves and lesions; in the fifth stage, SAM uses the leaf and lesion target boxes to segment the leaf and lesion areas in the staple food crop image; in the final stage, the ratio of the lesion area to the leaf area is calculated and mapped to the disease severity levels 1, 3, 5, 7, 9.

[0111] Embodiment 2

[0112] The present invention also provides a staple food crop disease grading system based on a prompt-driven large model for implementing the method, including:

[0113] A dataset construction module for collecting and preprocessing staple food crop images to obtain a staple food crop dataset;

[0114] An initial disease grading module for training a ResNet-18 network based on the staple food crop dataset and performing initial disease grading on the staple food crop to be graded based on the trained ResNet-18 network to obtain disease characterization prompt information;

[0115] A detection module for using the disease characterization prompt information to drive a prompt-driven large model to segment the lesion area and leaf area of the staple food crop image to be graded;

[0116] A final disease grading module for calculating the ratio of the lesion area to the leaf area to obtain the final fine-grained disease grading result of the staple food crop image to be graded, and completing the staple food crop disease grading based on the prompt-driven large model.

[0117] A further implementation manner is that it further includes an accuracy evaluation module for evaluating the segmentation accuracy of the lesion area and the leaf area.

[0118] The above-described embodiments are only descriptions of the preferred embodiments of the present invention, and do not limit the scope of the present invention. Without departing from the design spirit of the present invention, various deformations and improvements made by those of ordinary skill in the art to the technical solutions of the present invention shall fall within the protection scope determined by the claims of the present invention.

Claims

1. A staple food crop disease classification method based on a prompt-driven large model, characterized in that: The following steps are involved: S1: Collect and preprocess images of main grain crops to obtain a dataset of main grain crops; S2: Train the ResNet-18 network based on the main grain crop dataset, and perform initial disease classification on the main grain crops to be classified based on the trained ResNet-18 network to obtain disease characterization prompt information; S3: using the disease characterization prompt information to drive the prompt driving large model, segmenting the lesion area and the leaf area of ​​the main grain crop image to be graded; S4: Calculate the proportion of the lesion area to the leaf area, obtain the final fine-grained disease classification result of the main grain crop image to be classified, and complete the classification of the main grain crop disease based on the prompt-driven large model; In step S1, the main food crop dataset includes a classification dataset, a target detection fine-tuning dataset, and a lesion segmentation dataset; The classification data set includes a disease category data set, a crop category data set, a disease representation category data set, and a disease initial degree category data set; The target detection fine-tuning dataset uses Labelme to generate target boxes of leaves and lesion areas for fine-tuning the Grounding DINO model; The lesion segmentation dataset uses Labelme to annotate the mask of the lesion area to evaluate the lesion segmentation accuracy; In step S2, the method for obtaining disease characterization prompt information is: The first ResNet-18 network is trained using the disease initial severity category dataset, and the trained first ResNet-18 network is used to perform initial disease classification on the main grain crop images to be classified, and the healthy disease level, severe disease level, and moderate disease level are obtained based on the preset disease classification standards; A second ResNet-18 network is trained based on the disease representation category dataset, and the trained second ResNet-18 network is used to extract disease representation prompt information of the main grain crop images to be classified at a moderate disease level; wherein the disease representations include spots, spindles, perforations, and irregularities; In step S3, the method for obtaining the lesion area and the leaf area of ​​the main food crop image to be classified is: Fine-tune the Grounding DINO model based on the target detection fine-tuning dataset, and constrain the fine-tuning process based on the constructed total loss function; wherein the total loss function includes classification loss, regression loss and IoU loss; Based on the disease characterization prompt information, the fine-tuned Grounding DINO model is driven to obtain the target frame of the lesion area of ​​the main grain crop image to be graded; The lesion area target frame is used as lesion location prompt information to drive the SAM model to segment the lesion area of ​​the main grain crop image to be graded; The un-fine-tuned Grounding DINO model is driven based on leaf cue information to detect the target box of the leaf area of ​​the main grain crop image to be graded; The leaf region target frame is used as leaf position prompt information to drive the SAM model to segment the leaf region of the main food crop image to be graded.

2. The staple food crop disease classification method based on the prompt-driven large model according to claim 1 is characterized in that: Step S2 also includes: Training a third ResNet-18 network based on the crop category data set, and using the trained third ResNet-18 network to classify the main food crop images to be classified into crop categories; A fourth ResNet-18 network is trained based on the disease category data set, and the trained fourth ResNet-18 network is used to classify the disease categories of the main grain crop images to be classified.

3. The staple food crop disease classification method based on the prompt-driven large model according to claim 1 is characterized in that: The disease characterization prompt information includes a disease image of the main food crop to be classified and a text description of the disease characterization; the method for obtaining the target frame of the lesion area of ​​the main food crop image to be classified is: The fine-tuned GroundingDINO model is driven by the disease images of the main grain crops to be classified and the text descriptions of the disease representation. The fine-tuned Grounding DINO model uses an encoder to extract the visual features and text features of the disease images of the main grain crops to be classified. The visual features are refined based on language-guided queries, and the text features and the refined visual features are sent to a decoder for target detection to obtain a boundary box of the lesion area.

4. A staple food crop disease grading system based on a prompt-driven large model, used to implement the method according to any one of claims 1 to 3, characterized in that: include: A data set construction module is used to collect and pre-process main food crop images to obtain a main food crop data set; The initial disease classification module is used to train the ResNet-18 network based on the main grain crop data set, and perform initial disease classification on the main grain crops to be classified based on the trained ResNet-18 network to obtain disease characterization prompt information; A detection module, used to drive the prompt-driven large model using the disease characterization prompt information to segment the lesion area and the leaf area of ​​the main grain crop image to be graded; The final disease grading module is used to calculate the proportion of the diseased area to the leaf area, obtain the final fine-grained disease grading result of the main grain crop image to be graded, and complete the main grain crop disease grading based on the prompt-driven large model.

5. The staple food crop disease grading system based on the prompt-driven large model according to claim 4 is characterized in that: It also includes an accuracy evaluation module for evaluating the segmentation accuracy of the lesion area and the leaf area.

Images