Training-free plant leaf disease segmentation method and system based on multiple visual basis models

Abstract

The invention discloses a training-free plant leaf disease segmentation method and system based on multiple visual basis models, and the method comprises the steps: extracting disease visual features of a disease region according to a disease reference image provided by a user and a corresponding disease region mask, and constructing a visual prompt memory; obtaining a to-be-segmented image, and performing disease candidate region positioning and screening on the to-be-segmented image in combination with the open vocabulary target detection model to obtain a candidate region set; and obtaining a disease segmentation result based on the visual prompt memory and the candidate region set. According to the method, a visual prompt mechanism is introduced, stable visual prior information highly related to the disease is provided for the visual basic model in the reasoning stage, the accuracy of disease area positioning and segmentation is effectively improved, and dependence on a large amount of labeled data and the model training process is avoided.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing and computer vision, and particularly relates to a training-free plant leaf disease segmentation method set system based on multiple visual basic models. BACKGROUND

[0002] Plant disease area segmentation is one of the key technologies in agricultural image analysis, and its goal is to accurately locate and segment the disease area in the plant leaf image, providing an important basis for disease identification, disease assessment and precision prevention. The plant disease segmentation result can be further applied to disease diagnosis, yield assessment and growth monitoring in intelligent agricultural systems, and has important theoretical significance and application value. However, in natural environment, the plant disease area often presents the characteristics of small size, irregular shape, fuzzy boundary, scattered distribution, and is easily affected by light changes, background interference and noise, making the disease segmentation task highly challenging.

[0003] With the development of deep learning technology, image segmentation methods based on convolutional neural networks have been widely applied in plant disease analysis. Related researches usually construct end-to-end semantic segmentation models, and conduct supervised training on large-scale labeled disease data sets, so as to realize automatic segmentation of disease areas. This kind of method can achieve good results in controlled experimental environment, but its performance is highly dependent on a large amount of high-quality pixel-level labeled data. However, in real agricultural scenes, there are many types of diseases and complex disease stages, and the disease area labeling process requires a lot of professional knowledge and manual cost, making it difficult to obtain large-scale fine labeling data, which seriously limits the practical application of the above method.

[0004] In recent years, with the emergence of visual basic models such as large-scale pre-trained visual encoding models and general segmentation models, their powerful feature representation ability provides a new idea for training-free or weakly supervised image segmentation methods. Some researches attempt to directly use these visual basic models for training-free or few-shot segmentation, avoiding retraining the model for specific tasks. However, since the existing visual basic models are mainly trained on natural images and general semantic targets, they lack the semantic understanding ability in the field of agricultural diseases, which makes it difficult for them to accurately distinguish disease areas from healthy tissues in the plant disease segmentation task. At the same time, plant diseases are not complete and semantically clear independent objects, but local abnormal areas attached to the leaf surface, and there are numerous and fuzzy edges, making it difficult for existing training-free methods based on general targets or full-image segmentation to be directly applied and obtain ideal results.

[0005] Therefore, how to fully tap the representation ability of a visual base model without relying on large-scale labeled data and additional model training, and to design a task-specific design for the structural characteristics of plant diseases to achieve high-precision plant disease area segmentation, is still a technical problem that needs to be solved at present. SUMMARY

[0006] To solve the above technical problems, the present application provides a kind of without training plant leaf disease segmentation method and system based on multiple visual base models, the present application introduces visual prompt mechanism, provides stable and highly related visual prior information for visual base model in reasoning phase, effectively improves the accuracy of disease area positioning and segmentation, avoids the dependence on large amount of labeled data and model training process.

[0007] To achieve the above purpose, the present application provides a kind of without training plant leaf disease segmentation method based on multiple visual base models, comprising:

[0008] According to the disease reference image and the corresponding disease area mask provided by the user, the disease visual features of the disease area are extracted and the visual prompt memory is constructed;

[0009] Obtain the image to be segmented, combine the open vocabulary target detection model to locate and screen the disease candidate area of the image to be segmented, and obtain the candidate area set;

[0010] Based on the visual prompt memory and the candidate area set, the disease segmentation result is obtained.

[0011] Optionally, extracting the visual features of the disease area and constructing the visual prompt memory comprises:

[0012] The disease reference image is encoded using a pre-trained visual encoder to obtain a feature map;

[0013] According to the disease area mask, the features of each connected disease area are extracted from the feature map;

[0014] The features of each connected disease area are aggregated to obtain a disease area representation;

[0015] The disease area representation is collected and stored as the visual prompt memory.

[0016] Optionally, combining the open vocabulary target detection model to locate and screen the disease candidate area of the image to be segmented to obtain the candidate area set comprises:

[0017] Based on the open vocabulary target detection model, different prompt words and detection thresholds are set to obtain leaf bounding boxes and preliminary disease bounding boxes, respectively;

[0018] Screening a preliminary disease boundary box completely located in the leaf boundary box in the image to be segmented to obtain a disease screening set;

[0019] Based on the disease screening set, different screening methods are used according to the characteristics of different plant diseases to obtain a disease boundary box set;

[0020] Based on the disease boundary box set, the candidate region set is obtained.

[0021] Optionally, based on the disease screening set, different screening methods are used according to the characteristics of different plant diseases to obtain a disease boundary box set, which includes:

[0022] When the area of the plant disease region is greater than a preset value, the area ratio of each disease boundary box in the screening set to the corresponding leaf boundary box is calculated; when the area ratio exceeds a preset threshold, the corresponding disease boundary box is removed from the screening set to obtain the disease boundary box set;

[0023] When the area of the plant disease region is not greater than a preset value, it is detected whether there is a nesting relationship in which a first boundary box contains a second boundary box in the screening set; when the nesting relationship exists, the internal boundary box is retained and the external boundary box is removed from the screening set to obtain the disease boundary box set.

[0024] Optionally, after obtaining the candidate region set, the candidate region set is input into a segmentation model to generate a corresponding candidate region segmentation mask set.

[0025] Optionally, based on the visual cue memory and the candidate region set, the disease segmentation result is obtained, which includes:

[0026] Each mask region in the candidate region segmentation mask set is feature-extracted by using a pre-trained visual encoder to obtain a candidate region feature representation;

[0027] The similarity between each candidate region feature representation and all disease region representation vectors in the visual cue memory is calculated to obtain an average similarity score;

[0028] The average similarity score is compared with a preset threshold, and all candidate regions with an average similarity score greater than the preset threshold are retained to generate the disease segmentation result.

[0029] The application also provides a plant leaf disease segmentation system without training based on multiple visual basic models, which includes a visual cue extraction module, a candidate disease region extraction module and a prompt guided optimization module, which includes:

[0030] The visual prompt extraction module is configured to extract disease visual features of the disease area and construct a visual prompt memory according to a user-provided disease reference image and a corresponding disease area mask.

[0031] The candidate disease area extraction module is configured to acquire a to-be-segmented image, combine an open-vocabulary object detection model to locate and screen disease candidate areas of the to-be-segmented image, and acquire a candidate area set.

[0032] The prompt guidance optimization module is configured to acquire a disease segmentation result based on the visual prompt memory and the candidate area set.

[0033] Compared with the prior art, the present application has the following advantages and technical effects:

[0034] 1) The present application introduces a visual prompt mechanism, uses a user-provided disease image-mask pair to construct stable and highly discriminative disease visual features in the reasoning stage, guides the visual basic model to accurately locate disease areas with consistent semantics in the to-be-segmented image, effectively alleviates the segmentation difficulty caused by the small scale, fuzzy boundary and irregular shape of the plant disease area, and improves the accuracy of disease area location and segmentation.

[0035] 2) The present application fully utilizes the powerful semantic representation ability obtained by pre-training the visual basic model on large-scale natural image data, and adapts it to the plant disease segmentation task through method design. Without any additional training or parameter updating for plant diseases, high-precision segmentation of the same type of plant disease is realized, and the dependence on large-scale artificial annotation data and training cost is significantly reduced.

[0036] 3) The present application designs a two-stage screening strategy based on candidate areas according to the characteristics of the large number of plant disease areas, dense distribution and easy false detection, first performs coarse-grained positioning of the disease area, and then combines visual prompts for fine screening, effectively reducing redundant candidates and background interference, and improving the stability and reliability of the segmentation result.

[0037] 4) Since the overall process of the method of the present application does not require model training, it can be directly applied to auxiliary data annotation tasks, and can maintain consistent segmentation performance in laboratory environments and complex natural scenes, showing stronger generalization ability and robustness. BRIEF DESCRIPTION OF DRAWINGS

[0038] The accompanying drawings, which form a part of this application, are included to provide a further understanding of the application and are incorporated in and constitute a part of this application. The embodiments of this application, and their description, are used to explain the application, and do not constitute an improper limitation on the application. In the drawings:

[0039] Figure 1is a flow chart of a plant leaf disease segmentation method without training based on multiple visual basic models according to an embodiment of the present application;

[0040] Figure 2 is a comparison chart of the candidate disease area extraction module proposed in the embodiment and the current mainstream candidate area extraction method in disease area extraction capability, wherein (a) is the extraction effect of the existing method based on SAM to obtain the initial disease area, and (b) is the extraction effect of the candidate disease area extraction module proposed in the embodiment on the disease area;

[0041] Figure 3 is a visual display schematic diagram of the intermediate results and final results of each component of the embodiment of the present application;

[0042] Figure 4 is a visual comparison chart of the method of the embodiment of the present application and other methods. DETAILED DESCRIPTION

[0043] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0044] It should be noted that the steps shown in the flow chart of the drawings can be executed in a computer system such as a group of computer executable instructions, and although the logical order is shown in the flow chart, in some cases, the steps shown or described can be executed in an order different from that here.

[0045] The embodiment proposes a plant leaf disease segmentation method without training based on multiple visual basic models, as shown in Figure 1 The specific steps include the following steps:

[0046] According to the disease reference image and the corresponding disease area mask provided by the user, the disease visual features of the disease area are extracted and the visual cue memory is constructed;

[0047] Obtain the image to be segmented, combine the open vocabulary object detection model to locate and screen the disease candidate area of the image to be segmented, and obtain the candidate area set;

[0048] Based on the visual cue memory and the candidate area set, the disease segmentation result is obtained.

[0049] Specifically, one of the great advantages of the embodiment is that the complex training process required by the traditional method is eliminated, so there is no training stage. The steps of the method of the embodiment are as follows:

[0050] Step 1: Prepare reference disease samples (visual cues);

[0051] The user first prepares a labeled plant disease reference image and its corresponding disease area mask. This image-mask pair is used to describe the visual characteristics of the target disease and is the only prior information for the entire system.

[0052] Step two: Construct visual cue memory

[0053] The system uses DINOv2 to extract features from the reference disease image and combines the disease area mask to extract discriminative visual features of the disease area from the feature map. These features are then aggregated and stored as visual cue memory, which serves as a semantic reference for subsequent disease screening and matching.

[0054] Step three: Input the image to be segmented

[0055] The user inputs any number of plant images to be segmented into the system without additional labeling or training data.

[0056] Step four: Rough positioning of disease candidate areas

[0057] The system uses the candidate disease area extraction module to perform coarse-grained disease area positioning on the image to be segmented, generating a set of candidate areas that may contain diseases. Given the small scale and dense distribution of plant diseases, the system performs preliminary screening on the candidate areas to eliminate obviously irrelevant or unreasonable areas.

[0058] Step five: Fine selection of candidate areas

[0059] The system uses DINOv2 to extract visual features from each candidate area and match them with the reference disease features in the visual cue memory. Through the visual cue-based screening mechanism, the system retains areas that are highly consistent with the reference disease in terms of semantics and appearance, further eliminating false positives and background interference.

[0060] Step six: Generate the final segmentation result

[0061] For the high-confidence candidate areas after screening, the system calls SAM2 to generate a fine-grained pixel-level segmentation mask, outputting the final plant disease segmentation result.

[0062] In the above six steps, only steps one and three require user involvement, and the remaining steps are completed automatically by the system, without the need for the user to participate in any model training, parameter adjustment, or complex interactive operations. The entire process relies only on one reference sample + several images to be segmented, and can complete high-quality plant disease segmentation in complex real-world scenarios. This "one example, generalization" usage makes this embodiment very suitable for agricultural application scenarios where data is scarce and labeling costs are high.

[0063] More specifically, the embodiment is built on multiple pre-trained visual base models, and high-precision segmentation of the same disease is achieved only by relying on a single labeled disease sample through task-specific design. The embodiment has the following advantages: 1) without any additional training of the model, high-precision segmentation can be achieved only by relying on a single reference disease image, significantly reducing the dependence on labeled data; 2) by introducing a visual cue mechanism, the shortcomings of general visual base models in semantic understanding of agricultural diseases are effectively made up; 3) using a two-stage strategy of “candidate region generation-prompt guided screening”, the characteristics of small disease area size, fuzzy boundary and complex distribution can be adapted, and good segmentation effect is still achieved in complex real scenes.

[0064] Further, the visual feature of the disease area is extracted and the visual prompt memory is constructed, including:

[0065] encoding the disease reference image by using the pre-trained visual encoder to obtain a feature map;

[0066] extracting the features of each connected disease area from the feature map according to the disease area mask;

[0067] aggregating the features of each connected disease area to obtain a disease area representation;

[0068] pooling the disease area representations and storing them as visual prompt memories.

[0069] Specifically, first, the embodiment inputs the plant disease reference image provided by the user into DINOv2 for encoding to obtain the corresponding feature map. Then, the features within each disease area are extracted from the feature map by using the corresponding disease area mask, and the average of the features is calculated to obtain the feature vector corresponding to each disease area. These feature vectors are stored as visual prompt memories as the basis for subsequent comparison and screening.

[0070] More specifically, for the image provided by the user and its corresponding label mask, first, the image is encoded by using the DINOv2 encoder to obtain its feature representation. . .

[0071] ;

[0072] For the mask part, the embodiment decomposes it into multiple connected branches according to the connectivity relationship, and each mask maintains the same spatial resolution as the original image. Then, the feature map is adjusted to the same spatial resolution as the mask to obtain . For each mask , the features of the connected disease area are extracted from the feature map The corresponding regions are cropped and the average feature vectors are calculated to obtain a set of disease region representations .

[0073] ;

[0074] Finally, all disease region representations are collected to form a visual cue memory, which is used as a semantic reference for further screening and matching in the subsequent cue guidance optimization module.

[0075] Further, the open-vocabulary object detection model is combined to locate and screen the disease candidate regions of the image to be segmented to obtain a candidate region set, including:

[0076] Based on the open-vocabulary object detection model, leaf boundary boxes and preliminary disease boundary boxes are obtained by setting different cue words and detection thresholds;

[0077] The preliminary disease boundary boxes in the image to be segmented that are completely located in the leaf boundary boxes are screened to obtain a disease screening set;

[0078] Based on the disease screening set, different screening methods are used according to the characteristics of different plant diseases to obtain a disease boundary box set;

[0079] Based on the disease boundary box set, the candidate region set is obtained.

[0080] Based on the disease screening set, different screening methods are used according to the characteristics of different plant diseases to obtain a disease boundary box set, including:

[0081] When the area of the plant disease region is greater than a preset value, the area ratio of each disease boundary box in the screening set to the corresponding leaf boundary box is calculated; when the area ratio exceeds a preset threshold, the corresponding disease boundary box is removed from the screening set to obtain the disease boundary box set;

[0082] When the area of the plant disease region is not greater than a preset value, it is detected whether there is a nesting relationship in which a first boundary box contains a second boundary box in the screening set; when the nesting relationship exists, the internal boundary box is retained and the external boundary box is removed from the screening set to obtain the disease boundary box set.

[0083] Specifically, obtaining candidate segmentation regions is a key step in the two-stage training-free segmentation method. Existing methods usually rely on the "everything" mode of SAM, i.e., the model uses a grid composed of 32x32 uniformly distributed points to generate segmentation results for all targets in the image. This strategy performs well in semantic segmentation tasks, because target objects are usually moderately sized, have clear boundaries, and are not easily confused with the background. However, when this method is applied to plant disease segmentation, it faces significant challenges. Because disease regions are usually small in size, have blurred edges, and are highly similar in appearance to the background, the "everything" mode of SAM often leads to missed detection or even complete segmentation failure, as shown in Figs. 1 (a) and (b). Figure 2

[0084] To solve this problem, the present embodiment introduces Grounding-DINO in the framework. Specifically, first, the language prompt word "leaf" is used, and a high confidence threshold is set to obtain the bounding box of the leaf , which serves as a reference for subsequent screening. Subsequently, the language prompt word "little spots" is used, and a lower threshold is set to extract the bounding boxes of all spot-like regions . These detected spots may be either true disease regions or contain irrelevant background textures or noise. To further refine the results, the present embodiment performs preliminary screening to obtain disease candidate regions. Considering the diversity of plant diseases in appearance and spatial distribution, the present embodiment designs two complementary screening strategies.

[0085] The first step of the two strategies is the same: based on the prior knowledge that plant diseases must occur on the plant body, only disease bounding boxes that are completely within the leaf bounding box are retained . The mathematical form is as follows.

[0086] ;

[0087] In the second step, the present embodiment designs two different screening strategies based on the relationship between the disease bounding box and the leaf bounding box, and the mutual relationship between the disease bounding boxes, respectively. The motivation is that when Grounding-DINO uses relatively vague prompt words such as "little spot", it may generate partially inaccurate bounding boxes. For example, the model may mistakenly include part of the leaf region in the potential disease bounding box, or generate a bounding box that is too large, covering multiple disease regions. Such cases can seriously interfere with the subsequent segmentation process of SAM2. To alleviate this problem and reduce the impact of false positives, the present embodiment adopts the following strategies: ​

[0088] Screening strategy based on the relationship between disease area and leaf: This strategy is suitable for plant diseases with large variation in lesion size and rich internal texture features. When the area of a certain disease bounding box exceeds a certain proportion of the area of its corresponding leaf bounding box, the bounding box is considered as a false detection and is removed.

[0089]

[0090] Screening strategy based on the relationship between disease bounding boxes: This strategy is suitable for plant diseases with small lesion size, large number and dense distribution. When a larger disease bounding box contains a smaller bounding box, the smaller bounding box is retained in this embodiment.

[0091]

[0092] After the above screening, the obtained bounding box set is input into SAM2 as a prompt for generating candidate segmentation masks .

[0093] Further, after obtaining the candidate region set, the candidate region set is input into the segmentation model to generate a corresponding candidate region segmentation mask set.

[0094] Further, based on the visual prompt memory and the candidate region set, the disease segmentation result is obtained, including:

[0095] using the pre-trained visual encoder to extract features from each mask region in the candidate region segmentation mask set, obtaining candidate region feature representations;

[0096] calculating the similarity between each candidate region feature representation and all disease region representation vectors in the visual prompt memory, obtaining an average similarity score;

[0097] comparing the average similarity score with a preset threshold, retaining all candidate regions with an average similarity score greater than the preset threshold, and generating a disease segmentation result.

[0098] Specifically, DINOv2 is used to extract features from disease candidate regions, and these features are compared with reference representations stored in the visual prompt memory. By calculating the cosine similarity between each candidate region feature and the reference prompt feature, the most reliable disease regions are further refined and retained.

[0099] Specifically, for each candidate mask , the DINOv2 backbone network is used to extract its corresponding feature representation ​​These features characterize both the structural and semantic information of the disease area, enabling more discriminative comparison with the reference features based on visual cues.

[0100] To determine the most reliable segmentation result, the embodiment evaluates the similarity between each candidate feature representation and the set of cue features Unlike relying on a single match, the embodiment takes the average of the cosine similarity between the candidate features and all cue features, obtaining a more stable and representative similarity score for each candidate region. Candidate regions with an average similarity score greater than 0.5 will be retained as the final segmentation output. This screening strategy ensures that only regions highly consistent with visual cues in semantics are retained, effectively suppressing background noise and false positive regions while maintaining high accuracy in disease area segmentation.

[0101] ;

[0102] The embodiment also provides a plant leaf disease segmentation system based on multiple visual base models without training, including a visual cue extraction module, a candidate disease area extraction module, and a cue-guided optimization module, comprising:

[0103] The visual cue extraction module is used to extract disease visual features of the disease area and construct a visual cue memory according to a user-provided disease reference image and a corresponding disease area mask.

[0104] The candidate disease area extraction module is used to obtain a to-be-segmented image, combine an open vocabulary object detection model to locate and screen disease candidate regions of the to-be-segmented image, and obtain a candidate region set.

[0105] The cue-guided optimization module is used to obtain a disease segmentation result based on the visual cue memory and the candidate region set.

[0106] Specifically, the method framework of the embodiment mainly consists of the following three core modules:

[0107] I. Visual cue extraction module.

[0108] This module takes the user-provided plant disease image and its corresponding disease area mask as input, extracts deep visual features of the disease area based on a visual base model, aggregates and represents the disease area features, and constructs a visual cue library for subsequent matching, thereby forming the most representative reference features for the target disease.

[0109] II. Candidate disease area extraction module.

[0110] The module performs coarse-grained positioning on potential disease areas in the image to be segmented based on an open vocabulary target detection model, and generates a set of disease candidate areas. In view of the characteristics of a large number of plant disease areas, small scales and fuzzy boundaries, the module designs a candidate area screening strategy suitable for the disease scene to improve the accuracy and effectiveness of the candidate areas.

[0111] III. Prompt guidance optimization module.

[0112] The module extracts features of the candidate disease areas by using a visual base model, and performs semantic matching with reference features constructed in the visual prompt extraction module. The candidate areas are further screened by similarity measurement to remove false detections and background areas, and finally a high-precision plant disease segmentation result is obtained.

[0113] The present embodiment will be described in detail below with reference to the accompanying drawings:

[0114] The method disclosed in the present embodiment is verified on the disclosed apple disease dataset, grape and strawberry disease dataset, and potato disease dataset built by the present embodiment, and compared with existing plant disease segmentation methods to verify the effectiveness and superiority of the present invention in the plant disease area segmentation task.

[0115] The present embodiment performs comparative experiments with various deep learning methods, covering full supervision, semi-supervision, weak supervision, unsupervised and training-free methods based on prompts, to comprehensively verify the performance advantages of the proposed method. Specifically, U-Net and DeepLab are representative full-supervised methods; ResNet-CAM is a weak-supervised method, and WSLSS is a semi-supervised method. In terms of unsupervised methods, the present embodiment focuses on comparing the present method with FOUND and UPLS recently proposed. In addition, in the training-free method, the present embodiment selects VPII, FVP and the latest TRIS as comparison objects.

[0116] The present embodiment uses three commonly used evaluation indexes to evaluate the segmentation performance: Intersection over Union (IoU), Precision and Recall. Among them, IoU is used to measure the degree of overlap between the predicted segmentation region and the real labeled region, and is a comprehensive index reflecting the overall accuracy of segmentation; Precision represents the proportion of pixels that are truly in the disease area among all pixels predicted to be disease, and is used to evaluate the accuracy of the segmentation result; Recall represents the proportion of pixels that are successfully predicted among all real disease pixels, and is used to measure the integrity of the segmentation result. The above three indexes complement each other, and can comprehensively and intuitively evaluate the segmentation result from the aspects of accuracy and integrity. The quantitative evaluation results of comparison are shown in Table 1. It can be seen that the method of the present embodiment exceeds all non-full supervision methods in the core index IoU.

[0117] Table 1

[0118]

[0119] Figure 3 The present embodiment shows the output results of each module of the method on multiple challenging images, including the following scenarios: the lesion area is extremely small and the boundary is blurred, the lesion area is too large, and the background is complex and easy to be confused with the disease area. The first behavior is the original image; the second behavior is the initial bounding box obtained by Grounding-DINO, wherein the red box represents the leaf detection result and the blue box represents the disease detection result; the third behavior is the bounding box after initial screening; the fourth behavior is the real label (annotated true value); the fifth behavior is the candidate disease area generated by SAM2 using the screened bounding box as the frame prompt; and the sixth behavior is the final segmentation result after optimization processing.

[0120] Figure 4The segmentation results of different methods are intuitively compared. It can be observed that the fully supervised method performs well on the disease types with sufficient samples in the training set, but its performance will drop significantly (as shown in the second and third rows) once it is applied to a data set without corresponding training samples (i.e., relying only on the model's generalization ability). On more complex data sets (the training set cannot cover all disease morphologies in the test image), this method also performs poorly on the segmentation of subtle disease regions or regions adjacent to the disease boundary, and its performance is inferior to the method of the present embodiment. As one of the latest unsupervised methods, UPLS performs well on relatively simple data sets, but its segmentation quality in terms of fine-grained details is still inferior to the method of the present embodiment; when applied to complex data sets with limited data, its performance degrades significantly, indicating that it is not robust enough in challenging scenarios. In contrast, the no-training method relies heavily on the feature discrimination ability of models such as DINO, which often misjudges the entire leaf as a disease area; even if TRIS does not segment the entire leaf as a disease area, it cannot capture most of the real disease areas. This problem stems from the inherent limitations of the SAM "everything" mode, which generates a mask based only on simulated grid points, making it difficult to accurately detect small or blurred disease areas. The present embodiment not only achieves more complete segmentation coverage, but also has better boundary accuracy, and is superior to existing methods in terms of region integrity and fine-grained edge depiction.

[0121] The above is only a preferred specific embodiment of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A training-free method for segmenting plant leaf diseases based on multiple visual baseline models, characterized in that, include: Based on the disease reference images and corresponding disease area masks provided by the user, extract the visual features of the disease area and construct a visual cue memory; The image to be segmented is obtained, and the open vocabulary object detection model is used to locate and filter the disease candidate regions in the image to be segmented, and a set of candidate regions is obtained. Based on the visual cue memory and candidate region set, the disease segmentation result is obtained.

2. The method for segmenting plant leaf diseases without training based on multiple visual basic models according to claim 1, characterized in that, Extracting visual features of diseased areas and constructing visual cue memory includes: The disease reference image is encoded using a pre-trained visual encoder to obtain a feature map; Based on the disease area mask, the features of each connected disease area are extracted from the feature map; The features of each connected diseased region are aggregated to obtain a representation of the diseased region. The diseased areas are represented and stored as the visual cue memory.

3. The method for segmenting plant leaf diseases without training based on multiple visual baseline models according to claim 1, characterized in that, The open-vocabulary object detection model is used to locate and filter candidate regions of disease in the image to be segmented, and the candidate region set is obtained, including: Based on the open vocabulary target detection model, by setting different prompt words and detection thresholds, the leaf bounding box and the preliminary disease bounding box are obtained respectively. Filter the preliminary disease bounding boxes that are completely located within the leaf bounding boxes in the image to be segmented to obtain a disease screening set; Based on the disease screening set, different screening methods are used to obtain disease bounding box sets according to the characteristics of different plant diseases. Based on the set of disease boundary boxes, the set of candidate regions is obtained.

4. The method for segmenting plant leaf diseases without training based on multiple visual basic models according to claim 3, characterized in that, Based on the aforementioned disease screening set, different screening methods are used to obtain disease bounding box sets according to the characteristics of different plant diseases, including: When the area of ​​the plant diseased region is greater than a preset value, the area ratio of each disease boundary box to the corresponding leaf boundary box in the filter set is calculated; when the area ratio exceeds a preset threshold, the corresponding disease boundary box is removed from the filter set to obtain the disease boundary box set. When the area of ​​the plant diseased region is not greater than a preset value, it is detected whether there is a nested relationship in the filter set where the first bounding box contains the second bounding box; when the nested relationship exists, the inner bounding box is retained and the outer bounding box is removed from the filter set to obtain the disease bounding box set.

5. The method for segmenting plant leaf diseases without training based on multiple visual basic models according to claim 1, characterized in that, After obtaining the candidate region set, the candidate region set is input into the segmentation model to generate the corresponding candidate region segmentation mask set.

6. The method for segmenting plant leaf diseases without training based on multiple visual basic models according to claim 5, characterized in that, Based on the visual cue memory and candidate region set, the disease segmentation results are obtained as follows: The pre-trained visual encoder is used to extract features from each mask region in the candidate region segmentation mask set to obtain the candidate region feature representation; Calculate the similarity between the feature representation of each candidate region and the representation vectors of all diseased regions in the visual cue memory to obtain the average similarity score; The average similarity score is compared with a preset threshold, and all candidate regions with an average similarity score greater than the preset threshold are retained to generate the disease segmentation result.

7. A training-free plant leaf disease segmentation system based on multiple visual baseline models, used to implement the method as described in any one of claims 1-6, characterized in that, include: The visual cue extraction module, the candidate disease area extraction module, and the cue guidance optimization module include: The visual cue extraction module is used to extract the visual features of the disease area and construct a visual cue memory based on the disease reference image and the corresponding disease area mask provided by the user. The candidate disease region extraction module is used to acquire the image to be segmented, and combine the open vocabulary target detection model to locate and filter the candidate disease regions in the image to be segmented, thereby obtaining a set of candidate regions. The prompting and guidance optimization module is used to obtain disease segmentation results based on the visual prompt memory and candidate region set.

Images