Target detection model and training method, and plant disease diagnosis method and system

Abstract

The application discloses a target detection model and a training method and a plant disease diagnosis method and system, relates to the technical field of intelligent agriculture and artificial intelligence, and the plant disease diagnosis method comprises the following steps: receiving input information of a user, wherein the input information comprises input text and input images; positioning and classifying disease areas of the input images based on a trained target detection model, and outputting detection results; constructing diagnosis prompt words based on the input text, the detection results and environmental context information, using a large language model to perform reasoning based on the diagnosis prompt words and generating responses, and the method can realize high-precision plant disease diagnosis.

Description

Technical Field

[0001] This application relates to the fields of smart agriculture and artificial intelligence technology, and in particular to a target detection model and training method, and a plant disease diagnosis method and system. Background Technology

[0002] During field cultivation, citrus fruits are frequently affected by both biotic stress (pathogenic fungi, bacteria, viruses, and pests) and abiotic stress (extreme weather, nutrient imbalance), making them highly susceptible to phenotypic abnormalities, which can lead to fruit deterioration and even large-scale yield reduction. For a long time, orchard disease inspection has relied heavily on manual experience, resulting in low efficiency, strong subjectivity, and a high rate of missed diagnoses. Especially when facing devastating and highly contagious diseases such as citrus Huanglongbing (HLB), traditional experience-based diagnosis often lags behind, missing the golden period for prevention and control, leading to large-scale disease spread and incalculable economic losses.

[0003] With the development of artificial intelligence technology, computer vision technologies, represented by hyperspectral imaging, image classification, and target detection, have been gradually introduced into the fields of citrus phenotypic analysis and disease detection. However, traditional vision technologies are mostly limited to a single "perception and recognition" level, making it difficult to integrate complex agricultural planting context information and directly output practical and precise prevention and control strategies. At the same time, although LLM has excellent natural language understanding and logical reasoning capabilities, it is limited by the "text-only" barrier and cannot directly analyze visual signals; while Visual-Language Large Model (VLM) has initially broken through the dual-modal interaction of text and images, it still suffers from serious "domain knowledge illusion" and information output failure problems when facing highly vertical professional scenarios such as "citrus micro-lesion identification and complex disease characterization and plant protection plan generation." In addition, its high fine-tuning threshold and computing power cost make it difficult to deploy on a large scale in real field environments. Summary of the Invention

[0004] Therefore, it is necessary to address the problem of insufficient accuracy in diagnosing plant diseases and pests in existing technologies by providing a target detection model and training method, as well as a plant disease diagnosis method and system. The specific technical solution is as follows: Firstly, this application proposes an object detection model, including a backbone network, a neck network, and a decoder. The backbone network is used to extract visual features from the input image. The neck network is connected to the backbone network and is used to perform multi-scale fusion on the extracted visual features. The decoder is connected to the neck network and is used to perform bounding box decoding on the multi-scale fused features. The decoder includes a fine-grained distribution refinement module and an edge correlation constraint distribution refinement module. The fine-grained distribution refinement module is used to output the corresponding edge probability distribution for the four edges of the bounding box. The edge correlation constraint distribution refinement module is connected between the output end of the fine-grained distribution refinement module and the bounding box coordinate decoding end, and is used to perform correlation constraint refinement on the probability distribution of the four edges of the bounding box.

[0005] Furthermore, the specific details of refining the probability distribution of the four edges of the bounding box by applying correlation constraints are as follows: Step 1: Add a learnable edge embedding matrix to the probability distribution of the four edges. Distinguish the distribution characteristics of the top, bottom, left, and right edges:

[0006] in The first output of the fine-grained distribution refinement module Layer edge probability distribution To refine the number of intervals in the distribution; Step 2: Project the type-embedded distribution features onto a low-dimensional interaction space:

[0007] in For the input projection matrix, For input projection bias, For dimensions; Step 3: Calculate the relevance weights between edges using a self-attention mechanism, and set the query matrix. Key matrix Value matrix ,in Given the linear projection matrices of the query, key, and value respectively, the attention output is:

[0008] in For batch size, the Softmax function normalizes along the key dimension, and the left and right edges are separated by attention weights. Learning the width constraint correlation, the upper and lower edges are connected via... Learning highly constrained correlations, and The components are taken from the corresponding edge positions in the attention weight matrix; Step 4: Project the attention output back to the original distribution dimension to obtain the geometric constraint correction:

[0009] in To output the projection matrix, For output projection bias, This is a geometric constraint correction amount; Step 5: Generate an adaptive gating value based on the splicing features of the original distribution, and obtain the edge probability distribution after gating fusion based on the adaptive gating value; The expression for the adaptive gating value is:

[0010] in This represents concatenating the probability distributions of the four edges along the edge type dimension. splicing operation of dimensional tensors. For gated projection matrix, For gated bias. Given a sigmoid function, the marginal probability distribution after gated fusion is as follows:

[0011] in This represents element-wise multiplication. This represents the edge probability distribution after gating fusion. Step 6: Renormalize the fused marginal probability distribution:

[0012] The Softmax function normalizes along the distribution dimension. The refined edge probability distribution output by the edge correlation constraint distribution refinement module.

[0013] Secondly, this application proposes a training method for an object detection model, which includes a backbone network, a neck network, and a decoder. The decoder includes a fine-grained distribution refinement module and an edge correlation constraint distribution refinement module. The training method includes three stages: Initialization phase: Load the pre-trained official weights into the original network parameters of the backbone network, encoder and decoder, and initialize the parameters of the edge correlation constraint distribution refinement module to an approximately identity mapping state; Warmup phase: Freeze the original network parameters and train only the edge correlation constraint distribution refinement module to adapt it to the edge distribution characteristics. Joint fine-tuning phase: Unfreeze the original network parameters and adjust the model to convergence using the preset learning rate.

[0014] Thirdly, this application proposes a plant disease diagnosis method based on a trained target detection model, comprising the following steps: S1: Receive user input information, which includes text and images; S2: Based on the trained target detection model, locate and classify the diseased areas in the image, and output the detection results; S3: Construct diagnostic prompts based on text, detection results, and environmental context information, and use a large language model to perform inference and generate responses based on the diagnostic prompts.

[0015] Furthermore, the specific content of the response is as follows: When the large language model determines that it has sufficient plant protection knowledge to complete the diagnosis, it directly outputs a disease diagnosis prescription report. When the large language model determines that plant protection knowledge needs to be supplemented, the output tool call command calls the corresponding knowledge base tool. The knowledge base tool is then used to search the knowledge base based on an iterative deep-search strategy until the large language model determines that it has sufficient plant protection knowledge to complete the diagnosis, at which point a disease diagnosis prescription report is output.

[0016] Furthermore, the iterative deep-search strategy includes a directory exploration phase, a file reading phase, and a cross-validation phase. The directory exploration phase uses a directory exploration tool to explore layer by layer from the root directory of the knowledge base. The large language model decides the next exploration direction based on the returned directory list until the leaf directory containing the target file is located. The file reading phase uses a file reading tool to read the entire content of the target file and returns it to the large language model context. The cross-validation phase uses the large language model to determine whether the acquired plant protection knowledge is sufficient. If insufficient, it generates new tool call instructions to iteratively execute the directory exploration phase and the file reading phase.

[0017] Furthermore, the knowledge base is constructed as a tree-structured directory knowledge base, with the root directory of the knowledge base organized according to the agricultural plant protection knowledge system, including a plant directory, a disease type database, a plant protection equipment database, and a cultivation technology database.

[0018] Furthermore, the execution flow of the directory exploration tool includes: firstly, validating the input path through secure path parsing, including path traversal protection and directory existence verification; then, traversing all entries under the directory, filtering hidden files, and classifying subdirectories and files of preset types respectively; finally, returning a structured directory list.

[0019] Furthermore, the execution process of the file reading tool includes: firstly, verifying the validity of the input path through secure path parsing, including path traversal protection, file existence verification, and file format verification; then, reading the entire content of the file in a preset format encoding.

[0020] Fourthly, this application proposes a plant disease diagnosis system based on a pre-trained target detection model, including an interactive output module, a reasoning and decision-making module, a visual perception module, and a knowledge base module. The interactive output module receives user input information and distributes it to the corresponding module for processing according to the type of input information; it also outputs a disease diagnosis prescription report. The visual perception module receives user input images distributed by the interactive output module and locates and classifies the diseased areas in the images based on the pre-trained target detection model, outputting the detection results. The reasoning and decision-making module receives user input text distributed by the interactive output module and the detection results output by the visual perception module, and constructs diagnostic prompts by combining environmental context information. A large language model generates a disease diagnosis prescription report based on the prompts and calls the knowledge base module, which is then transmitted to the interactive output module. The knowledge base module stores plant protection knowledge and provides a standardized query interface for the reasoning and decision-making module to call.

[0021] Beneficial effects: This application proposes a target detection model and training method, as well as a plant disease diagnosis method and system. The plant disease diagnosis method classifies and locates diseases based on the trained target detection model, thereby achieving high-precision diagnosis of plant diseases and pests. Attached Figure Description

[0022] Figure 1 This is a flowchart of a plant disease diagnosis method based on a trained target detection model, as described in an embodiment of this application. Figure 2 This is an architecture diagram of a plant disease diagnosis system based on a trained target detection model, as described in an embodiment of this application. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0024] Example 1 This embodiment proposes an object detection model, including a backbone network, a neck network, and a decoder. The backbone network is used to extract visual features from the input image. The neck network is connected to the backbone network and is used to perform multi-scale fusion of the extracted visual features. The decoder is connected to the neck network and is used to perform bounding box decoding on the multi-scale fused features. The decoder includes a fine-grained distribution refinement module and an edge correlation constraint distribution refinement module. The fine-grained distribution refinement module is used to output the corresponding edge probability distribution for the four edges of the bounding box. The edge correlation constraint distribution refinement module is connected between the output end of the fine-grained distribution refinement module and the bounding box coordinate decoding end, and is used to perform correlation constraint refinement on the probability distribution of the four edges of the bounding box.

[0025] In one specific embodiment, the target detection model is an improvement on the D-FINE-L target detection model.

[0026] It should be noted that the specific details of refining the probability distribution of the four edges of the bounding box by applying correlation constraints are as follows: Step 1: Add a learnable edge embedding matrix to the probability distribution of the four edges. Distinguish the distribution characteristics of the top, bottom, left, and right edges:

[0027] in The first output of the fine-grained distribution refinement module Layer edge probability distribution Indicates the decoder layer index. The number of intervals for distribution refinement (i.e., the number of discretized grids used in D-FINE-L to model edge offsets); Step 2: Project the type-embedded distribution features onto a low-dimensional interaction space:

[0028] in For the input projection matrix, For input projection bias, Set as a dimension ; Step 3: Calculate the relevance weights between edges using a 4-head self-attention mechanism, and set the query matrix. Key matrix Value matrix ,in Given the linear projection matrices of the query, key, and value respectively, the attention output is:

[0029] in For batch size, the Softmax function normalizes along the key dimension (i.e., the edge dimension), with the left and right edges separated by attention weights. Learning the width constraint correlation, the upper and lower edges are connected via... Learning highly constrained correlations, and The components are taken from the corresponding edge positions in the attention weight matrix; Step 4: Project the attention output back to the original distribution dimension:

[0030] in To output the projection matrix, For output projection bias, This is a geometric constraint correction amount; Step 5: Generate an adaptive gate value based on the splicing features of the original distribution. The expression for the adaptive gate value is:

[0031] in This represents the concatenation of the probability distributions of the four edges along the edge type dimension (i.e., dimension 3). splicing operation of dimensional tensors. For gated projection matrix, For gated bias. The sigmoid function is used. The resulting distribution after fusion is:

[0032] in This represents element-wise multiplication. This represents the edge probability distribution after gating fusion. Step 6: Renormalize the fused marginal probability distribution:

[0033] The Softmax function normalizes along the distribution dimension (i.e., dimension 4, size N+1) to ensure that the sum of the probability distributions for each edge is 1. The refined edge probability distribution output by the edge correlation constraint distribution refinement module is converted into edge offsets through the D-FINE original weight function (i.e., weighted summation of discretized grid positions), and then decoded into bounding box coordinates.

[0034] Example 2 This embodiment proposes a training method for an object detection model. The object detection model includes a backbone network, a neck network, and a decoder. The decoder includes a fine-grained distribution refinement module and an edge correlation constraint distribution refinement module. The training method includes three stages: Initialization phase: Load the pre-trained official weights into the original network parameters of the backbone network, encoder and decoder, and initialize the parameters of the edge correlation constraint distribution refinement module to an approximately identity mapping state; Warmup phase: Freeze the original network parameters and train only the edge correlation constraint distribution refinement module to adapt it to the edge distribution characteristics. Joint fine-tuning phase: Unfreeze the original network parameters and adjust the model to convergence using the preset learning rate.

[0035] In one specific embodiment, the training and deployment of an improved D-FINE-L model for citrus diseases includes the following specific aspects: Dataset Preparation: A dataset of images of citrus diseases was collected, covering various common disease types such as citrus canker, Huanglongbing (HLB), sooty mold, and scale insect disease. Each disease is labeled with bounding box coordinates and category labels. The dataset was divided into training and validation sets at a ratio of 9:1, and augmented using strategies such as Mosaic data augmentation, random horizontal flipping, and color matting.

[0036] Model modification: D-FINE-L is adopted as the basic target detection backbone network. An edge correlation constraint module EC-FDR is inserted into the fine-grained distribution refinement process of the decoder. The EC-FDR module performs edge type embedding, interactive spatial projection, edge self-attention interaction, output projection, gated residual fusion and Softmax normalization on the probability distribution of the four edges of the bounding box, and outputs a refined distribution of fused geometric constraints.

[0037] Weight loading and initialization: D-FINE-L weights pre-trained on the COCO dataset are used as initialization. During loading, the original parameters in the backbone, hybrid encoder and decoder are directly loaded from the official weights. The EC-FDR module parameters are specially initialized: edge_embed is initialized to N(0, 0.01), attention projection matrices WQ and WK are initialized to identity matrices, WV and Wout are initialized to zero matrices, and the gating bias bg is initialized to -2.0 (making the initial gating value sigmoid(-2)≈0.12).

[0038] Phase 1 Training (Warmup): Freeze all parameters except EC-FDR, use the AdamW optimizer, learning rate 1e-4, batch size 16, and train for 20 epochs. In this phase, the EC-FDR module learns the edge distribution features of citrus disease lesions without destroying the already converged trunk representation.

[0039] Phase Two Training (Joint Fine-Tuning): Unfreeze all parameters, reduce the learning rate to 5e-5, employ cosine annealing, and continue training for 280 epochs until convergence. Monitor the validation set mAP metric and save the optimal model.

[0040] Model Export and Deployment: The trained improved D-FINE-L model is exported to ONNX format and encapsulated as a standalone image processing API interface, utilizing GPU acceleration for high-concurrency inference. The interface input is standardized image data, and the output is structured JSON data containing disease category, confidence level, and bounding box coordinates.

[0041] Example 3 This embodiment proposes a plant disease diagnosis method based on a trained target detection model. The flowchart of the method is as follows: Figure 1 As shown, it includes the following steps: S1: Receive user input information, which includes text and images; S2: Based on the trained target detection model, locate and classify the diseased areas in the image, and output the detection results; S3: Construct diagnostic prompts based on text, detection results, and environmental context information, and use a large language model to perform inference and generate responses based on the diagnostic prompts.

[0042] In one specific embodiment, the detection results are injected into the LLM's input sequence in structured JSON format. The JSON fields include disease class, confidence level, and image resolution information. Simultaneously, the user's text description (e.g., "yellow spots appear on the leaves") and environmental context information (e.g., geographical location, temperature, humidity, season) are concatenated to construct a complete diagnostic Prompt. The Prompt template includes two parts: a System Prompt and a Tool Description. The System Prompt explicitly specifies the LLM's role as a citrus plant protection expert.

[0043] It should be noted that the specific content of the response described in step S3 is as follows: When the large language model determines that it has sufficient plant protection knowledge to complete the diagnosis, it directly outputs a disease diagnosis prescription report. When the large language model determines that plant protection knowledge needs to be supplemented, the output tool call command calls the corresponding knowledge base tool. The knowledge base tool is then used to search the knowledge base based on an iterative deep-search strategy until the large language model determines that it has sufficient plant protection knowledge to complete the diagnosis, at which point a disease diagnosis prescription report is output.

[0044] It should be noted that the knowledge base is constructed as a tree-structured directory knowledge base. The root directory of the knowledge base is organized according to the agricultural plant protection knowledge system, including a plant directory, a disease type database, a plant protection equipment database, and a cultivation technology database.

[0045] In one specific implementation, when the LLM determines that supplementary plant protection knowledge is needed, it generates a structured tool call instruction, which includes the tool name and arguments. The tool call instruction follows a JSON Schema format for easy backend parsing and execution. The LLM then decides whether to invoke the tool based on the context information.

[0046] It should be noted that the iterative deep-search strategy includes a directory exploration phase, a file reading phase, and a cross-validation phase. The directory exploration phase uses a directory exploration tool (kb_explore_dir) to explore layer by layer from the root directory of the knowledge base. The large language model decides the next exploration direction based on the returned directory list until the leaf directory containing the target file is located. The file reading phase uses a file reading tool (kb_read_file) to read the entire content of the target file and return it to the large language model context. The cross-validation phase determines whether the acquired plant protection knowledge is sufficient; if insufficient, it generates new tool call instructions to iteratively execute the directory exploration phase and the file reading phase.

[0047] More specifically, the execution flow of the directory exploration tool includes: first, validating the input path through secure path parsing, including path traversal protection and directory existence verification; then, traversing all entries under the directory, filtering hidden files, and classifying subdirectories and files of preset types; finally, returning a structured directory list. The execution flow of the file reading tool includes: first, validating the input path through secure path parsing, including path traversal protection, file existence verification, and file format verification; then, reading the entire content of the file using a preset encoding format.

[0048] In one specific implementation, each tool execution result is fed back into the LLM context in Observation format, forming a multi-round interactive loop of "inference-call-observation-re-inference". Iteration termination conditions include: the LLM determines that the knowledge is sufficient, or two consecutive tool calls do not acquire new information. The diagnostic prescription report includes disease characterization (confirming the disease name and severity), etiological analysis (inferring the inducing factors based on environmental context), and differential diagnosis (excluding similar diseases). Prescription-level solution generation includes: pesticide recommendation (extracting pesticide name, dilution ratio, and application method from the plant protection pesticide database), application timing (combining safety intervals and seasonal factors), cultivation management suggestions (extracting supporting measures such as fertilization, pruning, and irrigation from the cultivation technology database), and risk warning (indicating the risk of disease spread and the control window). The final output is a structured .md format prescription report with fields including diagnosis, confidence, treatment, prevention, and warning.

[0049] If a user has questions about the diagnostic prescription report, they can ask follow-up questions via text input (e.g., "Is this medicine harmful to bees?"). The LLM (Local Level Model) will then perform supplementary searches and answers based on the current dialogue history and knowledge base, without requiring a re-execution of visual detection. The system records the tool call history and knowledge reference sources for each round of interaction, forming a traceable diagnostic log that facilitates subsequent model optimization and knowledge base updates. The feedback optimization mechanism includes: user annotations of the correctness of the diagnostic results (correct / incorrect / uncertain), effectiveness ratings of the prescription, and suggestions for error correction in the knowledge base. This feedback data is used to periodically fine-tune the LLM's inference strategy and update knowledge base entries.

[0050] Example 4 This embodiment proposes a plant disease diagnosis system based on a trained target detection model, the architecture of which is shown in the figure below. Figure 2 As shown, the system includes an interactive output module, a reasoning and decision-making module, a visual perception module, and a knowledge base module. The interactive output module receives user input information and distributes it to the corresponding module for processing based on the type of input information. It also outputs a disease diagnosis prescription report. The visual perception module receives user input images distributed by the interactive output module and locates and classifies the diseased areas in the images based on a trained target detection model, outputting the detection results. The reasoning and decision-making module receives user input text distributed by the interactive output module and the detection results output by the visual perception module. It then constructs diagnostic prompts based on the environmental context information. The large language model generates a disease diagnosis prescription report based on the prompts and calls the knowledge base module, which is then transmitted to the interactive output module. The knowledge base module stores plant protection knowledge and provides a standardized query interface for the reasoning and decision-making module to use.

[0051] In one specific embodiment, the system supports three multimodal input modes: plain text input (user only provides text description), plain image input (user only uploads image), and mixed text and image input (user provides both image and text description). For image input, the visual perception module is invoked for detection, and the result is used to infer the decision-making module; for plain text input, the inference decision-making module is directly input for inference.

[0052] It should be noted that the trained object detection model includes an improved D-FINE-L object detection model. The visual perception module uses the improved D-FINE-L object detection model to locate and classify diseased areas in the image. The improved D-FINE-L object detection model uses D-FINE-L as the basic object detection model. After the fine-grained distribution refinement (FDR) module output of D-FINE-L and before coordinate decoding, an edge correlation constraint distribution refinement module (EC-FDR) is inserted. The original FDR module independently optimizes the probability distribution of the four edges of the bounding box without considering the geometric constraints between edges, which can easily lead to aspect ratio distortion when leaf edges are blurred, leaves are distorted, occluded, or there is uneven lighting. Therefore, the EC-FDR module is designed to perform correlation constraint refinement on the probability distribution of the four edges (top, bottom, left, and right) of the bounding box. Simultaneously, an official weight progressive adaptation training strategy is designed to ensure compatibility between the newly added edge correlation constraint distribution refinement module and the official pre-trained weights of D-FINE-L.

[0053] The specific details of refining the probability distribution of the four edges of the bounding box by applying correlation constraints are as follows: Step 1: Add a learnable edge embedding matrix to the probability distribution of the four edges. Distinguish the distribution characteristics of the top, bottom, left, and right edges:

[0054] in The first output of the fine-grained distribution refinement module Layer edge probability distribution Indicates the decoder layer index. This refers to the number of intervals for refining the distribution (i.e., the number of discretized grids used in D-FINE-L to model edge offsets). For batch size, The number of queries; Step 2: Project the type-embedded distribution features onto a low-dimensional interaction space:

[0055] in For the input projection matrix, For input projection bias, Set as a dimension ; Step 3: Calculate the relevance weights between edges using a 4-head self-attention mechanism, and set the query matrix. Key matrix Value matrix ,in Given the linear projection matrices of the query, key, and value respectively, the attention output is:

[0056] The Softmax function normalizes along the key dimension (i.e., the edge dimension), and the left and right edges are separated by attention weights. Learning the width constraint correlation, the upper and lower edges are connected via... Learning highly constrained correlations, and The components are taken from the corresponding edge positions in the attention weight matrix; Step 4: Project the attention output back to the original distribution dimension:

[0057] in To output the projection matrix, For output projection bias, This is a geometric constraint correction amount; Step 5: Generate an adaptive gate value based on the splicing features of the original distribution. The expression for the adaptive gate value is:

[0058] in This represents the concatenation of the probability distributions of the four edges along the edge type dimension (i.e., dimension 3). splicing operation of dimensional tensors. For gated projection matrix, For gated bias. The sigmoid function is used. The resulting distribution after fusion is:

[0059] in This represents element-wise multiplication. This represents the edge probability distribution after gating fusion. Step 6: Renormalize the fused marginal probability distribution:

[0060] The Softmax function normalizes along the distribution dimension (i.e., dimension 4, size N+1) to ensure that the sum of the probability distributions for each edge is 1. The refined edge probability distribution output by the edge correlation constraint distribution refinement module is converted into edge offsets through the D-FINE original weight function (i.e., weighted summation of discretized grid positions), and then decoded into bounding box coordinates.

[0061] The official weighted progressive adaptation training strategy consists of three stages, the details of which are as follows: Initialization Phase: Load the COCO pre-trained D-FINE-L official weights into the original parameters of the backbone network, encoder, and decoder. Perform special initialization on the EC-FDR module—initialize the edge type embedding E to a small value close to zero, initialize the attention projection matrices WQ and WK to identity matrices, initialize WV and output projection Wout to zero matrices, and initialize the gating bias bg to a negative value so that the initial gating G≈0. This initialization makes the initial state of EC-FDR approximately an identity mapping, avoiding disruption of the already converged distribution predictions.

[0062] Warmup phase: Freeze the original network parameters and train only the edge correlation constraint distribution refinement module to adapt it to the edge distribution characteristics of the disease. In one specific embodiment, the learning rate is set to 10. 4. Train 10 to 20 times.

[0063] Joint fine-tuning phase: Unfreeze the original network parameters and adjust the model to convergence using the preset learning rate.

[0064] In one specific embodiment, all parameters are unfrozen, at 5×10 5. The learning rate is jointly fine-tuned for the entire model. The feature pyramid neck network, detection head, and post-processing module of the improved D-FINE-L model all adopt the original D-FINE-L structure without introducing any additional improvements. After training, the model is exported in ONNX format and packaged into an independent image processing API interface.

[0065] It should be noted that the knowledge base module is built based on a tree-structured directory knowledge base. The root directory of the knowledge base is organized according to the agricultural plant protection knowledge system, including a plant directory, a disease type database, a plant protection equipment and pesticide database, and a cultivation technology database. Based on the model context protocol, knowledge base operations are encapsulated into independent callable utility functions. Automatic registration and metadata management of these tools are achieved through Python decorators. Multiple callable utility functions form a toolchain, whose core tools include a directory exploration tool and a file reading tool.

[0066] In one specific embodiment, a tree-structured knowledge base using Markdown files is constructed. The root directory of the knowledge base is "kb", and its subdirectories are organized according to the agricultural plant protection knowledge system, including but not limited to: a plant catalog (e.g., plants / citrus), a disease type database (containing descriptions of symptoms, disease patterns, and identification points for various diseases), a plant protection pesticide and equipment database (containing information such as pesticide names, dilution ratios, and safety intervals), and a cultivation technique database (containing management techniques such as fertilization, pesticide application, and pruning). Each Markdown file uses structured headings and tables to organize its content, facilitating LLM parsing and extraction of key information.

[0067] The directory exploration tool (kb_explore_dir) is used to explore the directory structure of the knowledge base. This tool accepts a string parameter `dir_path`, representing the relative path of the directory to be explored; the default value is "." (representing the root directory of the knowledge base). The tool's execution flow includes: first, validating the input path through a secure path parsing module, including path traversal protection and directory existence verification; then, traversing all entries under the directory, filtering hidden files (files starting with "."), and categorizing subdirectories and Markdown files separately; finally, returning a structured directory list, including the current path, a list of subdirectories, and a list of Markdown files. This tool uses the Pydantic library to define a parameter model (ToolParamBase), which is automatically registered to the system toolchain via the `@tool` decorator, specifying the tool category as "knowledge_base" and the category description as "local Markdown knowledge base query tool".

[0068] The file reading tool (kb_read_file) is used to read the contents of a Markdown file at a specified path. This tool accepts a string parameter, `file_path`, representing the relative path of the Markdown file to be read. The tool's execution flow includes: first, validating the input path using a secure path parsing module, including path traversal protection, file existence verification, and file format verification (only .md format files are allowed); then, reading the entire file content in UTF-8 encoding and returning it. This tool also uses the Pydantic parameter model and the `@tool` decorator for automatic registration. The tool description explicitly states that "this tool can only be called after you have confirmed the file's existence through the exploration tool," guiding the LLM to follow the standardized "explore first, then read" calling order.

[0069] The specific content of the secure path resolution module is as follows: A unified secure path resolution function, `safe_resolve`, is designed as the underlying security infrastructure for all knowledge base tools. This module includes a three-layer security verification mechanism: The first layer is path normalization and traversal protection. The input path is normalized using `os.path.normpath` to eliminate path traversal elements such as "..", and prefix matching verification ensures that the resolved absolute path is strictly located in the knowledge base root directory, preventing arbitrary file access attacks. The second layer is existence verification, verifying the existence and type matching of the target path based on the request type (directory or file). The third layer is format verification, restricting file read requests to only allow `.md` format to prevent reading sensitive configuration files or executable files. This module returns a verification result in the form of a tuple (`is_safe`, `result`), where `is_safe` is a boolean value indicating whether the verification passed, and `result` is the resolved absolute path or an error message.

[0070] It should be noted that the reasoning decision module is based on a large language model for reasoning and uses multi-round tool calls of the large language model for iterative deep reasoning.

[0071] The specific content of the iterative depth-first inference is as follows: Step 1: Receive the detection results and inject them into the input sequence of the large language model. At the same time, concatenate the user's text input and environmental context information to construct complete diagnostic prompt words. In one specific embodiment, the detection results output by the improved D-FINE-L model are injected into the input sequence of the LLM in structured JSON format. The JSON fields include disease class, confidence level, and image resolution information. Simultaneously, the user's text description (e.g., "yellow spots appear on the leaves") and environmental context information (e.g., geographical location, temperature, humidity, season, etc.) are concatenated to construct a complete diagnostic Prompt. The Prompt template includes two parts: a System Prompt and a Tool Description. The System Prompt explicitly specifies the LLM's role as a citrus plant protection expert, while the Tool Description enumerates the knowledge base tool functions registered in S3 and their parameter specifications.

[0072] Step 2: The large language model performs reasoning based on diagnostic prompts. When the large language model determines that the existing information is sufficient for diagnosis, it outputs a diagnostic prescription report and proceeds to Step 4; when the large language model determines that supplementary plant protection knowledge is needed, it generates tool call instructions and proceeds to Step 3. In one specific implementation, when the LLM determines that supplementary plant protection knowledge is needed, it generates a structured tool call instruction, which includes the tool name and arguments. The tool call instruction follows a JSON Schema format for easy backend parsing and execution. The LLM then decides whether to invoke the tool based on the context information.

[0073] Step 3: Based on the tool call command, the corresponding knowledge base tool is invoked to execute an iterative deep-search strategy, including a directory exploration phase, a file reading phase, and a cross-validation phase. In the directory exploration phase, the `kb_explore_dir` tool is executed to explore layer by layer starting from the root directory of the knowledge base. The large language model determines the next exploration direction based on the returned directory structure until the leaf directory containing the target file is located. In the file reading phase, the `kb_read_file` tool is executed to read the entire contents of the target file and return it to the large language model context. In the cross-validation phase, the large language model determines whether the acquired plant protection knowledge is sufficient. If insufficient, a new tool call command is generated until a diagnostic prescription report is output. In one specific implementation, each tool execution result is fed back into the LLM context in Observation format, forming a multi-round interactive loop of "inference-call-observation-re-inference". Iteration termination conditions include: the LLM determines that the knowledge is sufficient, or two consecutive tool calls do not acquire new information. The diagnostic prescription report includes disease characterization (confirming the disease name and severity), etiological analysis (inferring the inducing factors based on environmental context), and differential diagnosis (excluding similar diseases). Prescription-level solution generation includes: pesticide recommendation (extracting pesticide name, dilution ratio, and application method from the plant protection pesticide database), application timing (combining safety intervals and seasonal factors), cultivation management suggestions (extracting supporting measures such as fertilization, pruning, and irrigation from the cultivation technology database), and risk warning (indicating the risk of disease spread and the control window). The final output is a structured .md format prescription report with fields including diagnosis, confidence, treatment, prevention, and warning.

[0074] Step 4: If the user has questions about the diagnostic prescription report, the user can ask follow-up questions through text input. The large language model will supplement the search and answer based on the current dialogue history and knowledge base without re-performing visual detection. It also records the tool call history and knowledge reference sources of each round of interaction, forming a traceable diagnostic log for large language model optimization and knowledge base updates.

[0075] In one specific embodiment, users can ask follow-up questions via text input (such as "Is this medicine harmful to bees?"). The LLM performs supplementary retrieval and answers based on the current dialogue history and knowledge base, without requiring re-performing visual detection. The system records the tool call history and knowledge reference sources for each round of interaction, forming a traceable diagnostic log, which facilitates subsequent model optimization and knowledge base updates. The feedback optimization mechanism includes: user annotation of the correctness of the diagnostic results (correct / incorrect / uncertain), effectiveness rating of the prescription, and suggestions for error correction in the knowledge base content. This feedback data is used to periodically fine-tune the LLM's inference strategy and update knowledge base entries.

[0076] Example 5 This embodiment provides a specific implementation of a knowledge base query toolchain based on the decorator pattern, as follows: Tool base class and decorator definition: Define ToolParamBase as the base class for all tool parameter models, inheriting from Pydantic's BaseModel. Define the @tool decorator, which accepts parameters such as category (tool category), category_desc (category description), name (tool name), description (tool function description), and param_model (parameter model), to achieve automatic tool registration and metadata generation.

[0077] Directory exploration tool implementation: Define the KBExploreDirParams parameter model, including the dir_path field (default value is "."). Implement the kb_explore_dir asynchronous function, which internally calls safe_resolve for path security verification, traverses the directory contents using os.listdir, filters hidden files, and outputs subdirectories and Markdown files separately. Register it using the @tool decorator, specifying the category as "knowledge_base" and the name as "kb_explore_dir".

[0078] The file reading tool is implemented as follows: A KBReadFileParams parameter model is defined, including a file_path field. The asynchronous function kb_read_file is implemented, internally calling safe_resolve to perform path safety checks (including file existence and .md format verification), reading the entire file content in UTF-8 encoding and returning it. It is registered using the @tool decorator, specifying the category as "knowledge_base" and the name as "kb_read_file," with the description explicitly requiring that "this tool should only be called after you have confirmed the file's existence through exploration tools."

[0079] The secure path resolution module is implemented as follows: A constant `KB_ROOT` is defined, pointing to the root directory of the knowledge base. Upon startup, the existence of this directory is checked. The `safe_resolve` function is implemented, accepting `relative_path` and `check_type` parameters. It concatenates and normalizes the path using `os.path.normpath` and `os.path.join`, prevents path traversal through prefix matching, and performs existence and format checks on directories or files based on `check_type`. It returns a tuple of `(is_safe, result)`.

[0080] Example 6 This embodiment provides an end-to-end deployment and interactive workflow for a plant disease diagnosis system: Backend service construction: The backend includes three core services: (1) Visual reasoning service, which loads the target detection model (improved D-FINE-L model) in Example 1, encapsulates it into an independent image processing API interface, and uses GPU acceleration to achieve high-concurrency reasoning; (2) Knowledge storage service, which maps the constructed tree-shaped directory knowledge base to the local file system of the server in the form of a read-only mounted volume to ensure absolute isolation and security of the path; (3) System scheduling service, which serves as the core control unit, has built-in LLMAPI client and toolchain executor, and is responsible for assembling the visual structured results and user text into a Prompt, scheduling the large model to perform multiple rounds of iterative reasoning, and managing the query history stack.

[0081] Front-end and back-end interface definitions: The back-end exposes a unified interface to the outside world. The core interfaces include the image upload interface, the text question interface, and the result query interface.

[0082] End-to-end interactive workflow: In actual field applications, the complete interaction chain after a fruit grower discovers abnormal citrus is as follows: The fruit grower opens a WeChat mini program, takes a picture of the diseased fruit, and the system automatically compresses the image to the specified resolution and uploads it along with the geographic coordinates; the cloud-based visual inference service receives the image, executes an improved D-FINE-L model inference, and outputs a structured result containing disease category, confidence level, and coordinates; the system scheduling service receives the visual result, constructs an input sequence based on environmental context information, and calls the cloud-based LLM; the LLM triggers kb_explore_dir to explore the knowledge base directory structure, locates relevant files, calls kb_read_file to read the content, cross-compares the plant protection pesticide and equipment library with the disease type library, and generates a JSON prescription containing the specific pesticide dilution factor; the mini program receives the JSON data, displays the diagnostic report on the screen, and completes the closed loop.

[0083] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A target detection model, characterized in that, The system includes a backbone network, a neck network, and a decoder. The backbone network extracts visual features from the input image. The neck network, connected to the backbone network, performs multi-scale fusion of the extracted visual features. The decoder, connected to the neck network, performs bounding box decoding on the multi-scale fused features. The decoder includes a fine-grained distribution refinement module and an edge correlation constraint refinement module. The fine-grained distribution refinement module outputs the corresponding edge probability distribution for the four edges of the bounding box. The edge correlation constraint refinement module is connected between the output of the fine-grained distribution refinement module and the bounding box coordinate decoding module, and performs correlation constraint refinement on the probability distribution of the four edges of the bounding box.

2. The target detection model according to claim 1, characterized in that, The specific details of refining the probability distribution of the four edges of the bounding box by applying correlation constraints are as follows: Step 1: Add a learnable edge embedding matrix to the probability distribution of the four edges. Distinguish the distribution characteristics of the top, bottom, left, and right edges: in The first output of the fine-grained distribution refinement module Layer edge probability distribution To refine the number of intervals in the distribution; Step 2: Project the type-embedded distribution features onto a low-dimensional interaction space: in For the input projection matrix, For input projection bias, For dimensions; Step 3: Calculate the relevance weights between edges using a self-attention mechanism, and set the query matrix. Key matrix Value matrix ,in Given the linear projection matrices of the query, key, and value respectively, the attention output is: in For batch size, the Softmax function normalizes along the key dimension, and the left and right edges are separated by attention weights. Learning the width constraint correlation, the upper and lower edges are connected via... Learning highly constrained correlations, and The components are taken from the corresponding edge positions in the attention weight matrix; Step 4: Project the attention output back to the original distribution dimension to obtain the geometric constraint correction: in To output the projection matrix, For output projection bias, This is a geometric constraint correction amount; Step 5: Generate an adaptive gating value based on the splicing features of the original distribution, and obtain the edge probability distribution after gating fusion based on the adaptive gating value; The expression for the adaptive gating value is: in This represents concatenating the probability distributions of the four edges along the edge type dimension. splicing operation of dimensional tensors. For gated projection matrix, For gated bias. Given a sigmoid function, the marginal probability distribution after gated fusion is as follows: in This represents element-wise multiplication. The edge probability distribution after gating fusion; Step 6: Renormalize the fused marginal probability distribution: The Softmax function normalizes along the distribution dimension. The refined edge probability distribution output by the edge correlation constraint distribution refinement module.

3. A training method for an object detection model, wherein the object detection model includes a backbone network, a neck network, and a decoder, and the decoder includes a fine-grained distribution refinement module and an edge correlation constraint distribution refinement module, characterized in that, The training method includes three stages: Initialization phase: Load the pre-trained official weights into the original network parameters of the backbone network, encoder and decoder, and initialize the parameters of the edge correlation constraint distribution refinement module to an approximately identity mapping state; Warmup phase: Freeze the original network parameters and train only the edge correlation constraint distribution refinement module to adapt it to the edge distribution characteristics. Joint fine-tuning phase: Unfreeze the original network parameters and adjust the model to convergence using the preset learning rate.

4. A method for diagnosing plant diseases based on a trained target detection model, characterized in that, Includes the following steps: S1: Receive user input information, including input text and input image; S2: Based on the trained target detection model, locate and classify the diseased areas in the input image, and output the detection results; S3: Construct diagnostic prompts based on input text, detection results, and environmental context information, and use a large language model to perform inference and generate responses based on the diagnostic prompts.

5. The plant disease diagnosis method based on a trained target detection model according to claim 4, characterized in that, The specific content of the response is as follows: When the large language model determines that it has sufficient plant protection knowledge to complete the diagnosis, it directly outputs a disease diagnosis prescription report. When the large language model determines that plant protection knowledge needs to be supplemented, the output tool call command calls the corresponding knowledge base tool. The knowledge base tool is then used to search the knowledge base based on an iterative deep-search strategy until the large language model determines that it has sufficient plant protection knowledge to complete the diagnosis, at which point a disease diagnosis prescription report is output.

6. The plant disease diagnosis method based on a trained target detection model according to claim 5, characterized in that, The iterative deep learning strategy includes a directory exploration phase, a file reading phase, and a cross-validation phase. The directory exploration phase uses a directory exploration tool to explore layer by layer from the root directory of the knowledge base. The large language model decides the next exploration direction based on the returned directory list until the leaf directory containing the target file is located. The file reading phase uses a file reading tool to read the entire content of the target file and returns it to the large language model context. The cross-validation phase uses the large language model to determine whether the acquired plant protection knowledge is sufficient. If insufficient, it generates new tool call instructions to iteratively execute the directory exploration phase and the file reading phase.

7. A plant disease diagnosis method based on a trained target detection model according to claim 5, characterized in that, The knowledge base is constructed as a tree-structured directory knowledge base. Under the root directory of the knowledge base, it is classified and organized according to the agricultural plant protection knowledge system, including a plant directory, a disease type database, a plant protection pesticide and equipment database, and a cultivation technology database.

8. A method for diagnosing plant diseases based on a trained target detection model according to claim 6, characterized in that, The execution flow of the directory exploration tool includes: first, validating the input path through secure path parsing, including path traversal protection and directory existence verification; then, traversing all entries under the directory, filtering hidden files, and classifying subdirectories and files of preset types respectively; finally, returning a structured directory list.

9. A method for diagnosing plant diseases based on a trained target detection model according to claim 6, characterized in that, The execution process of the file reading tool includes: first, verifying the legality of the input path through secure path parsing, including path traversal protection, file existence verification, and file format verification; then, reading the entire content of the file in a preset format encoding.

10. A plant disease diagnosis system based on a trained target detection model, characterized in that, The system includes an interactive output module, a reasoning and decision-making module, a visual perception module, and a knowledge base module. The interactive output module receives user input information and distributes it to the corresponding module for processing based on the type of input information. It also outputs a disease diagnosis prescription report. The visual perception module receives user input images distributed by the interactive output module and locates and classifies the diseased areas in the images based on a trained target detection model, outputting the detection results. The reasoning and decision-making module receives user input text distributed by the interactive output module and the detection results output by the visual perception module. It then constructs diagnostic prompts based on the environmental context information. A large language model generates a disease diagnosis prescription report based on the prompts and calls the knowledge base module, which is then transmitted to the interactive output module. The knowledge base module stores plant protection knowledge and provides a standardized query interface for the reasoning and decision-making module to use.

Images