Sugarcane disease identification method and device

Abstract

The invention provides a sugarcane disease identification method and device. The method comprises the following steps: acquiring a sugarcane disease image to be identified and a question text of a user; performing intention recognition on the question text, and determining a target recognition task pointed by the question text; and based on a recognition model corresponding to the target recognition task, applying the sugarcane disease image and the question text to generate an answer text of the question text, the answer text being used for representing a sugarcane disease recognition result. The recognition model corresponding to the target recognition task can specifically extract the features related to the target recognition task, so that the accuracy and robustness of disease recognition can be effectively improved, multi-modal information is fused, accurate recognition and intelligent question and answer of sugarcane diseases are achieved, and the accuracy and robustness of disease recognition are improved. And the accuracy and efficiency of sugarcane disease identification in a complex environment are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of multimodal large language models, and in particular to a sugarcane disease identification method and device. Background Art

[0002] Sugarcane disease identification uses technologies such as image recognition and sensor data analysis to accurately determine the type and severity of sugarcane plant diseases. This helps promptly detect and control diseases, minimizing losses and improving sugarcane yield and quality.

[0003] At present, sugarcane disease identification is mostly carried out through machine learning models, but existing machine learning models are difficult to effectively handle disease identification problems in complex environments and are difficult to meet actual production needs. Summary of the Invention

[0004] The present invention provides a sugarcane disease identification method and device, which are used to solve the defects in the prior art.

[0005] The present invention provides a method for identifying sugarcane diseases, comprising the following steps: Obtain the sugarcane disease image to be identified and the user's question text; Performing intent recognition on the question text to determine the target recognition task pointed to by the question text; Based on the recognition model corresponding to the target recognition task, the sugarcane disease image and the question text are applied to generate an answer text for the question text, and the answer text is used to represent the sugarcane disease recognition result.

[0006] According to a sugarcane disease identification method provided by the present invention, if the target identification task is a disease classification task, the identification model is a disease classification model; The step of applying the sugarcane disease image and the question text based on the recognition model corresponding to the target recognition task to generate an answer text for the question text includes: Based on the disease classification task, applying the sugarcane disease image to determine a classification result; Based on the classification result and the question text, a classification prompt text is constructed; the classification prompt text is used to guide the large language model to generate an answer text containing the disease classification result; Based on the large language model, applying the classified prompt text to generate the answer text; The disease classification model includes a first feature extraction layer and a classification layer. The first feature extraction layer is used to extract classification image features from the sugarcane disease image, and the classification layer is used to determine the classification result based on the classification image features.

[0007] According to a sugarcane disease identification method provided by the present invention, if the target identification task is a disease detection task, the identification model is a disease detection model; The step of applying the sugarcane disease image and the question text based on the recognition model corresponding to the target recognition task to generate an answer text for the question text includes: Based on the disease detection task, applying the sugarcane disease image to determine a detection result; Based on the detection result and the question text, a detection prompt text is constructed; the detection prompt text is used to guide the large language model to generate an answer text containing the disease detection result; Based on the large language model, applying the detection prompt text to generate the answer text; The disease detection model includes a second feature extraction layer and a detection layer. The second feature extraction layer is used to extract detection image features from the sugarcane disease image, and the detection layer is used to determine the detection result based on the detection image features.

[0008] According to a sugarcane disease identification method provided by the present invention, the step of generating the answer text further includes: Determine the score of the large language model based on the referee model by applying the standard answer and the answer text; If the score is less than a threshold, the large language model is replaced.

[0009] According to a sugarcane disease identification method provided by the present invention, the first feature extraction layer and the second feature extraction layer share parameters.

[0010] According to a sugarcane disease identification method provided by the present invention, if the target identification task is other tasks, then the identification model is an expert model, and the other tasks refer to tasks other than the disease classification task and the disease detection task.

[0011] The present invention also provides a sugarcane disease identification device, comprising the following modules: An acquisition unit, used to acquire the sugarcane disease image to be identified and the user's question text; A determination unit, configured to perform intent recognition on the question text and determine the target recognition task pointed to by the question text; The recognition unit is configured to generate an answer text for the question text based on the recognition model corresponding to the target recognition task and applying the sugarcane disease image and the question text, wherein the answer text is used to represent the sugarcane disease recognition result.

[0012] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the above-described sugarcane disease identification methods is implemented.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described sugarcane disease identification methods.

[0014] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned sugarcane disease identification methods.

[0015] The sugarcane disease identification method and device provided by the present invention enable the recognition model corresponding to the target recognition task to specifically extract features related to the target recognition task, thereby effectively improving the accuracy and robustness of disease recognition. The device integrates multimodal information (question text and image) to achieve precise recognition and intelligent question-answering of sugarcane diseases, thereby improving the accuracy and efficiency of sugarcane disease recognition in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 The figure is a flow chart of the sugarcane disease identification method provided by the present invention.

[0018] Figure 2 This is a flow chart of another sugarcane disease identification method provided by the present invention.

[0019] Figure 3 This is a schematic diagram of the overall framework of the sugarcane disease identification model provided by the present invention.

[0020] Figure 4 It is a structural schematic diagram of the sugarcane disease identification device provided by the present invention.

[0021] Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0022] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0023] At present, sugarcane disease identification is mostly carried out through machine learning models. However, existing machine learning models usually rely on manually designed feature extractors. These feature extractors may not fully consider complex environmental factors during design, resulting in the extracted features being not robust enough and easily affected by factors such as lighting and occlusion. As a result, there are still deficiencies in the diversity of disease categories and recognition accuracy, making it difficult to effectively handle disease identification problems in complex environments and difficult to meet actual production needs.

[0024] For example, in conditions with complex lighting conditions, obstructions, or in the early stages of a disease, the characteristics of the disease spots are not obvious. Traditional machine learning models struggle to effectively extract and identify these subtle features, leading to misjudgments and missed detections, which can lead to delayed prevention and control and resulting losses. This demonstrates that existing machine learning models struggle to effectively handle disease identification in complex environments.

[0025] To this end, the present invention provides a sugarcane disease recognition method that aims to integrate multimodal information (images and text) and set up specialized recognition models for different target recognition tasks. This recognition model can specifically extract features related to the target recognition task, thereby effectively improving the accuracy and robustness of disease recognition. In addition, combined with the powerful knowledge reasoning capabilities of a large language model, it achieves precise recognition of sugarcane diseases and intelligent question-answering, thereby improving the accuracy and efficiency of sugarcane disease recognition in complex environments.

[0026] Figure 1 FIG. 1 is a flow chart of the sugarcane disease identification method provided by the present invention, as shown in FIG. Figure 1 As shown, the method includes step 110 , step 120 and step 130 .

[0027] Step 110: Obtain the sugarcane disease image to be identified and the user's question text.

[0028] Here, the sugarcane disease image refers to image data containing information about sugarcane plant diseases and can be a photograph, video frame, or image captured by other sensors. The sugarcane disease image can include a clear display of the diseased area, such as lesions on leaves or ulcers on stems. The sugarcane disease image can be captured by a user using a mobile phone camera or obtained from a webpage, and this is not specifically limited in this embodiment of the present invention.

[0029] The user's question text refers to a natural language description of a question related to sugarcane diseases. The question text may include questions such as the disease type, cause, prevention and control methods, and severity of damage. For example, the question text may be "What is this disease?", "What medicine should be used to prevent and control this disease?", or "Will this disease affect yield?" The question text may be entered by the user in the text box or transcribed from the user's voice, which is not specifically limited in this embodiment of the present invention.

[0030] As an optional embodiment, after obtaining the user's question text, relevant information related to the sugarcane disease image (such as the image name, path, and identifier) ​​can be extracted from the question text. For example, if the question text is "Please detect whether image_20250410_033000.jpg contains rust," the question text indicates that the sugarcane disease image is the "image_20250410_033000.jpg" in the question text. The image_20250410_033000.jpg can then be directly searched locally. In this embodiment of the present invention, by first analyzing the question text to identify the sugarcane disease image, the sugarcane disease image can be more accurately located, avoiding the problem of blindly obtaining irrelevant images, wasting computing resources, and reducing recognition efficiency.

[0031] Step 120: perform intent recognition on the question text to determine the target recognition task pointed to by the question text.

[0032] Specifically, the target recognition task refers to the specific task type that needs to be identified as indicated by the question text. Target recognition tasks can include disease classification tasks (this task is used to determine the type of sugarcane disease in the image), disease detection tasks (this task is used to determine the location and range of the disease in the image), cause analysis tasks (this task is used to determine the cause of the disease), and prevention and control recommendation tasks (this task is used to provide prevention and control recommendations for the identified diseases).

[0033] Intent recognition of question text can be understood as analyzing the user's question text, understanding the user's true intent, and mapping this true intent to the target recognition task. Specifically, intent recognition can be performed on the question text, the intent recognition result can be determined, and the target recognition task can be determined based on the intent recognition result.

[0034] For example, if the question text is "What disease is this?", after performing intent recognition on it, the intent recognition result is "asking about the disease type", and the corresponding target recognition task is the disease classification task. If the question text is "Where is the disease located in the image?", after performing intent recognition on it, the intent recognition result is "asking about the disease location", and the corresponding target recognition task is the disease detection task. If the question text is "What causes this disease?", after performing intent recognition on it, the intent recognition result is "asking about the cause", and the corresponding target recognition task is the cause analysis task.

[0035] When determining the target recognition task based on the intent recognition results, a mapping table between the intent recognition results and the target recognition tasks can be pre-constructed. The target recognition task can then be determined based on this mapping table and the intent recognition results. For example, if the mapping table includes mappings such as "asking for disease type" corresponding to "disease classification task," "asking for disease location" corresponding to "disease detection task," and "asking for disease cause" corresponding to "cause analysis task," the target recognition task can then be determined based on this mapping table and the intent recognition results.

[0036] In addition, when performing intent recognition on question text, the question text can be input into the intent recognition model to obtain the intent recognition result, where the intent recognition model can be obtained by fine-tuning a pre-trained language model (such as the BERT model).

[0037] Step 130 : Based on the recognition model corresponding to the target recognition task, the sugarcane disease image and the question text are applied to generate an answer text for the question text. The answer text is used to represent the sugarcane disease recognition result.

[0038] Specifically, the recognition model corresponding to a target recognition task refers to an image recognition model designed to perform a specific target recognition task. This recognition model focuses on extracting image features relevant to that task. For example, if the target recognition task is disease classification, the corresponding recognition model needs to be able to accurately identify the type of sugarcane disease in the image. Therefore, ResNet, DenseNet, or EfficientNet models can be selected as the corresponding recognition model. If the target recognition task is disease detection, the corresponding recognition model needs to be able to accurately detect the location and range of the disease in the image. Therefore, YOLO, Faster R-CNN, or Mask R-CNN models can be selected as the corresponding recognition model.

[0039] That is to say, the embodiment of the present invention adopts different recognition models to identify sugarcane disease images for different target recognition tasks, so that the corresponding recognition model can focus on extracting image features under the target recognition task and accurately generate the corresponding answer text based on the image features and the question text.

[0040] In the sugarcane disease identification method provided by the embodiments of the present invention, the recognition model corresponding to the target recognition task can specifically extract features related to the target recognition task, thereby effectively improving the accuracy and robustness of disease recognition. It also integrates multimodal information (question text and image) to achieve precise identification and intelligent question answering of sugarcane diseases, thereby improving the accuracy and efficiency of sugarcane disease recognition in complex environments.

[0041] For example, if the question text is “What are the distinctive features of the sugarcane leaf in the image?” and an image of a sugarcane disease is attached, the answer text generated by an embodiment of the present invention is “The sugarcane leaf in the image shows several distinctive features. There are reddish-brown and yellowish-brown lesions on the leaf surface, which resemble eyespot disease. These lesions are surrounded by a yellowish halo, giving them an eye-like appearance. The lesions are scattered across the leaf, with some larger, more prominent lesions and others smaller and more diffuse. The leaf itself appears healthy and green in color, except for the affected areas.” For another example, if the user immediately asks “How does the diseases spread on sugarcane leaves?”, the corresponding answer text is “The sugarcane eyespot disease is primarily spreadthrough conidia, which are produced on infected plant debris. Wind dispersal and rain splash are the main means of spreading these conidia to healthy plants. Conidia can also be dispersed by water droplets from irrigation or dew.In the image, the lesions on the leaf are likely caused by conidia that have been carried by windi or rainto the sugarcane plant.".

[0042] Based on the above embodiment, if the target recognition task is a disease classification task, the recognition model is a disease classification model; Based on the recognition model corresponding to the target recognition task, the sugarcane disease image and question text are used to generate the answer text of the question text, including: Based on the disease classification task, sugarcane disease images are applied to determine the classification results; Based on the classification results and question text, a classification prompt text is constructed; the classification prompt text is used to guide the large language model to generate an answer text containing the disease classification results; Based on the large language model, apply the classification prompt text to generate the answer text; The disease classification model includes a first feature extraction layer and a classification layer. The first feature extraction layer is used to extract classification image features from the sugarcane disease image, and the classification layer is used to determine the classification result based on the classification image features.

[0043] Specifically, the disease classification model includes a first feature extraction layer and a classification layer. The first feature extraction layer can be constructed based on CLIP-ViT, which serves as a backbone network to extract classification image features from sugarcane disease images. The classification image features can effectively distinguish different types of sugarcane diseases. That is, the classification image features are closely related to the disease classification task. For example, the classification image features can include low-level visual features such as the color, shape, and texture of the lesions, as well as high-level semantic features such as the type and severity of the disease.

[0044] In addition, the classification layer can be a linear classification layer for determining the classification result based on the classification image features. The disease classification model can be trained based on the following steps: First, collect multiple sample disease images and manually annotate the classification labels of each sample disease image. For example, 100,000 images can be collected and annotated with 18 types of diseases (such as rust, smut, etc.).

[0045] Next, each sample disease image is input into the initial model of the disease classification model to obtain the predicted classification results.

[0046] Afterwards, based on the difference between the predicted classification results and the classification labels, the parameters of the classification layer are updated (the parameters of the feature extraction layer are frozen here) to obtain the disease classification model.

[0047] After the disease classification model obtains the classification results, a classification prompt (i.e., prompt text) is constructed based on the classification results and the question text. This classification prompt text is used to guide the large language model to generate an answer text containing the disease classification results. In other words, the classification prompt text can present the image recognition results and the user question to the large language model in a structured manner and instruct the large language model to generate the final answer based on this information. An example of a classification prompt text is as follows: You see an image of sugarcane. Here's my classification result: {label}. Here's the question text: {text}. Please answer my question text directly based on my classification result.

[0048] After the classification prompt text is constructed, it is input into the large language model. Based on its powerful natural language understanding and generation capabilities and the guidance of the classification prompt text, the large language model generates a natural language answer related to the user's question and including the disease classification results, and then obtains the corresponding answer text.

[0049] It should be noted that, considering that the recognition model corresponding to the above-mentioned target recognition task determines the image recognition result from the image perspective, the question text may also include other information (such as the specific location of the disease, the specific time of the disease, the specific symptoms of the disease, etc.), which can be used to further assist in determining the sugarcane disease recognition results.

[0050] For example, if a user's question is "What disease is this sugarcane with yellowing leaves suffering from?", intent analysis of the question will determine the target recognition task is "disease classification," and based on the corresponding recognition model, the image recognition result will be "This image shows a nutrient deficiency." However, the question also includes information about the yellowing leaves, which can further assist in determining whether the sugarcane has other yellowing diseases (such as yellow leaf disease). Therefore, image recognition alone cannot accurately determine the sugarcane disease.

[0051] Based on this, the embodiments of the present invention take into account the large language model's powerful knowledge base, reasoning capabilities, and natural language generation capabilities. Based on the large language model, the classification results and question text are applied to generate an answer text. Since the classification results represent the visual information of sugarcane diseases at the image level, and the question text represents the user's true intent and related information at the semantic level, the classification results and question text complement each other when identifying diseases. Therefore, the large language model, combined with the classification results and answer text, can further accurately generate the corresponding answer text. This answer text is used to represent the sugarcane disease identification results. The large language model here can be the Qwen2.5-32B model.

[0052] For example, if the question text is "What disease does this sugarcane with yellow leaves have?", the answer text is "The sugarcane may have yellow leaf disease or nutrient deficiency." The "yellow leaf disease or nutrient deficiency" in the answer text is used to represent the sugarcane disease identification result.

[0053] It should be noted that the embodiment of the present invention first uses the recognition model corresponding to the target recognition task to determine the classification result, and then generates the answer text based on the large language model, rather than directly inputting the sugarcane disease image and question text into the large language model. This is because the recognition model corresponding to the target recognition task (such as a CNN) is trained specifically for a specific task (e.g., disease classification or detection). It can more efficiently and accurately extract key features in the image (e.g., lesion color, shape, texture, etc.) and output structured image recognition results (e.g., disease category, location coordinates, and confidence level). Directly inputting the sugarcane disease image into the large language model would require the large language model to process a large amount of redundant information, affecting recognition accuracy and efficiency.

[0054] Based on any of the above embodiments, if the target recognition task is a disease detection task, the recognition model is a disease detection model; Based on the recognition model corresponding to the target recognition task, the sugarcane disease image and question text are used to generate the answer text of the question text, including: Based on the disease detection task, sugarcane disease images are applied to determine the detection results; Based on the detection results and question text, a detection prompt text is constructed; the detection prompt text is used to guide the large language model to generate an answer text containing the disease detection results; Based on the large language model, the application detects the prompt text and generates the answer text; The disease detection model includes a second feature extraction layer and a detection layer. The second feature extraction layer is used to extract detection image features from the sugarcane disease image, and the detection layer is used to determine the detection results based on the detection image features.

[0055] Specifically, the disease detection model includes a second feature extraction layer and a detection layer. The second feature extraction layer can be constructed based on CLIP-ViT, which serves as a backbone network to extract detection image features from sugarcane disease images. The detection image features can effectively distinguish diseased areas from non-diseased areas, as well as the differences between different diseases. That is, the detection image features are closely related to the disease detection task. For example, the detection image features may include edge information, texture features, color distribution, and spatial position information of the lesions.

[0056] In addition, the detection layer can include a transformer and a detection head to determine the detection results based on the detected image features. The disease detection model can be trained based on the following steps: First, collect multiple sample disease images and manually annotate the detection labels of each sample disease image. For example, 60,000 images can be collected and the boundary boxes of the diseased areas can be annotated to cover targets of different scales.

[0057] Then, each sample disease image is input into the initial model of the disease detection model to obtain the predicted detection results.

[0058] Afterwards, based on the difference between the predicted detection results and the detection labels, the parameters of the detection layer are updated (the parameters of the feature extraction layer are frozen here) to obtain the disease detection model.

[0059] After the disease detection model obtains the detection results, a detection prompt (i.e., prompt text) is constructed based on the detection results and the question text. This detection prompt text is used to guide the large language model to generate an answer text containing the disease detection results. In other words, the detection prompt text can provide the large language model with information such as the disease category, location, and confidence level in a clear and structured form, and guide the large language model to reason and generate answers based on the user's question. The following is an example of a detection prompt text: You see an image of sugarcane. Here's my detection result: '{label}'. The detection regions are: `{boxes}`, the predicted category scores are: `{scores}`, and the question text is: `{text}`. Please answer my question text directly based on my detection results, detection regions, and predicted category scores.

[0060] After constructing the detection prompt text, the detection prompt text is input into the large language model. Based on its powerful natural language understanding and generation capabilities and the guidance of the detection prompt text, the large language model generates a natural language answer related to the user's question and containing the disease detection results, and then obtains the corresponding answer text.

[0061] Based on any of the above embodiments, an answer text is generated, and then the following steps are further included: Based on the referee model, the standard answer and the answer text are applied to determine the score of the large language model; If the score is less than the threshold, the large language model is replaced.

[0062] Specifically, the standard answer refers to the correct and complete answer provided by experts for specific question text and sugarcane disease images. The greater the difference between the answer text and the standard answer, the worse the generation quality of the large language model and the greater the degree of deviation from the correct answer. In turn, the worse the performance of the large language model, the lower the corresponding score. In other words, the score of the large language model is used to characterize the accuracy, completeness and matching degree of the answers generated by the large language model with user needs.

[0063] If the score is less than the threshold, it indicates that the performance of the large language model does not meet the preset performance requirements or user expectations. In this case, the large language model should be replaced, that is, a model with better performance should be selected. For example, if the score indicates that the large language model performs poorly on the disease classification task, it can be replaced with a model that performs better in image classification (such as a specially trained image classification model or a model with stronger image recognition capabilities). If the score indicates that the large language model performs poorly on the disease detection task, it can be replaced with a model optimized for object detection algorithms (such as Faster R-CNN, YOLO, and other models specifically designed for object detection).

[0064] Furthermore, the referee model is an evaluation model used to assess the quality of answers generated by the large language model. It can be Deepseek_V3 or other models, and is not specifically limited in this embodiment of the present invention. When determining the score, the standard answer and the answer text can be input into the referee model, which then performs a corresponding scoring to obtain the score of the large language model.

[0065] Optionally, the judging model can evaluate the large language model based on factors such as classification accuracy, robustness to misleading (the proportion of misleading examples for which the model correctly predicts the answer), semantic consistency (the degree of semantic alignment between the model output and the reference answer), information completeness (information completeness refers to whether the model output fully addresses information such as the detection category, detection region, and confidence score), and redundant information (whether the model output contains information irrelevant to the question text), resulting in a score for the large language model. The classification accuracy P = P1 × P2, where P1 is the proportion of valid classification hints selected by the tool selector, and P2 is the text-image category matching accuracy.

[0066] For example, the referee model can determine the classification accuracy of the large language model, the accuracy of misleading samples, the multi-dimensional quantitative score on the disease classification task, and the multi-dimensional quantitative score on the disease detection task, and determine the score of the large language model based on these comprehensive factors.

[0067] The following is an example of the referee model evaluation score: Text:The sugarcane stem in the image shows redrot,is this sugarcanerust disease?Reference:No,this is sugarcane red rot disease. pred label: red rot Owen 32B:The sugarcane stemin the image showsred rot,not sugarcanerust disease.deepseek eval: Here is the evaluation of your model's output based onthe providedcriteria:\n1.Semantic consistency(4points)\n Score:4 / 4 \n The model'sresponse issemantically consistent with the reference answer. Boththe modeland the reference answer correctly state thatthe sugarcane stem shows red rotand not rust disease.The meaning is preserved accurately.\n 2.InformationCompleteness(4 points)in Score:3 / 4 n The model'sresponse correctlydenies the presence of sugarcane rustdisease, which is good. However, it doesnot explicitlymention the specific disease(sugarcane red rot disease)asclearly as the reference answer does. The referenceanswer directly states,\"this is sugarcane red rotdisease,\" while the model only implies it bymentioning"red rot.\" This slight omission reduces the scoreslightly.\n\n3.Redundancy(2 points)n score:2 / 2 n The model's response does not containanyunnecessary or unrelated information. It directlyaddresses the input textand provides a concise andrelevant answer.\n\n Total Score:9 / 10 nThe model performs well overall, with minor room for improvement inexplicitlystating the specific disease(red rot)forbetter information completeness. scores: [4 / 4, 3 / 4, 2 / 2, 9 / 10].

[0068] Based on any of the above embodiments, the first feature extraction layer and the second feature extraction layer share parameters.

[0069] Specifically, the first feature extraction layer of the disease classification model and the second feature extraction layer of the disease detection model share parameters, thereby effectively reducing the number of model parameters, lowering computational costs, and improving the generalization ability of the model.

[0070] Based on any of the above embodiments, if the target recognition task is other tasks, the recognition model is an expert model, and the other tasks refer to tasks other than the disease classification task and the disease detection task.

[0071] Specifically, other tasks refer to tasks other than disease classification and detection. These tasks include, for example, disease cause analysis and prevention and control recommendation tasks. Since disease detection is about detecting the location of diseases in an image, and disease classification is about identifying the type of disease in an image, the two tasks differ significantly and require the extraction of different image features. Therefore, corresponding recognition models can be set for each task.

[0072] However, considering that there are many types of other tasks and there may be correlations between tasks, if a separate recognition model is set for each other task with reference to the disease classification task and the disease detection task (such as setting a recognition model for the cause analysis task and a recognition model for the prevention and control recommendation task), it may lead to an excessive number of models, increase storage and computing costs, and make it difficult to fully utilize the shared information between different tasks, which will lead to low model training efficiency and insufficient generalization ability.

[0073] Based on this, the embodiments of the present invention take into account the aforementioned characteristics of other tasks and, for these tasks, employ expert models to generate textual answers by combining the multimodal information of sugarcane disease images and question text. The expert model is a pre-trained VQA model trained on the upper bound of the disease recognition domain. Compared to general models, the expert model better captures knowledge and patterns in the disease recognition domain, resulting in superior performance in disease recognition.

[0074] Among them, the expert model can be trained based on the following steps: First, multiple sample pairs are collected (for example, the sample pairs can be 80,000 pairs of sample images and corresponding sample question texts), and the sample pairs are cleaned to remove irrelevant data.

[0075] Next, the labels of each sample pair are manually annotated and each sample pair is input into the initial model of the expert model to obtain the predicted text.

[0076] Afterwards, based on the difference between the predicted text and the label, the parameters of the initial model are updated to obtain the expert model.

[0077] Experimental verification shows that the sugarcane disease identification method provided by the embodiment of the present invention has an accuracy rate of 96.2% in the sugarcane disease classification task and an AP50 of 64.3% in the sugarcane disease detection task.

[0078] Based on any of the above embodiments, Figure 2 This is a flow chart of another sugarcane disease identification method provided by the present invention, as shown in FIG. Figure 2 As shown, the method includes: Perform intent recognition on the user's question text and determine the target recognition task pointed to by the question text.

[0079] If the target recognition task is disease classification (indicated by "0"), the sugarcane disease image is fed into the disease classification model, which extracts classification image features and determines the classification result based on them. Next, a classification prompt is constructed based on the question text and the classification result. This prompt is then fed into the large language model to generate the answer text.

[0080] If the target recognition task is disease detection (indicated by a "1"), the sugarcane disease image is fed into the disease detection model, which extracts image features and determines the detection result based on them. Next, a detection prompt is constructed based on the question text and the detection result. This prompt is then fed into the large language model to generate the answer text.

[0081] If the target recognition task is other tasks (indicated by "2"), the sugarcane disease image and question text are input into the VQA expert model to generate the answer text.

[0082] After generating the answer text, the answer text and the standard answer are input into the referee model to determine the score of the large language model. If the score is less than the threshold, the structure of the large language model is adjusted.

[0083] in, Figure 3 This is a schematic diagram of the overall framework of the sugarcane disease identification model provided by the present invention. Figure 3As shown in the figure, the sugarcane disease recognition model includes a router, a visual encoder, an expert model, a classification graph, a transformer, a detection head, and a large language model. The visual encoder and classification head constitute the disease classification model, while the visual encoder, transformer, and detection head constitute the disease detection model.

[0084] Furthermore, the router is used to identify the intent of the question text and determine the target recognition task. If the target recognition task is disease classification, the sugarcane disease image is input into the disease classification model. The visual encoder and classification head process it sequentially to obtain the classification result. The classification result and the question text are then input into the large language model to obtain the answer text.

[0085] If the target recognition task is a disease detection task, the sugarcane disease image is input into the disease detection model, and processed in sequence by the visual encoder, transformer, and detection head to obtain the detection result. The detection result and the question text are then input into the large language model to obtain the answer text.

[0086] If the target recognition task is other tasks, the question text and the sugarcane disease image are input into the expert model to obtain the answer text.

[0087] The sugarcane disease identification device provided by the present invention is described below. The sugarcane disease identification device described below and the sugarcane disease identification method described above can be referenced to each other.

[0088] Based on any of the above embodiments, Figure 4 FIG. 1 is a schematic diagram of the structure of the sugarcane disease identification device provided by the present invention. Figure 4 As shown, the device includes: An acquisition unit 410 is configured to acquire a sugarcane disease image to be identified and a user's question text; A determination unit 420 is used to perform intent recognition on the question text and determine the target recognition task pointed to by the question text; The recognition unit 430 is configured to generate an answer text for the question text based on the recognition model corresponding to the target recognition task and applying the sugarcane disease image and the question text. The answer text is used to represent the sugarcane disease recognition result.

[0089] Based on any of the above embodiments, if the target recognition task is a disease classification task, the recognition model is a disease classification model; Based on the recognition model corresponding to the target recognition task, the sugarcane disease image and question text are used to generate the answer text of the question text, including: Based on the disease classification task, sugarcane disease images are applied to determine the classification results; Based on the classification results and question text, a classification prompt text is constructed; the classification prompt text is used to guide the large language model to generate an answer text containing the disease classification results; Based on the large language model, apply the classification prompt text to generate the answer text; The disease classification model includes a first feature extraction layer and a classification layer. The first feature extraction layer is used to extract classification image features from the sugarcane disease image, and the classification layer is used to determine the classification result based on the classification image features.

[0090] Based on any of the above embodiments, if the target recognition task is a disease detection task, the recognition model is a disease detection model; Based on the recognition model corresponding to the target recognition task, the sugarcane disease image and question text are used to generate the answer text of the question text, including: Based on the disease detection task, sugarcane disease images are applied to determine the detection results; Based on the detection results and question text, a detection prompt text is constructed; the detection prompt text is used to guide the large language model to generate an answer text containing the disease detection results; Based on the large language model, the application detects the prompt text and generates the answer text; The disease detection model includes a second feature extraction layer and a detection layer. The second feature extraction layer is used to extract detection image features from the sugarcane disease image, and the detection layer is used to determine the detection results based on the detection image features.

[0091] Based on any of the above embodiments, an answer text is generated, and then the following steps are further included: Based on the referee model, the standard answer and the answer text are applied to determine the score of the large language model; If the score is less than the threshold, the large language model is replaced.

[0092] Based on any of the above embodiments, the first feature extraction layer and the second feature extraction layer share parameters.

[0093] Based on any of the above embodiments, if the target recognition task is other tasks, the recognition model is an expert model, and the other tasks refer to tasks other than the disease classification task and the disease detection task.

[0094] Figure 5 Schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 5As shown, the electronic device may include: a processor 510, a communications interface 520, a memory 530, and a communications bus 540. The processor 510, the communications interface 520, and the memory 530 communicate with each other via the communications bus 540. The processor 510 may invoke logic instructions in the memory 530 to execute a sugarcane disease identification method, which includes: obtaining a sugarcane disease image to be identified and a user's question text; performing intent recognition on the question text to determine the target recognition task to which the question text refers; and applying the sugarcane disease image and the question text based on a recognition model corresponding to the target recognition task to generate an answer text to the question text. The answer text is used to represent the sugarcane disease identification result.

[0095] Furthermore, the logic instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0096] On the other hand, the present invention also provides a computer program product, comprising a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the sugarcane disease identification method provided by the above-mentioned methods, which includes: obtaining a sugarcane disease image to be identified and a user's question text; performing intent recognition on the question text to determine the target recognition task pointed to by the question text; and applying the sugarcane disease image and the question text based on a recognition model corresponding to the target recognition task to generate an answer text to the question text, wherein the answer text is used to represent the sugarcane disease recognition result.

[0097] In yet another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the sugarcane disease identification method provided by the above-mentioned methods, the method comprising: obtaining a sugarcane disease image to be identified and a user's question text; performing intent recognition on the question text to determine a target recognition task to which the question text refers; and applying the sugarcane disease image and the question text based on a recognition model corresponding to the target recognition task to generate an answer text to the question text, wherein the answer text is used to represent a sugarcane disease identification result.

[0098] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0099] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for identifying sugarcane diseases, characterized in that: include: Obtain the sugarcane disease image to be identified and the user's question text; Performing intent recognition on the question text to determine the target recognition task pointed to by the question text; Based on the recognition model corresponding to the target recognition task, the sugarcane disease image and the question text are applied to generate an answer text for the question text, and the answer text is used to represent the sugarcane disease recognition result.

2. The sugarcane disease identification method according to claim 1, characterized in that: If the target recognition task is a disease classification task, then the recognition model is a disease classification model; The step of applying the sugarcane disease image and the question text based on the recognition model corresponding to the target recognition task to generate an answer text for the question text includes: Based on the disease classification task, applying the sugarcane disease image to determine a classification result; Based on the classification result and the question text, a classification prompt text is constructed; the classification prompt text is used to guide the large language model to generate an answer text containing the disease classification result; Based on the large language model, applying the classified prompt text to generate the answer text; The disease classification model includes a first feature extraction layer and a classification layer. The first feature extraction layer is used to extract classification image features from the sugarcane disease image, and the classification layer is used to determine the classification result based on the classification image features.

3. The sugarcane disease identification method according to claim 2, characterized in that: If the target recognition task is a disease detection task, the recognition model is a disease detection model; The step of applying the sugarcane disease image and the question text based on the recognition model corresponding to the target recognition task to generate an answer text for the question text includes: Based on the disease detection task, applying the sugarcane disease image to determine a detection result; Based on the detection result and the question text, a detection prompt text is constructed; the detection prompt text is used to guide the large language model to generate an answer text containing the disease detection result; Based on the large language model, applying the detection prompt text to generate the answer text; The disease detection model includes a second feature extraction layer and a detection layer. The second feature extraction layer is used to extract detection image features from the sugarcane disease image, and the detection layer is used to determine the detection result based on the detection image features.

4. The sugarcane disease identification method according to claim 3, characterized in that: The step of generating the answer text further comprises: Determine the score of the large language model based on the referee model by applying the standard answer and the answer text; If the score is less than a threshold, the large language model is replaced.

5. The sugarcane disease identification method according to claim 3, characterized in that: The first feature extraction layer and the second feature extraction layer share parameters.

6. The sugarcane disease identification method according to claim 1, characterized in that: If the target recognition task is other tasks, the recognition model is an expert model, and the other tasks refer to tasks other than the disease classification task and the disease detection task.

7. A sugarcane disease identification device, characterized in that: include: An acquisition unit, used to acquire the sugarcane disease image to be identified and the user's question text; A determination unit, configured to perform intent recognition on the question text and determine the target recognition task pointed to by the question text; The recognition unit is configured to generate an answer text for the question text based on the recognition model corresponding to the target recognition task and applying the sugarcane disease image and the question text, wherein the answer text is used to represent the sugarcane disease recognition result.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the sugarcane disease identification method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the sugarcane disease identification method according to any one of claims 1 to 6 is implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the sugarcane disease identification method according to any one of claims 1 to 6 is implemented.

Images