Display method, display system and readable storage medium for plant disease diagnosis information

Abstract

The present invention provides a method, a display system and a readable storage medium for displaying plant disease diagnosis information. The method includes: acquiring a plant image; using a disease location detection model to identify and process the plant image to determine whether it has a suspected disease area; if the above identification result contains at least one suspected disease area, the suspected disease area is marked and displayed according to a first preset method; using a disease diagnosis model to identify and process the plant image to acquire species information and determine whether it has a suspected disease; if the above identification result contains at least one suspected disease, the suspected disease information is displayed according to a second preset method.

Description

Technical Field

[0001] The present invention relates to the technical field of object recognition, and in particular to a method for displaying plant disease diagnosis information, a display system and a readable storage medium. Background Art

[0002] Plants often encounter problems such as diseases and pests during their growth. Currently, professional managers are typically responsible for identifying and addressing these issues. However, if these conditions are not identified, they can have a serious negative impact on plant growth. Furthermore, existing solutions for displaying plant disease diagnostic information are not intuitive enough and require improvement. Summary of the Invention

[0003] One of the purposes of the present disclosure is to provide a method for displaying plant disease diagnosis information, the method comprising:

[0004] Acquire plant images;

[0005] Using a disease location detection model to identify and process the plant image to determine whether it has a suspected disease area;

[0006] If there is at least one suspected disease area in the above recognition result, the suspected disease area is marked and displayed according to the first preset method;

[0007] Using a disease diagnosis model to identify and process the plant image, obtain species information, and determine whether it has a suspected disease;

[0008] If there is at least one suspected disease in the above identification result, the suspected disease is displayed in a second preset manner.

[0009] In some embodiments, marking and displaying the suspected disease area includes: marking and displaying the suspected disease area using a marking frame.

[0010] In some embodiments, marking and displaying the suspected disease area in a first preset manner includes: obtaining the confidence of the marking box, and displaying at least a portion of the plant image centered around the marking box with the highest confidence.

[0011] In some embodiments, when there are more than one annotation boxes with the highest confidence, the annotation box with the original position that is more centered is selected as the target for center display.

[0012] In some embodiments, when there is a partial overlap between the annotation boxes, if the area of ​​the overlapping region exceeds 1 / 2 of any of the annotation boxes, only the annotation box with the highest confidence is retained.

[0013] In some embodiments, when the size of the annotation box is smaller than the minimum limit size, the size of the annotation box is set to the minimum limit size; when the size of the annotation box is larger than the maximum limit size, the size of the annotation box is set to the maximum limit size.

[0014] In some embodiments, displaying information about the suspected disease in accordance with a second preset manner includes: extracting diagnostic information of the suspected disease in a content management system and outputting the diagnostic information, wherein, for different plant images, when the determined suspected disease is the same, at least part of the diagnostic information changes with different plant images.

[0015] In some embodiments, the at least part of the diagnostic information includes a reference image, the reference image corresponds to the suspected disease, and the reference image is similar to the plant image.

[0016] In some embodiments, the number of the reference images is less than or equal to 3.

[0017] In some embodiments, extracting the diagnostic information of the suspected disease in the content management system and outputting the diagnostic information includes:

[0018] In the content management system, determining a corresponding candidate reference image library according to the suspected disease;

[0019] In the candidate reference image library, determining one or more reference images to be extracted and a priority corresponding to each of the one or more reference images based on a similarity with the plant image and / or a matching degree with the species information; and

[0020] The one or more reference images are outputted such that the one or more reference images are arranged in descending order of priority.

[0021] In some embodiments, in the candidate reference image library, determining one or more reference images to be extracted and a priority corresponding to each of the one or more reference images based on similarity with the plant image and / or matching degree with the species information includes:

[0022] According to the species information of the plant image, searching the candidate reference image library for a candidate reference image that matches the species information of the plant image and outputting and displaying the candidate reference image as a first reference atlas;

[0023] When the first reference atlas is not established, searching the candidate reference atlas library for a candidate reference atlas that matches the genus information of the plant image and outputting the candidate reference atlas for display as the second reference atlas;

[0024] When the first reference atlas and the second reference atlas are not established, searching the candidate reference atlas library for a candidate reference atlas that matches the family information of the plant image and outputting the candidate reference atlas as a third reference atlas for display;

[0025] When the first reference atlas, the second reference atlas, and the third reference atlas are all not established, a preset default atlas corresponding to the suspected disease is determined as a reference atlas.

[0026] In some embodiments, in a content management system, determining a corresponding candidate reference image library according to the suspected disease includes:

[0027] When the species information of the plant image includes a result with a confidence greater than a first preset value, the corresponding candidate reference library is determined based on the species information and its corresponding suspected disease; otherwise, the corresponding candidate reference library is determined based on the species information with a confidence ranking before the second preset value and its corresponding suspected disease.

[0028] In some embodiments, using a disease diagnosis model to identify and process the plant image, obtain species information, and determine whether it has a suspected disease includes:

[0029] Pre-identifying the plant image using a disease diagnosis model to obtain a disease pre-identification result;

[0030] If the confidence level of the disease pre-identification result is less than a third preset value, outputting an interactive question associated with the disease pre-identification result; and

[0031] The answers to the interactive questions are obtained, and the suspected disease result information of the plant image is obtained according to the answers.

[0032] In some embodiments, the interactive question includes at least two options, and the answer to the interactive question is selected from the at least two options.

[0033] In some embodiments, the interactive question includes at least two levels, and different selection branches in the upper level correspond to different branch questions in the lower level.

[0034] In some embodiments, each of the condition pre-identification results is associated with at least one of the interactive questions, or at least two of the condition pre-identification results are associated with at least one of the interactive questions.

[0035] In some embodiments, the method further comprises:

[0036] When there are multiple suspected disease areas, pre-identify the multiple suspected disease areas using the disease diagnosis model to obtain disease pre-identification results for the multiple suspected disease areas respectively;

[0037] When the multiple suspected disease areas have two or more disease pre-identification results, the marking and displaying the suspected disease areas according to the first preset method includes: marking information on the suspected disease areas of different disease pre-identification results respectively;

[0038] When any of the suspected disease areas or the information marked thereon is clicked, the suspected disease information is displayed in a second preset manner.

[0039] According to another aspect of the present disclosure, a readable storage medium is provided, on which a program is stored, wherein when the program is executed, the method for displaying plant disease diagnosis information as described above is implemented.

[0040] According to another aspect of the present disclosure, a system for displaying plant disease diagnosis information is proposed, which is characterized by comprising a processor and a memory, wherein a program is stored on the memory, and when the program is executed by the processor, the method for displaying plant disease diagnosis information as described above is implemented.

[0041] Other features and advantages of the present disclosure will become more apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0043] The present disclosure can be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

[0044] Figure 1 FIG2 is a flow chart of a method for displaying plant disease diagnosis information according to an embodiment of the present invention;

[0045] Figure 2 Shown is a schematic diagram of marking and displaying suspected disease areas in a plant image according to an embodiment of the present invention;

[0046] Figure 3 Shown is a schematic diagram of an interactive question according to an embodiment of the present invention;

[0047] Figure 4 FIG. 1 is a schematic diagram showing a suspected disease information display page including a reference diagram portion according to an embodiment of the present invention.

[0048] Note that in the embodiments described below, the same reference numerals are sometimes used in common across different drawings to denote the same parts or parts having the same functions, and their repeated descriptions are omitted. In some cases, similar reference numerals and letters are used to denote similar items, so once an item is defined in one drawing, it need not be further discussed in subsequent drawings.

[0049] For ease of understanding, the positions, sizes, and ranges of various structures shown in the drawings and the like may not represent actual positions, sizes, and ranges, etc. Therefore, the present disclosure is not limited to the positions, sizes, and ranges disclosed in the drawings and the like. DETAILED DESCRIPTION

[0050] Various exemplary embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. It should be noted that unless otherwise specifically stated, the relative arrangements of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure.

[0051] The following description of at least one exemplary embodiment is merely illustrative and is not intended to limit the present disclosure, its application, or use. In other words, the structures and methods herein are presented in an exemplary manner to illustrate various embodiments of the structures and methods of the present disclosure. However, those skilled in the art will appreciate that these are merely exemplary of the disclosure that may be implemented, and are not exhaustive. Furthermore, the drawings are not necessarily drawn to scale, and some features may be exaggerated to illustrate details of specific components.

[0052] Technologies, methods and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, such technologies, methods and equipment should be considered part of the authorization specification.

[0053] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0054] Figure 1 The flowchart of the method for displaying plant disease diagnosis information according to an embodiment of the present invention is shown. The method can be implemented in an application (app) installed on a smart terminal such as a mobile phone or tablet computer. Figure 1 As shown, the method may include:

[0055] Step S1: Acquire plant images;

[0056] Step S2: using a disease location detection model to identify and process the plant image to determine whether it has a suspected disease area;

[0057] Step S3: If there is at least one suspected disease area in the above recognition result, the suspected disease area is marked and displayed according to a first preset method;

[0058] Step S4: using a disease diagnosis model to identify and process the plant image, obtain species information, and determine whether it has a suspected disease;

[0059] Step S5: If there is at least one suspected disease in the above identification result, the suspected disease is displayed in a second preset manner.

[0060] The following describes in detail the steps of the method for displaying plant disease diagnosis information provided by this embodiment with reference to the accompanying drawings and several examples.

[0061] Step S1: Obtain plant images. In some examples, plant images uploaded by users can be directly obtained. In other examples, corresponding prompt information can be generated and output after receiving user instructions to prompt users to upload plant images. Furthermore, the prompt information can also include specific requirements for plant images, such as prompting users to upload images of the entire plant, partial images of plant stems, leaves, etc., or partial images of parts with obvious lesions. In this case, multiple plant images can also be pre-processed by marking, such as marking the entire plant image, partial plant image (including marking the part of the plant reflected by the plant image), etc., so as to better identify species or diagnose pests and diseases.

[0062] Step S2: Using a disease location detection model to identify and process the plant image to determine whether it has a suspected disease area. The disease location detection model can be a pre-trained disease location detection model, which can include a neural network model, specifically a convolutional neural network model or a residual network model.

[0063] The convolutional neural network model is a deep feedforward neural network that uses convolution kernels to scan plant images, extracting the features to be identified, and then identifying the features. Furthermore, during plant image recognition, the original plant image can be directly input into the convolutional neural network model without preprocessing. Compared to other recognition models, the convolutional neural network model has higher recognition accuracy and efficiency.

[0064] Compared to convolutional neural network models, residual network models incorporate an additional identity mapping layer, which prevents accuracy saturation or even decline with increasing network depth (the number of layers in the network). The identity mapping function of the identity mapping layer in the residual network model must satisfy the following requirement: the sum of the identity mapping function and the residual network model input equals the residual network model output. The introduction of the identity mapping allows the residual network model to more clearly detect changes in the output, significantly improving the accuracy and efficiency of identifying plant disease locations.

[0065] After the plant image is input into the disease location detection model for processing, all suspected disease areas existing in the plant image can be identified.

[0066] Step S3: If there is at least one suspected disease area in the above recognition result, the suspected disease area is marked and displayed according to a first preset method; in some embodiments, marking and displaying the suspected disease area includes: marking and displaying the suspected disease area with a marking frame. For example, the suspected disease area in the plant image is marked with a red frame. Figure 2 FIG. 1 is a schematic diagram showing an example of an embodiment of the present invention in which suspected disease areas of a plant image are marked and displayed. Figure 2 As shown, multiple suspected disease areas are marked on the plant image 100, including a marking frame 100a and a marking frame 100b. The use of red frames can intuitively and eye-catchingly mark the suspected disease areas, making it easier for users to quickly find and confirm the suspected disease areas on the plant image.

[0067] In some embodiments, marking and displaying the suspected disease area in a first preset manner includes: obtaining the confidence of the marking box, and displaying at least a portion of the plant image centered around the marking box with the highest confidence.

[0068] Regarding the confidence level, since the symptom location detection model is not 100% reliable in identifying suspected symptom areas, it has a certain possibility of error. Therefore, the probability that the suspected symptom area identification result obtained by the symptom location detection model is consistent with the corresponding real symptom area (that is, the degree of credibility of the suspected symptom area identification result being close to the real symptom area) is called the confidence level. It is easy to understand that if the confidence level is closer to 1, it means that the suspected symptom area identification result obtained by the symptom location detection model is closer to the corresponding real symptom area, and the identification result is more credible. If the confidence level is closer to 0, it means that the suspected symptom area identification result obtained by the symptom location detection model is less credible.

[0069] In some embodiments, the plant image displayed to the user with the suspected disease area marked will be centered with the annotation box with the highest confidence as the center. The centered display will center the annotation box with the highest confidence to the maximum position that can be centered. Because the picture has edges, other annotation boxes will also affect the final display size and range of the plant picture. The annotation box with the highest confidence is not necessarily exactly in the center of the plant picture. Therefore, the plant picture will be cropped and displayed, and other annotation boxes will not be moved outside the captured range, that is, the screenshot includes all annotation boxes.

[0070] In some embodiments, when there are more than one annotation boxes with the highest confidence, the annotation box with the original position more centered is selected as the target for center display, so that the plant image can be modified as little as possible.

[0071] In some embodiments, when there is partial overlap between the annotation boxes, if the overlapping area exceeds 1 / 2 of any of the annotation boxes, only the annotation box with the highest confidence is retained. In this case, the overlapping annotation boxes are likely to be marking the same suspected disease area, so only the annotation box with the highest confidence is retained.

[0072] In some embodiments, when the size of the annotation box is smaller than the minimum limit size, the size of the annotation box is set to the minimum limit size, and when the size of the annotation box is larger than the maximum limit size, the size of the annotation box is set to the maximum limit size. Specifically, when the size of the annotation box is smaller or larger than the limit size specified by vision, the width and height defined by vision shall prevail, that is, the range of a maximum and minimum annotation box is limited to avoid the situation where the annotation box is too large or too small, otherwise it loses its meaning and is not beautiful. For example, the minimum limit size is limited to 10x10px (PX is pixels), and the maximum limit size can be set to a limit of 5px from the edge of the picture (the maximum limit size can also be limited to a numerical value, such as 500X500px as a limit, or it can be combined with a limit of 5px from the edge of the picture on this basis). The above data is only used for illustration and is not limited to this range. Those skilled in the art can make data adjustments based on this.

[0073] Step S4: using the disease diagnosis model to identify and process the plant image, obtain species information and determine whether it has suspected diseases.

[0074] In some embodiments, a pre-trained species recognition model may be used to determine candidate species or species information based on plant images. The species recognition model may be a neural network model, specifically a convolutional neural network model or a residual network model.

[0075] The convolutional neural network model is a deep feedforward neural network that uses convolution kernels to scan plant images, extracting the features to be identified, and then identifying the features. Furthermore, during plant image recognition, the original plant image can be directly input into the convolutional neural network model without preprocessing. Compared to other recognition models, the convolutional neural network model has higher recognition accuracy and efficiency.

[0076] Compared to convolutional neural network models, residual network models incorporate an additional identity mapping layer, which prevents accuracy saturation or even decline with increasing network depth (the number of layers in the network). The identity mapping function of the identity mapping layer in the residual network model must satisfy the following requirement: the sum of the identity mapping function and the residual network model input equals the residual network model output. The introduction of the identity mapping makes changes in the residual network model's output more pronounced, significantly improving both the accuracy and efficiency of plant species recognition.

[0077] In some embodiments, training a species recognition model may include:

[0078] Acquire a first sample set having a first preset number of plant images labeled with species;

[0079] Determining a first proportion of plant images from the first sample set as a first training set;

[0080] training a species recognition model using the first training set; and

[0081] The training ends when the first training accuracy is greater than or equal to the first preset accuracy, and a trained species recognition model is obtained.

[0082] Specifically, the first sample set may include a large number of plant images, and each plant image is labeled with a corresponding species. The plant image is input into the species recognition model to generate an output species, and then, based on the comparison result between the output species and the labeled species, the relevant parameters in the species recognition model can be adjusted, that is, the species recognition model is trained until the first training accuracy of the species recognition model is greater than or equal to the first preset accuracy, thereby obtaining a trained species recognition model. Based on a plant image, the species recognition model can also output multiple candidate species, each of which can have its corresponding species confidence for further analysis and screening.

[0083] Furthermore, the trained species recognition model can be tested, which may include:

[0084] determining a second proportion of plant images from the first sample set as a first test set;

[0085] determining a first model accuracy of the trained species recognition model using the first test set; and

[0086] When the accuracy of the first model is less than the second preset accuracy, the first training set and / or the species recognition model is adjusted for retraining.

[0087] Generally, the plant images in the first test set and the first training set are not exactly the same, so the first test set can be used to test whether the species recognition model also has a good recognition effect on plant images outside the first training set. During the test, the first model accuracy of the species recognition model is calculated by comparing the output species and the labeled species generated based on the plant images in the first test set. In some examples, the calculation method of the first model accuracy can be the same as the calculation method of the first training accuracy. When the first model accuracy obtained by the test is less than the second preset accuracy, it indicates that the recognition effect of the species recognition model is not good enough, so the first training set can be adjusted. For example, the number of plant images labeled with species in the first training set can be increased, or the species recognition model itself can be adjusted, or both of the above can be adjusted, and then the species recognition model can be retrained to improve its recognition effect. In some embodiments, the second preset accuracy can be set to be equal to the first preset accuracy.

[0088] Similarly, a pre-trained disease diagnosis model can be used to identify candidate diseases or disease information based on plant images. It should be noted that the disease information can include candidate diseases or undetected candidate diseases. The disease diagnosis model can be a neural network model, specifically a convolutional neural network model or a residual network model.

[0089] In some embodiments, training a disease diagnosis model may include:

[0090] Acquire a second preset number of second sample sets of plant images annotated with disease information;

[0091] determining a third proportion of plant images from the second sample set as a second training set;

[0092] training a disease diagnosis model using the second training set; and

[0093] When the second training accuracy is greater than or equal to the third preset accuracy, the training is completed, and a trained disease diagnosis model is obtained.

[0094] Specifically, the second sample set may include a large number of plant images, and each plant image is annotated with disease information. The disease information may be, for example, the disease suffered by the plant in the plant image, or an undetected disease corresponding to a healthy plant. The plant images in the second sample set may be at least partially identical to the plant images in the first sample set. The plant image is input into the disease diagnosis model to generate output disease information, and then, based on the comparison result between the output disease information and the annotated disease information, the relevant parameters in the disease diagnosis model may be adjusted, that is, the disease diagnosis model is trained until the second training accuracy of the disease diagnosis model is greater than or equal to the third preset accuracy, thereby obtaining a trained disease diagnosis model. Based on a plant image, the disease diagnosis model can output multiple candidate disease information, each of which can have its corresponding diagnostic confidence for further analysis and screening. The diagnostic confidence refers to the probability that the disease information corresponding to the plant image is the candidate disease information.

[0095] Furthermore, the disease diagnosis model can also be tested, which may include:

[0096] determining a fourth proportion of plant images from the second sample set as a second test set;

[0097] Determining a second model accuracy of the trained disease diagnosis model using the second test set; and

[0098] When the second model accuracy is less than a fourth preset accuracy, the second training set and / or the disease diagnosis model is adjusted for retraining.

[0099] Generally speaking, the plant images in the second test set and the second training set are not exactly the same, so the second test set can be used to test whether the disease diagnosis model also has a good diagnostic effect on plant images outside the second training set. During the test, the second model accuracy of the disease diagnosis model is calculated by comparing the output disease information and the labeled disease information generated based on the plant images in the second test set. In some examples, the calculation method of the second model accuracy can be the same as the calculation method of the second training accuracy. When the second model accuracy obtained by the test is less than the fourth preset accuracy, it indicates that the diagnostic effect of the disease diagnosis model is not good enough, so the second training set can be adjusted. For example, the number of plant images labeled with disease information in the second training set can be increased, or the disease diagnosis model itself can be adjusted, or both of the above can be adjusted, and then the disease diagnosis model can be retrained to improve its diagnostic effect. In some embodiments, the fourth preset accuracy can be set to be equal to the third preset accuracy.

[0100] Of course, in some embodiments, the identification and diagnosis of species and diseases can also be achieved by the same pre-trained model, that is, the model can integrate the functions of the above-mentioned species identification model and disease diagnosis model, directly obtain species information and determine whether it has suspected diseases.

[0101] In some embodiments, the plant image is identified and processed using a disease diagnosis model to obtain species information and determine whether it has a suspected disease, including: pre-identifying the plant image using a disease diagnosis model to obtain a disease pre-identification result; if the confidence level of the disease pre-identification result is less than a third preset value, outputting an interactive question associated with the disease pre-identification result; and obtaining an answer to the interactive question, and obtaining suspected disease result information of the plant image based on the answer.

[0102] Regarding confidence, since the disease diagnosis model is not 100% reliable in identifying diseases and has a certain possibility of error, the probability that the disease pre-identification result obtained by the disease diagnosis model matches the corresponding actual disease (that is, the degree of confidence that the disease pre-identification result is close to the actual disease) is called confidence. It is easy to understand that the closer the confidence is to 1, the closer the disease pre-identification result obtained by the disease diagnosis model is to the corresponding actual disease, and the more reliable the identification result is. The closer the confidence is to 0, the less reliable the disease pre-identification result obtained by the disease diagnosis model is.

[0103] Optionally, taking into account the recognition accuracy and ease of operation of the plant disease and insect pest diagnosis method, the first preset value can be set and adjusted according to the actual situation of different application scenarios. In an exemplary embodiment, the first preset value can be set to 0.9. If the confidence of the symptom pre-identification result is not less than the first preset value (such as ≥0.9), it can be considered that the authenticity of the recognition result is relatively high, and there is no need to assist in confirmation through interactive questions, but the symptom result can be obtained directly based on the symptom pre-identification result. If the confidence of the symptom pre-identification result is less than the first preset value, that is, the confidence is lower than the first preset value (such as <0.9), interactive questions are used to assist in confirmation to improve the diagnostic accuracy of plant diseases.

[0104] Regarding obtaining answers to interactive questions, in some examples, information obtained through various methods such as user touch, click, input, or voice recording can be obtained as feedback to the interactive questions. After obtaining the user's answer to the interactive question, such as after selecting an option, the disease results of the plant image can be obtained and diagnostic information can be output, such as jumping to the diagnosis results page.

[0105] In some embodiments, for a plant image to be diagnosed, after the disease diagnosis model performs pre-identification, only one disease pre-identification result may be obtained, that is, for a plant image to be diagnosed, after the disease diagnosis model performs pre-identification, only one disease pre-identification result may be obtained, or after obtaining two or more disease pre-identification results, after being screened out by other preset conditions (such as the third preset condition, see the following description for details), only a unique disease pre-identification result remains, then the interactive question may be asked for this unique disease pre-identification result. After obtaining the user's answer to the interactive question, the disease pre-identification result may be further determined as the disease result, or if the answer to the interactive question does not match the disease, the disease pre-identification result may not be output, and the disease pre-identification result may be further discarded, and it is determined that the plant is free of diseases and pests.

[0106] In some embodiments, for a plant image to be diagnosed, after the disease diagnosis model performs pre-identification, at least two disease pre-identification results can be obtained, or after obtaining multiple disease pre-identification results, at least two disease pre-identification results remain after being screened out by other preset conditions (such as the third preset condition, see the following description for details).

[0107] Regarding the method of outputting interactive questions, in some examples, a prompt may be popped up on the diagnostic page, such as a display or voice prompt, etc. Those skilled in the art may set and adjust it according to actual conditions.

[0108] Figure 3 Shown is a schematic diagram of an interactive problem according to an embodiment of the present invention, such as Figure 3 As shown, the following is an example to illustrate the display method of plant disease diagnosis information provided by this embodiment:

[0109] First, the user uploads an image of a plant to be diagnosed, which can be taken by the user. The disease diagnosis model pre-identifies the plant image and obtains two disease pre-identification results: disease A (e.g., yellowing leaves due to water shortage) and disease B (e.g., yellowing leaves due to overwatering). The confidence levels of both disease pre-identification results are less than a first preset value (0.9), thereby triggering the output of interactive questions 200 associated with the disease pre-identification results:

[0110] “Is the soil dry?

[0111] Stick your finger into the soil up to your first knuckle and feel if there's moisture present."

[0112] For preference, please refer to Figure 3 The interactive question includes at least two branches, and the answer to the interactive question is selected from the at least two branches. For the above example, two branches may be provided: 1-Yes (marked 200a) and 2-No (marked 200b), and the user will select an answer from these two branches. Corresponding to the answers to these two branches, if 1-Yes is selected, Symptom A (yellowing leaves due to lack of water) is confirmed as the symptom result; if 2-No is selected, Symptom B (yellowing leaves due to overwatering) is confirmed as the symptom result.

[0113] Preferably, the interactive questions may include at least two levels, and the different selection branches of the previous level correspond to different branch questions of the next level. In some embodiments, the interactive questions may include not only one level, but at least two levels. For example, if answer A is selected in the selection branch of the previous level, then the branch questions of the next level based on A will be further popped up, such as the branch question selection branches of A1 and A2. If the user selects answer B in the selection branch of the previous level, then the branch questions of the next level based on B will be further popped up, such as the branch question selection branches of B1 and B2. This can provide further auxiliary judgment information to further narrow the scope of the diagnosis results.

[0114] Preferably, each of the pre-identification results for a condition is associated with at least one interactive question, or at least two of the pre-identification results for a condition are associated with at least one interactive question. In some embodiments, each pre-identification result for a condition can be associated with one, two, or more interactive questions, and the interactive questions can have one, two, or more levels, with one, two, or more branch questions associated with one or more of the levels. For example, a pre-identification result for a condition is associated with two interactive questions: 1. Is the condition overwatered? 2. Is the condition underexposed?

[0115] In other embodiments, at least two pre-identification results for a condition are associated with at least one interactive question. This may be the case where at least two pre-identification results for a condition are associated with the same interactive question, or at least two pre-identification results for a condition are associated with two or more interactive questions, and this embodiment is not limited thereto. For example, two pre-identification results for a condition are associated with the same interactive question: "Is there too much watering?"; or two pre-identification results for a condition are associated with two interactive questions: "Is there too much watering?" and "Is there too little light?"

[0116] Step S5: If there is at least one suspected disease in the above identification result, the suspected disease is displayed in a second preset manner.

[0117] In some embodiments, displaying information about the suspected disease in accordance with a second preset manner includes: extracting diagnostic information of the suspected disease in a content management system and outputting the diagnostic information, wherein, for different plant images, when the determined suspected disease is the same, at least part of the diagnostic information changes with different plant images.

[0118] Wherein, content management system (CMS) can be a kind of software system between the system or process of WEB front-end and back-end. Content management system can be used to submit, modify, publish etc. to the data in for example text file, picture, database, form etc. Content management system can also provide content crawling tool, automatically crawls the content such as text file, HTML webpage, Web service, database etc. from third party, and puts it into the corresponding content library of this content management system itself after analysis and processing. Content management system can also assist WEB front-end to provide content to user in personalized way, that is, provide personalized portal framework, so that content is better pushed to user based on WEB technology. In the content management system in the embodiment of the present disclosure, descriptive content to plant and its disease can be stored, and these descriptive content can be text or picture, for example, can comprise various fields, articles etc., thereby make user can obtain introduction about plant and its disease in the diagnostic information extracted and output from content management system, for example interesting story, the purpose of plant, maintenance method and description of disease etc.

[0119] Each species information item can be matched to a species name (UID1) to distinguish different species. Similarly, each symptom information item can be matched to a symptom name (UID2 or ComnonName) to distinguish different symptom information. When extracting relevant diagnostic information from a content management system, retrieval can be performed based on UID1 and UID2. When a large amount of data is pre-stored in a content management system, most diagnostic scenarios can be covered, providing users with appropriate diagnostic information.

[0120] Based on the content management system, the relevant information of multiple species can be output to users in the form of one card for each species. Users can switch the display of each species and its related information by sliding cards on the interactive interface.

[0121] In some embodiments, for different plant images, even if the determined identification information remains the same, at least some of the diagnostic information can be modified for each plant image. This allows for more flexible output, helping to ensure that the output diagnostic information matches the user input, thereby improving the user experience and reducing confusion caused by input-output mismatches.

[0122] In some embodiments, the diagnostic data may include diagnostic summary data and / or diagnostic detailed data. Different fields may be set in the diagnostic summary data and the diagnostic detailed data to store the data extracted from the content management system in the corresponding fields. The step of outputting diagnostic information may include: in the content management system, according to the determined disease results, extracting corresponding data according to the preset output fields to generate diagnostic information, and outputting the diagnostic information. The preset output fields can be set by the user through the interactive interface according to his own needs, or the preset output fields can also be a relatively fixed number of fields. In the content management system, the corresponding diagnostic data extracted according to the determined identification information can be filled in the corresponding template with a preset output format to form diagnostic information.

[0123] Preferably, if complete data cannot be extracted according to the preset output fields in the content management system, the diagnostic information can be generated by searching the corresponding documents.

[0124] In some embodiments, the diagnosis summary data may include at least one of a symptom name corresponding to a symptom name field in a preset output field and a diagnosis summary corresponding to a diagnosis summary field in a preset output field. The diagnosis detail data may include at least one of a symptom analysis corresponding to a symptom analysis field in a preset output field, a solution corresponding to a solution field in a preset output field, and a preventive measure corresponding to a preventive measure field in a preset output field. By storing the symptom analysis, solution, and preventive measures in different fields, diagnostic information can be conveniently generated based on a content management system, which is also convenient for users to view.

[0125] In some embodiments, the at least part of the diagnostic information includes a reference image, the reference image corresponds to the suspected disease, and the reference image is similar to the plant image.

[0126] Figure 4 The diagram shows a suspected disease information display page including a reference diagram according to an embodiment of the present invention. Figure 4 As shown, at least part of the diagnostic information that can be adaptively changed following the plant image may include a reference image 300 (located at Figure 4The reference image at least corresponds to the symptom information and is similar to the plant image. In this way, the output diagnostic information is no longer fixed. Instead, the relevant explanatory images in the output diagnostic information can be replaced based on the plant image input by the user, making these explanatory images more similar to the plant image taken by the user. This will prevent the user from feeling that the image in the output diagnostic information is too different from the plant image they took, thus avoiding causing trouble for the user and improving the user experience.

[0127] In some embodiments, extracting the diagnostic information of the suspected disease in the content management system and outputting the diagnostic information includes:

[0128] In the content management system, determining a corresponding candidate reference image library according to the suspected disease;

[0129] In the candidate reference image library, determining one or more reference images to be extracted and a priority corresponding to each of the one or more reference images based on a similarity with the plant image and / or a matching degree with the species information; and

[0130] The one or more reference images are outputted such that the one or more reference images are arranged in descending order of priority.

[0131] The reference images can be pre-set in a candidate reference image library of a content management system. In the content management system, the corresponding candidate reference image library is determined according to the disease results; in the candidate reference image library, based on the similarity with the plant image and / or the matching degree with the species information, one or more reference images to be extracted and the priority corresponding to each reference image in the one or more reference images are determined; and then the one or more reference images are output, so that the one or more reference images are arranged in order from high to low priority.

[0132] Among them, each reference image in the content management system can be marked with the UID1 of the corresponding species information (UID1 may include species, variants, varieties, genera, families, etc.) and the UID2 of the disease results. Based on UID1 and UID2, the reference images can be classified, screened, etc. For example, according to UID2, one or more reference images corresponding to each disease result can be organized into a candidate reference image library corresponding to the corresponding disease results. When selecting the required reference image in the reference image library, the type of plant corresponding to the reference image can be determined according to the UID1 marked by each reference image. By displaying the reference image, users can better identify the diseases of plants, especially when the plant image taken by the user is unclear or the shooting position is not good.

[0133] Generally, reference images with higher similarity to plant images and higher matching degree to species information will have higher priority. Reference images with higher priority may be displayed first or arranged at the front of the displayed reference images for easier viewing by the user.

[0134] In some embodiments, in the candidate reference image library, determining one or more reference images to be extracted and a priority corresponding to each of the one or more reference images based on similarity with the plant image and / or matching degree with the species information includes:

[0135] According to the species information of the plant image, the candidate reference image that matches the species information of the plant image is searched in the candidate reference image library and output and displayed as the first reference atlas; when the first reference atlas is not established, the candidate reference image that matches the genus information of the plant image is searched in the candidate reference image library and output and displayed as the second reference atlas; when the first reference atlas and the second reference atlas are not established, the candidate reference image that matches the family information of the plant image is searched in the candidate reference image library and output and displayed as the third reference atlas; when the first reference atlas, the second reference atlas and the third reference atlas are not established, the preset default image corresponding to the suspected disease is determined as the reference image.

[0136] The reference image is selected based on the plant UID obtained after species identification from the plant image. The species UID is searched in the candidate reference image library corresponding to the suspected condition. If no species match is found, the genus information is matched. If no genus match is found, the family information is matched. If no family match is found, the default image from the condition article is directly displayed. The details page defaults to a maximum of three reference images. When there are two or three reference images, the page is adjusted to a horizontal scrolling style. When there is only one reference image, only one reference image is displayed.

[0137] In some embodiments, in a content management system, determining a corresponding candidate reference image library according to the suspected disease includes:

[0138] When the species information of the plant image includes a result with a confidence level greater than a first preset value, a corresponding candidate reference library is determined based on the species information and its corresponding suspected disease; otherwise, a corresponding candidate reference library is determined based on the species information with a confidence level ranked second above the preset value and its corresponding suspected disease. For example, when the species information of the plant image is identified with a confidence level of 0.9 or higher, the candidate reference library corresponding to the suspected disease of the species is output as the basis for the similarity map; otherwise, the candidate reference libraries corresponding to the suspected diseases of the species ranked in the top three in confidence are used as the basis for the similarity map. This can narrow the number of reference images for the current disease, thereby speeding up the matching process.

[0139] Typically, the display ratio of the images in the displayed diagnostic information is between 3:2 and 1:1, which provides a good display effect. However, the ratio of the reference images selected from the candidate reference image library may not be suitable for the above display ratio. Typically, such images can be stretched or cropped to adapt to the display ratio. However, considering that stretching the reference images may cause the features of certain diseases to be distorted, which is not conducive to the user's good identification of the disease, in an exemplary embodiment, the reference images can be processed by cropping.

[0140] The edge area of ​​the original image forming the reference image is cropped so that the ratio of the reference image obtained after cropping is consistent with the preset display ratio, and the image features corresponding to the disease results in the reference image are located in the middle area of ​​the reference image.

[0141] Specifically, based on the region recognition model, when selecting reference image materials for the content management system, images whose image features corresponding to the disease results are located in the edge area can be removed or ignored, and these images will not be included in the content management system. Alternatively, when storing the images in the content management system, they can be processed, such as cropped. Alternatively, after determining the reference image to be output based on the plant image, the reference image selected for the content management system can be processed, such as cropped, before being output. Of course, in some other embodiments, the positions of the disease features in the original image forming the reference image can also be determined in advance, and these positions can be avoided during the cropping process.

[0142] In some embodiments, the method further includes: when there are multiple suspected disease areas, using a disease diagnosis model to pre-identify the multiple suspected disease areas, and obtaining disease pre-identification results for the multiple suspected disease areas respectively; when the multiple suspected disease areas have two or more disease pre-identification results, marking and displaying the suspected disease areas according to a first preset method includes: separately marking information for the suspected disease areas with different disease pre-identification results; when any one of the suspected disease areas or its marked information is clicked, displaying information of the suspected disease according to a second preset method.

[0143] In some embodiments, after obtaining multiple suspected disease areas on the user's plant image, the multiple suspected disease areas of the user's plant image can be pre-identified according to the disease diagnosis model. When it is found that the multiple suspected disease areas on the user's plant image are diagnosed with multiple different diseases, a front page can be provided to display the disease mark information. The red boxes of different diseases can be marked with different disease names or corresponding marking symbols for distinction, informing the user that there are multiple different suspected diseases on the plant image they took. The user clicks the corresponding red box or mark to jump to the detailed information display page of the corresponding disease for viewing.

[0144] This embodiment also provides a readable storage medium having a program stored thereon, which, when executed, implements the above-described method for displaying plant disease diagnosis information. Furthermore, this embodiment also provides a system for displaying plant disease diagnosis information, comprising a processor and a memory, wherein the memory has a program stored thereon, which, when executed by the processor, implements the above-described method for displaying plant disease diagnosis information.

[0145] Although some specific embodiments of the present disclosure have been described in detail by way of examples, it should be understood by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present disclosure. The various embodiments disclosed herein may be combined in any manner without departing from the spirit and scope of the present disclosure. It should also be understood by those skilled in the art that various modifications may be made to the embodiments without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.

Claims

1. A method for displaying plant disease diagnosis information, characterized in that: The method comprises: Acquire plant images; Using a disease location detection model to identify and process the plant image to determine whether it has a suspected disease area; If there is at least one suspected disease area in the above recognition result, the suspected disease area is marked and displayed according to the first preset method; Using a disease diagnosis model to identify and process the plant image, obtain species information, and determine whether it has a suspected disease; If there is at least one suspected disease in the above identification result, the suspected disease is displayed according to the second preset method; When there are multiple suspected disease areas, pre-identify the multiple suspected disease areas using the disease diagnosis model to obtain disease pre-identification results for the multiple suspected disease areas respectively; When the multiple suspected disease areas have two or more disease pre-identification results, the marking and displaying the suspected disease areas according to the first preset method includes: marking information on the suspected disease areas of different disease pre-identification results respectively; When any of the suspected disease areas or the information marked thereon is clicked, the suspected disease information is displayed in a second preset manner.

2. The method for displaying plant disease diagnosis information according to claim 1, wherein: Marking and displaying the suspected disease area includes: marking and displaying the suspected disease area using a marking frame.

3. The method for displaying plant disease diagnosis information according to claim 2, wherein: Marking and displaying the suspected disease area in a first preset manner includes: obtaining the confidence of the marking frame, and displaying at least a portion of the plant image centered around the marking frame with the highest confidence.

4. The method for displaying plant disease diagnosis information according to claim 3, wherein: When there are more than one annotation boxes with the highest confidence, the annotation box with a more central original position is selected as the target to be displayed in the center.

5. The method for displaying plant disease diagnosis information according to claim 3, wherein: When there is partial overlap between the annotation boxes, if the overlapping area exceeds 1 / 2 of any annotation box, only the annotation box with the highest confidence is retained.

6. The method for displaying plant disease diagnosis information according to claim 3, wherein: When the size of the annotation box is smaller than the minimum limit size, the size of the annotation box is set to the minimum limit size; when the size of the annotation box is larger than the maximum limit size, the size of the annotation box is set to the maximum limit size.

7. The method for displaying plant disease diagnosis information according to claim 1, wherein: Displaying information about the suspected disease in accordance with a second preset manner includes: extracting diagnostic information of the suspected disease in a content management system and outputting the diagnostic information, wherein, for different plant images, when the determined suspected disease is the same, at least part of the diagnostic information changes with different plant images.

8. The method for displaying plant disease diagnosis information according to claim 7, wherein: At least part of the diagnosis information includes a reference image, the reference image corresponds to the suspected disease, and the reference image is similar to the plant image.

9. The method for displaying plant disease diagnosis information according to claim 8, characterized in that: The number of the reference images is less than or equal to 3.

10. The method for displaying plant disease diagnosis information according to claim 8, wherein: Extracting the diagnostic information of the suspected disease in the content management system and outputting the diagnostic information includes: In the content management system, determining a corresponding candidate reference image library according to the suspected disease; In the candidate reference image library, determining one or more reference images to be extracted and a priority corresponding to each of the one or more reference images based on a similarity with the plant image and / or a matching degree with the species information; and The one or more reference images are outputted such that the one or more reference images are arranged in descending order of priority.

11. The method for displaying plant disease diagnosis information according to claim 10, wherein: In the candidate reference image library, determining one or more reference images to be extracted and a priority corresponding to each of the one or more reference images based on a similarity with the plant image and / or a matching degree with the species information includes: According to the species information of the plant image, searching the candidate reference image library for a candidate reference image that matches the species information of the plant image and outputting and displaying the candidate reference image as a first reference atlas; When the first reference atlas is not established, searching the candidate reference atlas library for a candidate reference atlas that matches the genus information of the plant image and outputting the candidate reference atlas for display as the second reference atlas; When the first reference atlas and the second reference atlas are not established, searching the candidate reference atlas library for a candidate reference atlas that matches the family information of the plant image and outputting the candidate reference atlas as a third reference atlas for display; When the first reference atlas, the second reference atlas, and the third reference atlas are all not established, a preset default atlas corresponding to the suspected disease is determined as a reference atlas.

12. The method for displaying plant disease diagnosis information according to claim 10, wherein: In the content management system, determining the corresponding candidate reference image library according to the suspected disease includes: When the species information of the plant image includes a result with a confidence greater than a first preset value, the corresponding candidate reference library is determined based on the species information and its corresponding suspected disease; otherwise, the corresponding candidate reference library is determined based on the species information with a confidence ranking before the second preset value and its corresponding suspected disease.

13. The method for displaying plant disease diagnosis information according to claim 1, wherein: Using the disease diagnosis model to identify and process the plant image, obtain species information, and determine whether it has a suspected disease includes: Pre-identifying the plant image using a disease diagnosis model to obtain a disease pre-identification result; If the confidence level of the disease pre-identification result is less than a third preset value, outputting an interactive question associated with the disease pre-identification result; and The answers to the interactive questions are obtained, and the suspected disease result information of the plant image is obtained according to the answers.

14. The method for displaying plant disease diagnosis information according to claim 13, wherein: The interactive question includes at least two options, and the answer to the interactive question is selected from the at least two options.

15. The method for displaying plant disease diagnosis information according to claim 14, wherein: The interactive problem includes at least two levels, and different selection branches in the upper level correspond to different branch problems in the lower level.

16. The method for displaying plant disease diagnosis information according to claim 13, wherein: Each of the disease pre-identification results is associated with at least one of the interactive questions, or at least two of the disease pre-identification results are associated with at least one of the interactive questions.

17. A readable storage medium having a program stored thereon, characterized in that: When the program is executed, the method for displaying plant disease diagnostic information according to any one of claims 1 to 16 is implemented.

18. A display system for plant disease diagnosis information, characterized in that: The device comprises a processor and a memory, wherein a program is stored in the memory, and when the program is executed by the processor, the method for displaying plant disease diagnosis information according to any one of claims 1 to 16 is implemented.

Images