Method, device and equipment for diagnosing agricultural diseases and insect pests

Abstract

The invention provides an agricultural pest diagnosis method, device and equipment, and relates to the technical field of agriculture, and the method comprises the steps: obtaining multi-modal data of crops; according to the multi-modal data, determining disease and pest occurrence parts and disease and pest symptom information of the crops; according to the disease and pest occurrence part and disease and pest symptom information of the crop and a preset agricultural disease and pest fault tree, obtaining a disease and pest diagnosis result of the crop; the agricultural pest fault tree comprises a top event, a middle event and a bottom event; wherein the top event comprises a pest and disease damage type; the intermediate event comprises a part where diseases and insect pests occur; the bottom events comprise symptoms of the plant diseases and insect pests at all parts. The method provided by the embodiment of the invention can efficiently and accurately diagnose agricultural diseases and insect pests.

Description

Technical Field

[0001] The present invention relates to the field of agricultural technology, and in particular to a method, device and equipment for diagnosing agricultural pests and diseases. Background Art

[0002] Agricultural pest and disease diagnosis plays a vital role in agricultural production. It is not only a key link to ensure the healthy growth of crops and improve agricultural production efficiency, but also related to ecological environment protection and agricultural product safety. Through timely and accurate pest and disease diagnosis, farmers can quickly take preventive measures to reduce the damage of pests and diseases to crops and improve the yield and quality of crops.

[0003] In the related technologies, intelligent diagnosis of pests and diseases is performed through images or text descriptions. Although it is convenient and fast, it has significant drawbacks. For example, image recognition technology is difficult to ensure high accuracy, and the subjectivity and ambiguity of text descriptions may cause confusion and misunderstanding in diagnosis. Therefore, how to diagnose agricultural pests and diseases efficiently and accurately is a problem that technicians in this field need to solve urgently. Summary of the invention

[0004] The present invention provides a method, device and equipment for diagnosing agricultural pests and diseases, which fully utilizes multiple modal data such as text, images, videos and audio to extract the pest and disease characteristics of diseased crops from multiple angles, and inputs the pest and disease occurrence sites and pest and disease symptom information of diseased crops into a preset agricultural pest and disease fault tree, thereby qualitatively analyzing the occurrence probabilities of bottom events, intermediate events and top events of the fault tree, and accurately diagnosing agricultural pests and diseases.

[0005] The invention provides a method for diagnosing agricultural pests and diseases, comprising the following steps.

[0006] Obtain multimodal data of crops; Determining the occurrence location of the crop pests and diseases and the information on the symptoms of the pests and diseases according to the multimodal data; The diagnosis result of the crop pests and diseases is obtained according to the occurrence site of the crop pests and diseases, the symptom information of the pests and diseases and the preset agricultural pests and diseases fault tree; the agricultural pests and diseases fault tree includes top events, intermediate events and bottom events; wherein the top event includes the pest and disease type; the intermediate event includes the occurrence site of the pests and diseases; and the bottom event includes the symptoms of the pests and diseases in various sites.

[0007] According to a method for diagnosing agricultural pests and diseases provided by the present invention, the step of obtaining multimodal data of crops includes: At least two of image data of crops, text data of crops, video data of crops, and audio data of crops are acquired.

[0008] According to a method for diagnosing agricultural pests and diseases provided by the present invention, determining the occurrence location and symptom information of the pests and diseases of the crops according to the multimodal data includes: The multimodal data are input into a preset agricultural diagnosis large language model to obtain agricultural pest and disease triplet information; the agricultural pest and disease triplet information includes crop type, pest and disease occurrence site and pest and disease symptom information; the agricultural diagnosis large language model is used to extract pest and disease symptom information of various parts of the crop from the multimodal data of the crop.

[0009] According to a method for diagnosing agricultural pests and diseases provided by the present invention, the method obtains the diagnosis result of the crop pests and diseases according to the occurrence location of the crop pests and diseases, the symptom information of the pests and diseases and the preset agricultural pests and diseases fault tree, including: Determine the probability of the bottom event in the agricultural pest fault tree occurring in the crop according to the location of the crop pest and disease occurrence and the pest and disease symptom information; Determine the probability of the intermediate event in the agricultural pests and diseases fault tree of the agricultural crops according to the probability of the bottom event in the agricultural pests and diseases fault tree of the agricultural crops; Determine the probability of the crop occurring the type of pests and diseases corresponding to the top event according to the probability of the intermediate events in the fault tree of the crop occurring the pests and diseases; The agricultural pests and diseases occurring in the crops are determined according to the probability of the pest and disease type corresponding to the top event occurring in the crops.

[0010] According to a method for diagnosing agricultural pests and diseases provided by the present invention, the probability of the bottom event in the agricultural pest and disease fault tree occurring in the crop is determined based on the occurrence location of the crop pest and disease and the symptom information of the crop pest and disease, including: The probability of the bottom event in the agricultural pest fault tree occurring in the crop is determined according to the occurrence location of the crop pest, the pest symptom information and the severity of the symptom.

[0011] According to a method for diagnosing agricultural pests and diseases provided by the present invention, the probability of the occurrence of the intermediate events in the agricultural pests and diseases fault tree of the crops is determined according to the probability of the occurrence of the bottom events in the agricultural pests and diseases fault tree of the crops, including:

[0012] in, It represents the probability of the bottom event in the fault tree of agricultural pests and diseases occurring in crops; Represents the probability of an intermediate event in the fault tree of agricultural pests and diseases occurring in crops.

[0013] According to a method for diagnosing agricultural pests and diseases provided by the present invention, the probability of the occurrence of an intermediate event in a fault tree of agricultural pests and diseases in the crop, and determining the probability of the occurrence of a pest and disease type corresponding to the top event in the crop, include:

[0014] in, It represents the probability of the intermediate event in the fault tree of agricultural pests and diseases occurring in crops; Represents the probability of the top event in the fault tree of agricultural pests and diseases occurring in crops.

[0015] The present invention also provides an agricultural pest diagnosis device, comprising the following modules: An acquisition module, used to acquire multimodal data of crops; A determination module, used to determine the occurrence location of the pests and diseases of the crops and information on the symptoms of the pests and diseases according to the multimodal data; The diagnosis module is used to obtain the diagnosis result of the pests and diseases of the crops according to the occurrence sites of the pests and diseases of the crops, the symptom information of the pests and diseases and the preset agricultural pests and diseases fault tree; the agricultural pests and diseases fault tree includes top events, intermediate events and bottom events; wherein the top event includes the pest and disease type; the intermediate event includes the occurrence site of the pest and disease; and the bottom event includes the symptoms of the pest and disease in various sites.

[0016] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the agricultural pest diagnosis methods described above is implemented.

[0017] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the agricultural pest diagnosis method as described in any one of the above is implemented.

[0018] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the agricultural pest diagnosis method as described above is implemented.

[0019] The agricultural pest and disease diagnosis method, device and equipment provided by the present invention break the mode of diagnosing pests and diseases through a single modality in the prior art, and make full use of multiple modal data such as text, images, videos and audio to extract the pest and disease characteristics of diseased crops from multiple angles, and input the pest and disease occurrence sites and pest and disease symptom information of diseased crops into a preset agricultural pest and disease fault tree, so as to qualitatively analyze the occurrence probabilities of bottom events, intermediate events and top events of the fault tree, and accurately diagnose agricultural pests and diseases. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 It is a schematic diagram of the flow chart of the agricultural pest diagnosis method provided by the present invention.

[0022] Figure 2 It is a structural schematic diagram of the agricultural pest diagnosis device provided by the present invention.

[0023] Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] Combine the following Figure 1-Figure 3 The invention describes the agricultural pest diagnosis method, device and equipment.

[0026] In order to facilitate a clearer understanding of the technical solutions of the embodiments of the present application, some technical contents related to the embodiments of the present application are first introduced.

[0027] Agricultural pest and disease diagnosis plays a vital role in agricultural production. It is not only a key link to ensure the healthy growth of crops and improve the efficiency of agricultural production, but also related to ecological environment protection and agricultural product safety. Through timely and accurate diagnosis of pests and diseases, farmers can quickly take preventive measures to reduce the harm of pests and diseases to crops and improve the yield and quality of crops. However, the diagnosis of agricultural pests and diseases is a very complicated task. Due to the different pathogenesis, onset time and cycle of crop pests and diseases, the symptoms of the same pest and disease may be different in different periods and environments, and there is a high degree of similarity between samples of different categories. Therefore, there are problems such as small inter-class differences and large intra-class differences in crop pest and disease identification. At present, most intelligent diagnosis of pests and diseases is identified through images or text descriptions. Although it is convenient and fast, it has significant disadvantages.

[0028] Regarding image recognition of pests and diseases, it is difficult to ensure high accuracy of image recognition technology because pests and diseases are of various types and in different forms, and are greatly affected by environmental factors. At the same time, image recognition algorithms have limited adaptability to complex environments, and their ability to identify pests and diseases in the early stages is also weak, which can easily lead to misjudgment. Regarding text descriptions of pests and diseases, it is difficult for text to fully and accurately convey the visual characteristics of pests and diseases, such as color, shape, and texture, which may lead to errors in identification and diagnosis. At the same time, the subjectivity and ambiguity of text descriptions may also cause confusion and misunderstanding in diagnosis.

[0029] Figure 1 FIG. 1 is one of the flow charts of the agricultural pest diagnosis method provided by the present invention, such as Figure 1 As shown, the method includes the following: Step 101: Acquire multimodal data of crops.

[0030] Specifically, in the embodiment of the present application, multimodal data of crops are first obtained. Optionally, the multimodal data of crops may include image data of crops, text data of crops, video data of crops, audio data of crops, etc., and then based on the rich and complete multimodal data of crops, the accuracy and reliability of pest and disease identification diagnosis can be effectively improved. That is, the present application breaks the mode of diagnosing pests and diseases through a single modality in the prior art, and makes full use of multiple modal data such as text, images, videos and audio to extract features from multiple angles, thereby effectively improving the accuracy of the diagnosis results of crop pests and diseases.

[0031] Step 102: Determine the occurrence location of crop diseases and insect pests and their symptom information based on the multimodal data.

[0032] Specifically, after obtaining the multimodal data of crops, the implementation of this application can accurately determine the location of crop pests and diseases and the information on the symptoms of pests and diseases based on the multimodal data, thereby providing accurate data for the diagnosis of pests and diseases, and further improving the accuracy and reliability of pest and disease identification and diagnosis. Optionally, the location of crop pests and diseases includes roots, stems, leaves, flowers, fruits, plants and other parts. Pest and disease symptom information includes rotting, discoloration and deformity.

[0033] Step 103, obtaining crop pest and disease diagnosis results according to the occurrence sites of crop pests and diseases, pest and disease symptom information and a preset agricultural pest and disease fault tree; the agricultural pest and disease fault tree includes top events, intermediate events and bottom events; wherein the top event includes the pest and disease type; the intermediate event includes the occurrence site of the pest and disease; and the bottom event includes the symptoms of the pest and disease at each site.

[0034] Specifically, after determining the location of crop pests and diseases and the information of their symptoms, the embodiment of the present application further obtains the diagnosis results of crop pests and diseases based on the location of crop pests and diseases, the information of their symptoms and the preset agricultural pests and diseases fault tree; wherein the agricultural pests and diseases fault tree includes top events, intermediate events and bottom events; top events include pest and disease types; intermediate events include the location of pests and diseases; bottom events include the symptoms of pests and diseases at various locations. That is, in the embodiment of the present application, by analyzing the crop pest and disease information and symptoms input by the user from multiple angles, the location of crop pests and diseases and the information of their symptoms are input into the preset agricultural pest and disease fault tree, thereby qualitatively analyzing the occurrence probabilities of the bottom events, intermediate events and top events of the fault tree, and accurately diagnosing agricultural pests and diseases.

[0035] Exemplarily, the construction process of the preset agricultural pest and disease fault tree in the embodiment of the present application is as follows: (1) The agricultural pest and disease fault tree constructed in the embodiments of the present application involves organizing different agricultural events into a hierarchical tree structure in an orderly manner through logical symbols. In this structure, agricultural pests and diseases are set as the top event of the fault tree, which represents the core of the entire problem. The specific locations where pests and diseases occur are defined as intermediate events, which are key nodes for further subdividing the top events. The typical symptoms of pests and diseases constitute the bottom events of the fault tree, which are the direct basis for identifying and diagnosing pests and diseases. When constructing this fault tree, the logical symbol connecting the upper and lower events is an "OR gate", which means that the occurrence of any bottom event (typical symptoms) may lead to the occurrence of an intermediate event (location of pests and diseases), which in turn may lead to the occurrence of a top event (agricultural pests and diseases).

[0036] (2) Determine the top event of the agricultural pest and disease fault tree: The pest and disease diagnosis is used as the top event of the fault tree. In the embodiment of the present application, the top event code is defined as T, and the event name is agricultural pest and disease.

[0037] (3) Determine the intermediate events of the agricultural pest and disease fault tree: Analyze the top event, find out the main symptoms of the pest and disease, and use it as the next level intermediate event of the fault tree. At the same time, perform a qualitative analysis of the fault tree. In this invention, roots, stems, leaves, flowers, fruits, and plants are defined as intermediate events, and their codes are as follows: , , , , , .

[0038] (4) Determine the bottom events of the agricultural pest and disease fault tree: For each intermediate event, analyze it again to find out the necessary and sufficient direct events for its occurrence, and use these basic events as the bottom events of the fault tree. The bottom events in the embodiment of the present application are different symptoms of roots, stems, leaves, flowers, fruits, and plants. The bottom events corresponding to the roots include root cortex rot, root tumor, root decay, etc. The defined codes are , , …, ; The bottom events corresponding to the stem include stem necrosis, stem cracks, stem erosions, stem tumor-like growths, etc., and the defined codes are , , …, ; The bottom events corresponding to leaves include leaf discoloration, leaf spots, leaf rot, leaf deformity, etc., and the defined codes are , , …, ; The bottom events corresponding to flowers include flower discoloration, flower decay, flower deformity, etc., and the defined codes are , , …, The bottom events corresponding to fruits include fruit discoloration, fruit rot, fruit spots, fruit deformity, etc. The defined codes are , , …, ; The bottom events corresponding to the plants include plant discoloration, plant wilting, plant deformity, etc., and the defined codes are , , …, .

[0039] (5) Qualitative analysis of the pest and disease fault tree: Calculate all the fault factors that may cause the top event to occur, and find all the minimum cut sets of the fault tree. Since the upper and lower events of the fault tree are in an "or" logical relationship, this application uses Boolean algebra expressions to express them with logical addition. The expression for solving the top event is as follows: + + + + +

[0040] Among them, each intermediate event is affected by the corresponding bottom event, and the bottom events are in a logical relationship of "or" with each other, so the expression for solving the intermediate event is still expressed by Boolean algebra expression using logical addition, which are as follows: + +…+

[0041] + +…+

[0042] + +…+

[0043] = + +…+

[0044] = + +…+

[0045] = + +…+

[0046] From the formula, we can see that the fault tree of agricultural pests and diseases is composed of A minimal cut set, , define each bottom event as , .

[0047] The method of the above embodiment breaks the mode of diagnosing pests and diseases through a single modality in the prior art, and makes full use of multiple modal data such as text, images, videos and audio to extract the pest and disease characteristics of diseased crops from multiple angles, and inputs the pest and disease occurrence sites and pest and disease symptom information of diseased crops into a preset agricultural pest and disease fault tree, thereby qualitatively analyzing the occurrence probabilities of bottom events, intermediate events and top events of the fault tree, and accurately diagnosing agricultural pests and diseases.

[0048] In one embodiment, obtaining multimodal data of crops includes: At least two of image data of crops, text data of crops, video data of crops, and audio data of crops are acquired.

[0049] Specifically, in the embodiments of the present application, multimodal data of crops are obtained, so that the accuracy and reliability of pest and disease identification and diagnosis can be effectively improved based on the rich and complete multimodal data of crops.

[0050] For example, the specific process of obtaining multimodal data of crops is as follows: (1) Dataset preprocessing.

[0051] (a) Image dataset.

[0052] Image collection: Agricultural pest and disease images are collected through various methods such as field photography and web crawlers.

[0053] Data cleaning: Set the threshold for the imaging of pest and disease image features, and remove images with pixel values lower than the threshold. Screen qualified images through machine learning methods, delete images with low pixel values and unclear pest and disease features that are difficult to distinguish, improve data quality, and enhance the accuracy of the present invention in identifying agricultural pests and diseases.

[0054] Data augmentation: Increase the diversity of the dataset by means of rotation, flipping, scaling, translation, etc., to improve the generalization ability of the model. Adjust attributes such as brightness, contrast, and saturation of the images to adapt to pest and disease images under different lighting conditions. Add interference factors such as noise and blur to enhance the robustness of the model.

[0055] Image annotation: Use rectangular boxes, polygons or other shapes to annotate the pest and disease areas in the images, providing the target position information required for model training. At the same time, classify and annotate pests and diseases for multi-classification tasks.

[0056] Normalization: Normalize the pixel values of the images to between 0 and 1 to reduce the differences caused by factors such as lighting and contrast between different images.

[0057] Resize the image: According to the requirements of the model input, use operations such as scaling and interpolation to resize the image to an appropriate size.

[0058] Color space conversion: Convert the images from the RGB color space to the HSV color space, which helps to extract specific features of the images, such as color and brightness.

[0059] Data splitting: Split the dataset into a training set, a validation set, and a test set. Among them, the training set is used to train the model, the validation set is used to adjust the model parameters and select the best model, and the test set is used to evaluate the performance of the model.

[0060] (b) Text dataset.

[0061] Data cleaning: Clean the input text data, removing noise, outliers, and invalid data. This may include removing HTML tags, special characters, punctuation marks, stop words (such as common but meaningless words like "de", "shi", etc.). For English text, convert the text to lowercase to reduce the lexical diversity.

[0062] Word segmentation: Most of the text of the present invention is sourced from network collection and actual user consultations. For Chinese text, perform word segmentation, splitting the continuous text into individual words or phrases, using Chinese word segmentation tools such as jieba to complete.

[0063] Constructing word vectors: In deep learning, text data needs to be converted into numerical form before it can be processed. The present invention uses a more commonly used method to map each word into a vector, i.e., a word vector, using pre-trained word vector models such as Word2Vec and GloVe. By mapping each word into a word vector, the text is represented as a matrix as the input of the convolutional neural network in deep learning.

[0064] Data conversion: Convert the text into word vectors or bag-of-words models and perform operations such as sequence padding and truncation to adapt to the input form of the network. This usually involves converting text data into fixed-length sequences so that they can be input into convolutional neural networks.

[0065] Data normalization: word vectors are normalized to unit length, or methods such as TF-IDF are used to extract the importance and weight of features, reduce the scale differences between different word vectors, and make the model easier to learn and understand.

[0066] (c) Video dataset: Video collection: Videos related to agricultural pests and diseases were collected through web crawling and filming.

[0067] Video reading: Use OpenCV library to read video files and use the read method of cv2.VideoCapture object to extract video frames.

[0068] Frame extraction: Extract video frames from the video at a certain frame rate (such as 5 or 10 frames per second). Save the extracted frames as image files (such as JPEG, PNG, etc.) for subsequent processing.

[0069] Data enhancement: Rotate, flip, crop, add noise, etc. Considering the particularity of agricultural pest videos, data enhancement may need to focus on some specific transformations, such as lighting changes, color changes, etc., to simulate various situations that may occur in the actual environment.

[0070] Image recognition and labeling: This step and subsequent steps of the image are the same as the image processing method in step 11. The identified pests and diseases are labeled for subsequent use in training the deep learning model. Finally, the data features are standardized.

[0071] (d) Audio Dataset: Data collection: Mainly collect data sets such as farmers' audio consultation information and agricultural audio answers from agricultural information websites.

[0072] Noise removal: Use noise estimation, spectrum subtraction, Wiener filtering and other methods to reduce or eliminate background noise.

[0073] Audio enhancement: First, the volume is standardized to ensure that the volume level of all audio samples is similar, which helps the model better handle audio at different volume levels. Then the frequency response of the audio is adjusted to make the audio more balanced in each frequency band, which helps improve the sound quality and speech recognition accuracy. Finally, if there are long periods of silence in the audio, they can be removed to reduce the amount of calculation and storage requirements.

[0074] Audio trimming or padding: For audio samples with inconsistent lengths, trim or pad them so that all samples have the same length and the length of the dataset is fixed. When trimming, select the key parts of the audio to keep and remove the silence at the beginning and end; when padding, use silence, repetitions, or other suitable audio segments to fill shorter audio samples.

[0075] Audio framing: Before speech recognition, the continuous audio signal is divided into shorter frames (for example, 20-30 milliseconds per frame). When framing, a sliding window method is used so that there is a certain overlap between adjacent frames (for example, 10 milliseconds) to capture the continuity and changes in the audio signal.

[0076] Pre-emphasis: Enhances the high-frequency components in the audio signal. Because the high-frequency components in the speech signal are easily attenuated during transmission, pre-emphasis helps to improve the sensitivity of the speech recognition model to high-frequency components.

[0077] Audio to text: Use audio to text (ASR, Automatic Speech Recognition) technology to convert audio data into text. This can be achieved by using existing ASR libraries or APIs, such as Google Cloud Speech-to-Text, IBM Watson Speech to Text, Amazon Transcribe, etc.

[0078] Text preprocessing: This step can be performed with reference to step 12. Perform operations such as text cleaning (removing punctuation, special characters, stop words, etc.), text segmentation (splitting text into words or tokens), and text encoding (converting text into a digital format that the model can understand) on the text data to build a dataset.

[0079] (2) Multimodal data fusion.

[0080] (a) Feature preprocessing: Perform Z-score standardization (also known as standard deviation standardization) on the above data, calculate the mean and standard deviation of each feature, and then replace the original feature with the value of (feature value - mean) / standard deviation. After this processing, the feature will have a distribution with a mean of 0 and a standard deviation of 1. Use One-Hot Encoding to convert the category into a binary vector and encode the feature. Use filtering, packaging, or embedding methods to find the optimal feature subset to eliminate features that are irrelevant or redundant to the annotation, reduce the number of features, improve model accuracy, and reduce running time.

[0081] (b) Determine the weighting strategy based on feature importance: Give different features different weights by analyzing the output feature importance scores of the machine learning model.

[0082] (c) Calculate feature weights: Calculate the weight of each feature based on the selected weighting strategy. This method uses a machine learning model-based approach to infer weights by training the model and observing how much the model depends on the features.

[0083] (d) Apply weights: Apply the calculated weights to the features by multiplying each feature value by its corresponding weight, thereby obtaining the fused multimodal data.

[0084] The method of the above embodiment, by integrating multimodal data such as images, sounds, and texts of crop pests and diseases, breaks the current diagnostic barrier of determining the names of pests and diseases based solely on a single description. In addition, based on the fused multimodal data, information from different sources can be comprehensively considered, effectively improving the comprehensiveness and accuracy of the diagnosis.

[0085] In one embodiment, determining the occurrence location and symptom information of crop pests and diseases based on multimodal data includes: The multimodal data are input into a preset agricultural diagnosis large language model to obtain agricultural pest and disease triplet information; the agricultural pest and disease triplet information includes crop type, pest and disease occurrence site and pest and disease symptom information; the agricultural diagnosis large language model is used to extract pest and disease symptom information of various parts of the crop from the multimodal data of the crop.

[0086] Specifically, the embodiment of the present application accurately obtains the crop type, the location of pests and diseases, and the symptom information of pests and diseases by inputting the multimodal data of crops into a preset large language model for agricultural diagnosis, thereby providing accurate data for the diagnosis of pests and diseases, and further improving the accuracy and reliability of pest and disease identification and diagnosis. Optionally, the locations of pests and diseases of crops include roots, stems, leaves, flowers, fruits, plants and other parts. Pest and disease symptom information includes rotting, discoloration and deformity. Optionally, the large language model for agricultural diagnosis is used to extract the symptom information of pests and diseases of various parts of crops from the multimodal data of crops; optionally, the large language model for agricultural diagnosis can be trained based on multimodal sample data, so that the trained large language model for agricultural diagnosis can accurately extract the symptom information of pests and diseases of various parts of crops.

[0087] For example, in this application, a large agricultural diagnosis language model is used for feature extraction. By inputting the fused multimodal features of crops into the large agricultural diagnosis language model, agricultural pest and disease triples (crop variety, location of pest and disease occurrence, and crop pest and disease symptoms) are output. Based on the agricultural pest and disease triples and the preset agricultural pest and disease fault tree, the agricultural pest and disease diagnosis can be accurately performed.

[0088] The method of the above embodiment, by inputting the fused multimodal features of crops into the large language model for agricultural diagnosis, can accurately obtain information on crop types, locations of pests and diseases, and symptoms of pests and diseases, thereby providing accurate data for the diagnosis of crop pests and diseases, and can effectively improve the accuracy and reliability of pest and disease identification and diagnosis.

[0089] In one embodiment, according to the occurrence location of crop pests and diseases, the symptom information of the pests and diseases, and the preset agricultural pests and diseases fault tree, the crop pests and diseases diagnosis results are obtained, including: According to the occurrence location and symptom information of crop pests and diseases, determine the probability of the bottom event in the fault tree of crop pests and diseases; According to the probability of the bottom event in the fault tree of agricultural pests and diseases of crops, the probability of the intermediate event in the fault tree of agricultural pests and diseases of crops is determined; According to the probability of the intermediate events in the fault tree of agricultural pests and diseases, the probability of the type of pests and diseases corresponding to the top event of crops is determined; The agricultural pests and diseases occurring in crops are determined based on the probability of the pest and disease type corresponding to the top event occurring in crops.

[0090] Specifically, in the embodiments of the present application, the probability of the bottom event in the fault tree of agricultural pests and diseases of crops is determined according to the occurrence site of pests and diseases and the symptom information of pests and diseases; the probability of the intermediate event in the fault tree of agricultural pests and diseases of crops is determined according to the probability of the bottom event in the fault tree of agricultural pests and diseases of crops; the probability of the type of pests and diseases corresponding to the top event of crops is determined according to the probability of the intermediate event in the fault tree of agricultural pests and diseases of crops; the agricultural pests and diseases occurring in crops are determined according to the probability of the type of pests and diseases corresponding to the top event of crops. That is, in the embodiments of the present application, the occurrence site of crop pests and diseases and the symptom information of pests and diseases in multimodal data are input into the preset agricultural pests and diseases fault tree, and the occurrence probabilities of the bottom event, intermediate event and top event of the fault tree are qualitatively analyzed, so that the agricultural pests and diseases occurring in crops can be accurately diagnosed, effectively improving the efficiency and accuracy of agricultural pest and disease diagnosis.

[0091] In one embodiment, the probability of the bottom event in the agricultural pest fault tree occurring in the crops is determined based on the occurrence location of the pests and diseases of the crops and the information on the symptoms of the pests and diseases, including: According to the occurrence location of crop pests and diseases, the information of pest and disease symptoms and the severity of the symptoms, the probability of the bottom event in the fault tree of agricultural pests and diseases is determined.

[0092] Specifically, in the process of determining the probability of the bottom event in the fault tree of agricultural pests and diseases in crops, the embodiment of the present application also takes into account the influence of the severity of the symptoms of agricultural pests and diseases in the multimodal data on the accuracy of the diagnosis result, thereby effectively improving the accuracy of the diagnosis of agricultural pests and diseases. Optionally, in the embodiment of the present application, according to the distribution area of ​​the symptoms of agricultural pests and diseases in various parts of the crops, the different symptoms of agricultural pests and diseases are divided into 5 levels, and the severity of agricultural pests and diseases is determined to be 0 if the symptoms do not exist, 0.25 if there are only a few symptoms, 0.5 if the symptoms exist in half of the area, 0.75 if the symptoms exist in most of the area, and 1 if the severity of agricultural pests and diseases is covered, so that based on the occurrence site of the pests and diseases of the crops, the information of the symptom of the pests and diseases and the severity of the symptoms, the probability of the bottom event in the fault tree of agricultural pests and diseases in crops can be determined more efficiently and accurately.

[0093] For example, in the embodiment of the present application, the user selects any one or more data in the multimodal data according to the actual situation and inputs them into the agricultural diagnosis large language model for feature extraction, obtains the agricultural pest feature triples input by the user, and determines whether the bottom event in the agricultural pest fault tree occurs, and according to the severity of the symptoms in the bottom event, accurately determines the probability of the bottom event in the agricultural pest fault tree occurring in the crop. If the leaf pest symptom information A in the triple is consistent with the symptom in the bottom event in the agricultural pest fault tree, the probability of the bottom event in the agricultural pest fault tree occurring in the crop is determined according to the severity of the leaf pest symptom information A; if the severity of the symptom is 0.5, then the probability of the bottom event in the agricultural pest fault tree occurring in the crop is also determined to be 0.5, that is, the probability of the leaf symptom in the agricultural pest fault tree occurring in the crop is 0.5.

[0094] The method of the above embodiment, in the process of determining the probability of the bottom event in the agricultural pest and disease fault tree occurring in crops, also takes into account the impact of the severity of the agricultural pest and disease symptoms in the multimodal data on the accuracy of the diagnosis results, that is, the probability of the bottom event in the agricultural pest and disease fault tree occurring in crops is determined based on the severity of the symptoms of the diseased crops in the multimodal data input by the user, thereby effectively improving the accuracy of agricultural pest and disease diagnosis.

[0095] In one embodiment, the probability of the intermediate event in the crop occurrence fault tree of agricultural pests and diseases is determined according to the probability of the bottom event in the crop occurrence fault tree of agricultural pests and diseases, including:

[0096] in, It represents the probability of the bottom event in the fault tree of agricultural pests and diseases occurring in crops; Represents the probability of an intermediate event in the fault tree of agricultural pests and diseases occurring in crops.

[0097] Specifically, in the implementation of the present application, after determining the probability of the bottom event in the agricultural pest fault tree of crops, the probability of the intermediate event in the agricultural pest fault tree of crops can be qualitatively determined based on the following method:

[0098] in, It represents the probability of the bottom event in the fault tree of agricultural pests and diseases occurring in crops; It represents the probability of intermediate events in the fault tree of agricultural pests and diseases of crops; different intermediate events correspond to different bottom events.

[0099] The method of the above embodiment, after determining the probability of the bottom event in the agricultural pest fault tree of crops based on the symptoms of diseased crops in the multimodal data input by the user, can also qualitatively determine the probability of the intermediate events in the agricultural pest fault tree, that is, accurately determine the probability of a certain disease or pest occurring in a certain part of the crop.

[0100] In one embodiment, the probability of the type of pests and diseases corresponding to the top event of the crop is determined according to the probability of the intermediate event in the crop pest and disease fault tree, including:

[0101] in, It represents the probability of the intermediate event in the fault tree of agricultural pests and diseases occurring in crops; Represents the probability of the top event in the fault tree of agricultural pests and diseases occurring in crops.

[0102] Specifically, in the implementation of the present application, after determining the probability of the intermediate events in the fault tree of agricultural pests and diseases of crops, that is, after determining the probability of a certain pest and disease occurring in a certain part of the crop, the probability of the top event in the fault tree of agricultural pests and diseases of crops can be qualitatively determined based on the following method according to the probability of the intermediate events in the fault tree of agricultural pests and diseases:

[0103] in, It represents the probability of the intermediate event in the fault tree of agricultural pests and diseases occurring in crops; It represents the probability of the top event in the fault tree of agricultural pests and diseases, that is, the probability of a certain disease and pest occurring in crops.

[0104] Optionally, if the determined probability of a certain crop disease or pest is greater than a preset value, the determined disease or pest occurrence name and probability can be input into the agricultural diagnosis large language model, and automatically recommended disease or pest control measures can be output to the user.

[0105] The method of the above embodiment determines the probability of the bottom event in the agricultural pest fault tree of crops based on the symptoms of diseased crops in the multimodal data input by the user, and then can qualitatively determine the probability of the intermediate events in the agricultural pest fault tree and the probability of the top event in the agricultural pest fault tree of crops, thereby accurately determining the probability of a certain disease or pest occurring in crops.

[0106] The agricultural pest diagnosis device provided by the present invention is described below. The agricultural pest diagnosis device described below and the agricultural pest diagnosis method described above can be referred to in correspondence with each other. Figure 2 As shown, including: An acquisition module 210, for acquiring multimodal data of crops; A determination module 220 is used to determine the occurrence location of crop diseases and insect pests and symptom information of the diseases and insect pests based on the multimodal data; The diagnosis module 230 is used to obtain the diagnosis results of crop pests and diseases based on the location of crop pests and diseases, pest and disease symptom information and a preset agricultural pest and disease fault tree; the agricultural pest and disease fault tree includes top events, intermediate events and bottom events; among which, the top event includes the pest and disease type; the intermediate event includes the location of the pest and disease; and the bottom event includes the symptoms of the pest and disease in each location.

[0107] Optionally, the acquisition module 210 is specifically used for: At least two of image data of crops, text data of crops, video data of crops, and audio data of crops are acquired.

[0108] Optionally, the determining module 220 is specifically configured to: The multimodal data are input into a preset agricultural diagnosis large language model to obtain agricultural pest and disease triplet information; the agricultural pest and disease triplet information includes crop type, pest and disease occurrence site and pest and disease symptom information; the agricultural diagnosis large language model is used to extract pest and disease symptom information of various parts of the crop from the multimodal data of the crop.

[0109] Optionally, the diagnosis module 230 is specifically used to: determine the probability of the bottom event in the agricultural pest fault tree occurring in the crops according to the occurrence location of the pests and diseases of the crops and the information of the symptoms of the pests and diseases; According to the probability of the bottom event in the fault tree of agricultural pests and diseases of crops, the probability of the intermediate event in the fault tree of agricultural pests and diseases of crops is determined; According to the probability of the intermediate events in the fault tree of agricultural pests and diseases, the probability of the type of pests and diseases corresponding to the top event of crops is determined; The agricultural pests and diseases occurring in crops are determined based on the probability of the pest and disease type corresponding to the top event occurring in crops.

[0110] Optionally, the diagnosis module 230 is specifically used to determine the probability of the bottom event in the agricultural pest fault tree occurring in the crops according to the location of the pest occurrence, the pest symptom information and the severity of the symptoms.

[0111] Optionally,

[0112] in, It represents the probability of the bottom event in the fault tree of agricultural pests and diseases occurring in crops; Represents the probability of an intermediate event in the fault tree of agricultural pests and diseases occurring in crops.

[0113] Optionally,

[0114] in, It represents the probability of the intermediate event in the fault tree of agricultural pests and diseases occurring in crops; Represents the probability of the top event in the fault tree of agricultural pests and diseases occurring in crops.

[0115] Figure 3 An example of a physical structure diagram of an electronic device is provided, and the electronic device may include: a processor 310, a communication interface 320, a memory 330, and a communication bus 340, wherein the processor 310, the communication interface 320, and the memory 330 communicate with each other through the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute an agricultural pest diagnosis method, which includes: obtaining multimodal data of crops; determining the occurrence location of pests and diseases and the symptom information of pests and diseases of the crops according to the multimodal data; obtaining the diagnosis result of pests and diseases of the crops according to the occurrence location of pests and diseases, the symptom information of pests and diseases and a preset agricultural pests and diseases fault tree; the agricultural pests and diseases fault tree includes top events, intermediate events, and bottom events; wherein the top event includes the pest and disease type; the intermediate event includes the occurrence location of pests and diseases; and the bottom event includes the symptoms of pests and diseases at various locations.

[0116] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0117] On the other hand, the present invention also provides a computer program product, which includes a computer program, and the computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the agricultural pest and disease diagnosis method provided by the above-mentioned methods, and the method includes: obtaining multimodal data of crops; determining the occurrence location of pests and diseases and symptom information of the crops based on the multimodal data; obtaining the crop pest and disease diagnosis result based on the occurrence location of pests and diseases, the symptom information of pests and diseases and a preset agricultural pest and disease fault tree; the agricultural pest and disease fault tree includes top events, intermediate events and bottom events; wherein the top event includes the pest and disease type; the intermediate event includes the location where the pest and disease occurs; and the bottom event includes the symptoms of the pest and disease in each location.

[0118] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the agricultural pest and disease diagnosis method provided by the above-mentioned methods, the method comprising: obtaining multimodal data of crops; determining the occurrence site of pests and diseases and symptom information of the crops according to the multimodal data; obtaining the diagnosis result of pests and diseases of the crops according to the occurrence site of pests and diseases, the symptom information of pests and diseases and a preset agricultural pest and disease fault tree; the agricultural pest and disease fault tree comprises top events, intermediate events and bottom events; wherein, the top event comprises the pest and disease type; the intermediate event comprises the occurrence site of the pest and disease; and the bottom event comprises the symptoms of the pest and disease at each site.

[0119] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0120] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0121] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for diagnosing agricultural pests and diseases, characterized in that: include: Obtain multimodal data of crops; Determining the occurrence location of the crop pests and diseases and the information on the symptoms of the pests and diseases according to the multimodal data; The diagnosis result of the crop pests and diseases is obtained according to the occurrence site of the crop pests and diseases, the symptom information of the pests and diseases and the preset agricultural pests and diseases fault tree; the agricultural pests and diseases fault tree includes top events, intermediate events and bottom events; wherein the top event includes the pest and disease type; the intermediate event includes the occurrence site of the pests and diseases; and the bottom event includes the symptoms of the pests and diseases in various sites.

2. The agricultural pest diagnosis method according to claim 1, characterized in that: The obtaining of multimodal data of crops includes: At least two of image data of crops, text data of crops, video data of crops, and audio data of crops are acquired.

3. The agricultural pest diagnosis method according to claim 1, characterized in that: Determining the occurrence location and symptom information of the crop pests and diseases based on the multimodal data includes: The multimodal data are input into a preset agricultural diagnosis large language model to obtain agricultural pest and disease triplet information; the agricultural pest and disease triplet information includes crop type, pest and disease occurrence site and pest and disease symptom information; the agricultural diagnosis large language model is used to extract pest and disease symptom information of various parts of the crop from the multimodal data of the crop.

4. The agricultural pest diagnosis method according to any one of claims 1 to 3, characterized in that: The method of obtaining the crop pest diagnosis result according to the occurrence location of the crop pest, the pest symptom information and the preset agricultural pest fault tree includes: Determine the probability of the bottom event in the agricultural pest fault tree occurring in the crop according to the location of the crop pest and disease occurrence and the pest and disease symptom information; Determine the probability of the intermediate event in the agricultural pests and diseases fault tree of the agricultural crops according to the probability of the bottom event in the agricultural pests and diseases fault tree of the agricultural crops; Determine the probability of the crop occurring the type of pests and diseases corresponding to the top event according to the probability of the intermediate events in the fault tree of the crop occurring the pests and diseases; The agricultural pests and diseases occurring in the crops are determined according to the probability of the pest and disease type corresponding to the top event occurring in the crops.

5. The agricultural pest diagnosis method according to claim 4, characterized in that: Determining the probability of the bottom event in the agricultural pest fault tree occurring in the crop according to the location of the crop pest and disease occurrence and the pest and disease symptom information includes: The probability of the bottom event in the agricultural pest fault tree occurring in the crop is determined according to the occurrence location of the crop pest, the pest symptom information and the severity of the symptom.

6. The agricultural pest diagnosis method according to claim 4, characterized in that: The method of determining the probability of the intermediate events in the agricultural pests and diseases fault tree of the crops according to the probability of the bottom events in the agricultural pests and diseases fault tree of the crops comprises: in, It represents the probability of the bottom event in the fault tree of agricultural pests and diseases occurring in crops; Represents the probability of an intermediate event in the fault tree of agricultural pests and diseases occurring in crops.

7. The agricultural pest diagnosis method according to claim 4, characterized in that: Determining the probability of the crop occurring a pest type corresponding to the top event according to the probability of the intermediate event in the agricultural pest fault tree, comprises: in, It represents the probability of the intermediate event in the fault tree of agricultural pests and diseases occurring in crops; Represents the probability of the top event in the fault tree of agricultural pests and diseases occurring in crops.

8. An agricultural pest diagnosis device, characterized in that: include: An acquisition module, used to acquire multimodal data of crops; A determination module, used to determine the occurrence location of the pests and diseases of the crops and information on the symptoms of the pests and diseases according to the multimodal data; The diagnosis module is used to obtain the diagnosis result of the pests and diseases of the crops according to the occurrence sites of the pests and diseases of the crops, the symptom information of the pests and diseases and the preset agricultural pests and diseases fault tree; the agricultural pests and diseases fault tree includes top events, intermediate events and bottom events; wherein the top event includes the pest and disease type; the intermediate event includes the occurrence site of the pest and disease; and the bottom event includes the symptoms of the pest and disease in various sites.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the agricultural pest diagnosis method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the agricultural pest diagnosis method according to any one of claims 1 to 7 is implemented.

Images