Method, device, electronic device and storage medium for identifying plant diseases and insect pests

Abstract

The present application provides a method, device, electronic device and storage medium for identifying plant diseases and insect pests. The identification method includes: obtaining an image to be detected of a target plant; inputting the image to be detected into a pre-trained disease symptom recognition model to determine at least one target symptom description label of the image to be detected; based on the at least one target symptom description label, determining at least one disease suffered by the target plant by searching the disease knowledge graph of the plant species to which the target plant belongs; the disease includes at least one of diseases or insect pests. In this way, the present application performs multi-label recognition through a pre-trained disease symptom recognition model to determine at least one target symptom description label of the image to be detected of the target plant, and can simultaneously identify multiple diseases and / or insect pests suffered by the target plant through at least one symptom description label and the disease knowledge graph, thereby improving the accuracy of the plant disease and insect pest recognition results.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device, electronic device and storage medium for identifying plant diseases and insect pests. Background Art

[0002] Plant diseases and insect pests refer to diseases and insect pests that infect plants during their growth period. They are one of the main causes of damage to commercial agricultural products. The effective identification of plant diseases and insect pests plays a vital role in agricultural plant disease and insect pest prediction, food security and the stability of the agricultural economy.

[0003] However, the symptoms of plant diseases and pests are highly complex. The symptoms of the same disease or pest can vary to varying degrees at different stages of its onset, and some diseases and pests can even exhibit different characteristics at the same time on the same crop. For example, citrus Huanglongbing has two distinct characteristics: uniform yellowing of leaves and mottled yellowing of leaves. Furthermore, the same plant may harbor multiple diseases and pests. For example, a plant may simultaneously suffer from canker, nutrient deficiency, or epiphytic leafminer disease, resulting in the manifestation of these symptoms on the leaves as a superposition of multiple symptoms.

[0004] Existing plant pest and disease identification technologies mostly use multi-classification image recognition, assigning a single symptom label to each symptom image. However, this approach presents the following challenges: 1) Diseases with the same label can have two distinct symptom images, leading to confusion in training data and reduced dataset aggregation; 2) when multiple symptoms overlap, only the most likely label is output, failing to accurately reflect all symptom information contained in the image and failing to accurately identify multiple plant diseases. Therefore, effectively identifying plant pests and diseases and improving the accuracy of identification results are pressing technical challenges. Summary of the Invention

[0005] In view of this, the purpose of this application is to provide a method, device, electronic device and storage medium for identifying plant diseases and pests, which performs multi-label recognition through a pre-trained disease symptom recognition model, determines at least one target symptom description label of the target plant to be detected image, and simultaneously identifies multiple diseases and / or pests of the target plant through the determined at least one target symptom description label and the disease knowledge graph, thereby improving the accuracy of the plant disease and pest identification results.

[0006] The present invention provides a method for identifying plant diseases and insect pests, the method comprising:

[0007] Acquire an image of the target plant to be detected;

[0008] Inputting the image to be detected into a pre-trained disease symptom recognition model to determine at least one target symptom description label of the image to be detected;

[0009] Based on the at least one target symptom description tag, at least one disease suffered by the target plant is determined by searching the disease knowledge graph of the plant species to which the target plant belongs; the disease includes at least one of a disease or an insect pest.

[0010] Optionally, the disease symptom recognition model is trained by the following steps:

[0011] Obtaining a first training data set and a second training data set; the number of samples in the first training data set is greater than the number of samples in the second training data set, and the sample accuracy in the first training data set is lower than the sample accuracy in the second training data set;

[0012] Pre-training a disease symptom recognition neural network using the first training data set until the disease symptom recognition neural network converges to obtain an initial disease symptom recognition model;

[0013] The model parameters in the initial disease symptom recognition model are fine-tuned using the second training data set until the initial disease symptom recognition model converges to obtain a disease symptom recognition model.

[0014] Optionally, the disease symptom recognition neural network is a self-attention neural network with a hybrid structure, and the self-attention neural network with a hybrid structure includes a self-attention neural network and a deep residual neural network.

[0015] Optionally, using the first training data set to pre-train a disease symptom recognition neural network until the disease symptom recognition neural network converges to obtain an initial disease symptom recognition model, including:

[0016] Inputting each training sample image in the first training data set into the pre-trained deep residual neural network to perform image feature processing to determine a feature sequence for each training sample image; the image feature processing includes at least one of the following processing methods: feature extraction processing, dimensionality conversion processing, global average pooling processing, feature decoupling processing, and feature splicing processing;

[0017] The feature sequence of each training sample image is used as an input feature, and the disease symptom label of each training sample image is used as an output feature. The self-attention neural network is trained until the self-attention neural network converges to obtain an initial disease symptom recognition model; wherein each training sample image has multiple disease symptom labels.

[0018] Optionally, a disease knowledge graph of the plant species to which the target plant belongs is constructed by the following steps:

[0019] Determine the plant species to which the target plant belongs as a category node of the disease knowledge graph to be constructed;

[0020] Determining at least one disease suffered by the plant species to which the target plant belongs, and determining each disease as a disease node under a category node;

[0021] For each disease, each symptom description of the disease is determined as a symptom node under the disease node corresponding to the disease;

[0022] The category node is connected to each disease node, and each disease node is connected to its corresponding symptom node to construct a disease knowledge graph of the plant species to which the target plant belongs.

[0023] Optionally, the symptom description includes a description of the symptom location, a description of the symptom morphology, and a description of the symptom color.

[0024] Optionally, inputting the image to be detected into a pre-trained disease symptom recognition model to determine at least one target symptom description label of the image to be detected includes:

[0025] Inputting the image to be detected into a pre-trained disease symptom recognition model, and determining a feature sequence to be detected of the image to be detected through a pre-trained deep residual neural network in the disease symptom recognition model;

[0026] Identifying a feature sequence to be detected of the image to be detected by using the self-attention neural network in the disease symptom recognition model, and determining a probability value of each symptom description label of the image to be detected;

[0027] The symptom description label with a probability value greater than a preset probability is determined as the target symptom description label of the image to be detected.

[0028] Optionally, inputting the image to be detected into a pre-trained disease symptom recognition model, and determining a feature sequence to be detected of the image to be detected by using a deep residual neural network pre-trained in the disease symptom recognition model, includes:

[0029] Performing feature extraction processing on the image to be detected input into the first target convolutional layer of the deep residual neural network to obtain a first feature map;

[0030] Performing feature extraction processing on the image to be detected input into the second target convolutional layer of the deep residual neural network, and performing dimension conversion processing on the feature map after the feature extraction processing to obtain a second feature map;

[0031] Performing feature extraction processing on the image to be detected input into the third target convolutional layer of the deep residual neural network, and performing dimensionality conversion processing on the feature map after the feature extraction processing to obtain a third feature map;

[0032] Performing global average pooling processing on the first feature map, the second feature map, and the third feature map of the image to be detected, respectively, to determine the first eigenvector, the second eigenvector, and the third eigenvector of the image to be detected;

[0033] Decoupling the first feature map, the second feature map, and the third feature map of the image to be detected in spatial dimensions respectively, and determining a first feature tensor, a second feature tensor, and a third feature tensor of the image to be detected;

[0034] Feature splicing processing is performed based on the first eigenvector, the second eigenvector, the third eigenvector, the first eigentensor, the second eigentensor and the third eigentensor of the image to be detected to determine a feature sequence to be detected of the image to be detected.

[0035] The present application also provides a device for identifying plant diseases and insect pests, the device comprising:

[0036] An acquisition module, used for acquiring an image of a target plant to be detected;

[0037] a label determination module, configured to input the image to be detected into a pre-trained disease symptom recognition model to determine at least one target symptom description label of the image to be detected;

[0038] A disease determination module is used to determine at least one disease suffered by the target plant based on the at least one target symptom description tag by searching the disease knowledge graph of the plant species to which the target plant belongs; the disease includes at least one of a disease or an insect pest.

[0039] Optionally, the recognition device further includes a model training module, and the model training module is used to:

[0040] Obtaining a first training data set and a second training data set; the number of samples in the first training data set is greater than the number of samples in the second training data set, and the sample accuracy in the first training data set is lower than the sample accuracy in the second training data set;

[0041] Pre-training a disease symptom recognition neural network using the first training data set until the disease symptom recognition neural network converges to obtain an initial disease symptom recognition model;

[0042] The model parameters in the initial disease symptom recognition model are fine-tuned using the second training data set until the initial disease symptom recognition model converges to obtain a disease symptom recognition model.

[0043] Optionally, the disease symptom recognition neural network is a self-attention neural network with a hybrid structure, and the self-attention neural network with a hybrid structure includes a self-attention neural network and a deep residual neural network.

[0044] Optionally, when the model training module is used to pre-train a disease symptom recognition neural network using the first training data set until the disease symptom recognition neural network converges to obtain an initial disease symptom recognition model, the model training module is used to:

[0045] Inputting each training sample image in the first training data set into the pre-trained deep residual neural network to perform image feature processing to determine a feature sequence for each training sample image; the image feature processing includes at least one of the following processing methods: feature extraction processing, dimensionality conversion processing, global average pooling processing, feature decoupling processing, and feature splicing processing;

[0046] The feature sequence of each training sample image is used as an input feature, and the disease symptom label of each training sample image is used as an output feature. The self-attention neural network is trained until the self-attention neural network converges to obtain an initial disease symptom recognition model; wherein each training sample image has multiple disease symptom labels.

[0047] Optionally, the recognition device further includes a knowledge graph construction module, and the knowledge graph construction module is used to:

[0048] Determine the plant species to which the target plant belongs as a category node of the disease knowledge graph to be constructed;

[0049] Determining at least one disease suffered by the plant species to which the target plant belongs, and determining each disease as a disease node under a category node;

[0050] For each disease, each symptom description of the disease is determined as a symptom node under the disease node corresponding to the disease;

[0051] The category node is connected to each disease node, and each disease node is connected to its corresponding symptom node to construct a disease knowledge graph of the plant species to which the target plant belongs.

[0052] Optionally, the symptom description includes a description of the symptom location, a description of the symptom morphology, and a description of the symptom color.

[0053] Optionally, when the label determination module is used to input the image to be detected into a pre-trained disease symptom recognition model to determine at least one target symptom description label of the image to be detected, the label determination module is used to:

[0054] Inputting the image to be detected into a pre-trained disease symptom recognition model, and determining a feature sequence to be detected of the image to be detected through a pre-trained deep residual neural network in the disease symptom recognition model;

[0055] Identifying a feature sequence to be detected of the image to be detected by using the self-attention neural network in the disease symptom recognition model, and determining a probability value of each symptom description label of the image to be detected;

[0056] The symptom description label with a probability value greater than a preset probability is determined as the symptom description label of the image to be detected.

[0057] Optionally, when the label determination module is used to input the image to be detected into a pre-trained disease symptom recognition model and determine the feature sequence to be detected of the image to be detected through a deep residual neural network pre-trained in the disease symptom recognition model, the label determination module is used to:

[0058] Performing feature extraction processing on the image to be detected input into the first target convolutional layer of the deep residual neural network to obtain a first feature map;

[0059] Performing feature extraction processing on the image to be detected input into the second target convolutional layer of the deep residual neural network, and performing dimension conversion processing on the feature map after the feature extraction processing to obtain a second feature map;

[0060] Performing feature extraction processing on the image to be detected input into the third target convolutional layer of the deep residual neural network, and performing dimensionality conversion processing on the feature map after the feature extraction processing to obtain a third feature map;

[0061] Performing global average pooling processing on the first feature map, the second feature map, and the third feature map of the image to be detected, respectively, to determine the first eigenvector, the second eigenvector, and the third eigenvector of the image to be detected;

[0062] Decoupling the first feature map, the second feature map, and the third feature map of the image to be detected in spatial dimensions respectively, and determining a first feature tensor, a second feature tensor, and a third feature tensor of the image to be detected;

[0063] Feature splicing processing is performed based on the first eigenvector, the second eigenvector, the third eigenvector, the first eigentensor, the second eigentensor and the third eigentensor of the image to be detected to determine a feature sequence to be detected of the image to be detected.

[0064] An embodiment of the present application also provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the above-mentioned identification method are performed.

[0065] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned identification method are executed.

[0066] The embodiments of the present application provide a method, device, electronic device and storage medium for identifying plant diseases and pests. The identification method includes: obtaining an image to be detected of a target plant; inputting the image to be detected into a pre-trained disease symptom recognition model to determine at least one target symptom description label of the image to be detected; based on the at least one target symptom description label, determining at least one disease suffered by the target plant by searching a disease knowledge graph of the plant species to which the target plant belongs; the disease includes at least one of a disease or a pest.

[0067] In this way, this application combines image multi-label classification technology with knowledge graph technology, obtains detailed symptom description information through multi-label description of pest and disease parts, and then connects symptoms with pest and disease entities through knowledge graphs, so that this solution has the ability to identify complex pest and disease symptoms, especially complex pest and disease symptoms occurring in the same part. Secondly, by adopting a hierarchical hybrid self-attention neural network model, by extracting feature maps of different layers of the deep residual neural network, and by performing image feature processing on the extracted feature maps, the feature sequence corresponding to the feature map is determined, and self-attention is applied to the feature sequence so that the same pixel area on the input image can have multiple label attributes, so that the disease symptom recognition model has a stronger ability to express complex disease characteristics.

[0068] Furthermore, because this solution doesn't directly map input images to plant pest and disease types, but instead uses intuitive descriptions of disease symptoms, labelers don't need to use specialized knowledge to interpret disease data images. Instead, they only need to describe intuitive information contained in the image, such as location, spots, stripes, and color, reducing the professional requirements for labelers. This data labeling approach also makes it possible to create large-scale, low-precision datasets for plant pest and disease identification. Therefore, this solution can adopt a training method of "large-scale, low-precision dataset pre-training + small-scale, high-precision dataset fine-tuning," thereby reducing data labeling costs and expanding the source of training data samples.

[0069] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0071] Figure 1 A flow chart of a method for identifying plant diseases and insect pests provided in an embodiment of the present application;

[0072] Figure 2 Schematic diagram of the neural network structure for disease symptom recognition;

[0073] Figure 3 A schematic diagram of the disease knowledge graph of the plant species to which the target plant belongs;

[0074] Figure 4 This is one of the structural schematic diagrams of a plant disease and insect pest identification device provided in an embodiment of the present application;

[0075] Figure 5 This is a second structural diagram of a plant disease and insect pest identification device provided in an embodiment of the present application;

[0076] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0077] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application for which protection is claimed, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, each other embodiment obtained by those skilled in the art without making creative work falls within the scope of protection of the present application.

[0078] Plant diseases and insect pests refer to diseases and insect pests that infect plants during their growth period. They are one of the main causes of damage to commercial agricultural products. The effective identification of plant diseases and insect pests plays a vital role in agricultural plant disease and insect pest prediction, food security and the stability of the agricultural economy.

[0079] However, the symptoms of plant diseases and pests are highly complex. The symptoms of the same disease or pest can vary to varying degrees at different stages of its onset, and some diseases and pests can even exhibit different characteristics at the same time on the same crop. For example, citrus Huanglongbing has two distinct characteristics: uniform yellowing of leaves and mottled yellowing of leaves. Furthermore, the same plant may harbor multiple diseases and pests. For example, a plant may simultaneously suffer from canker, nutrient deficiency, or epiphytic leafminer disease, resulting in the manifestation of these symptoms on the leaves as a superposition of multiple symptoms.

[0080] Existing plant pest and disease identification technologies mostly use multi-classification image recognition, assigning a single symptom label to each symptom image. However, this approach presents the following challenges: 1) Diseases with the same label can have two distinct symptom images, leading to confusion in training data and reduced dataset aggregation; 2) when multiple symptoms overlap, only the most likely label is output, failing to accurately reflect all symptom information contained in the image and failing to accurately identify multiple plant diseases. Therefore, effectively identifying plant pests and diseases and improving the accuracy of identification results are pressing technical challenges.

[0081] Based on this, an embodiment of the present application provides a method for identifying plant diseases and pests, which performs multi-label recognition through a pre-trained disease symptom recognition model, determines at least one target symptom description label of the target plant's image to be detected, and uses the determined at least one target symptom description label and the disease knowledge graph to identify multiple diseases and / or pests of the target plant, thereby improving the accuracy of the plant disease and pest identification results.

[0082] See also Figure 1 , Figure 1 This is a flow chart of a method for identifying plant diseases and insect pests provided in an embodiment of the present application. Figure 1 As shown in , the plant disease and insect pest identification method provided in the embodiment of the present application includes:

[0083] S101: Acquire an image of a target plant to be detected.

[0084] In this step, the target plant is a plant that needs to be identified for pests and diseases. The image to be detected is generally an image collected of the diseased part of the target plant. The image to be detected is a color image, and the diseased part can be the leaves, branches, fruits, etc. of the target plant.

[0085] S102: Input the image to be detected into a pre-trained disease symptom recognition model to determine at least one target symptom description label of the image to be detected.

[0086] In this step, the disease symptom recognition model is a multi-label image recognition model, which is used to identify disease and pest symptoms in the input image to be detected, determine the target symptom description label in the image to be detected, and obtain at least one target symptom description label, thereby determining the diseases and / or pests of the target plant based on at least one target symptom description label.

[0087] Here, generally, the number of target symptom description labels is no less than 3, including description labels of symptom location, symptom morphology, and symptom color. The disease symptom recognition model refers to the disease symptom recognition model of the plant species to which the target plant belongs.

[0088] In one embodiment of the present application, the disease symptom recognition model is trained by the following steps: obtaining a first training data set and a second training data set; the number of samples in the first training data set is higher than the number of samples in the second training data set, and the sample accuracy in the first training data set is lower than the sample accuracy in the second training data set; using the first training data set to pre-train a disease symptom recognition neural network until the disease symptom recognition neural network converges to obtain an initial disease symptom recognition model; using the second training data set to fine-tune the model parameters in the initial disease symptom recognition model until the initial disease symptom recognition model converges to obtain a disease symptom recognition model. The disease symptom recognition neural network is a self-attention neural network with a hybrid structure, and the self-attention neural network with a hybrid structure includes a self-attention neural network and a deep residual neural network.

[0089] This step is the process of training the disease symptom recognition model. The training method of the disease symptom recognition model is "large-scale low-precision data set pre-training + small-scale high-precision data set fine-tuning". Through the training method of this application, a self-attention neural network with a hybrid structure including a self-attention neural network and a deep residual neural network is trained. When the convergence condition is reached, the training is terminated to obtain the disease symptom recognition model. The disease symptom recognition model can be used to identify the image to be detected of the target plant and determine multiple target symptom description labels for the image to be detected.

[0090] Here, the self-attention neural network is a ViT neural network, and the deep residual neural network is a ResNet. The labels of the samples in the first training data set can be labeled by ordinary labelers.

[0091] For example, when the target plant is an orange plant (the orange plant here includes six parts: roots, stems, leaves, flowers, fruits, and seeds), the constructed disease symptom identification model can be a disease symptom identification model for citrus. Here, the implementation method in the above embodiment is briefly described by taking the process of constructing the disease symptom identification model for citrus as an example.

[0092] First, prepare training samples for building a disease symptom recognition model, namely the above-mentioned first training data set and second training data set; wherein, the first training data set is a large-scale low-precision training data set. The training data in the first training data set does not completely use citrus plant images as training samples. Other similar crops of the same genus as citrus plants, such as lemons, grapefruits and other crops, can be added to the training set. Images of relevant parts (branches, leaves, etc.) are added to the data set to increase the training samples. When the data set is produced, the health status of the crops is not distinguished. Only the parts (roots, stems, leaves, flowers, fruits, seeds), morphology (such as spots, holes, vein thickness) and color (yellow, green) are annotated; wherein the annotation of the samples in the first training data set can be done by ordinary personnel who are not professional technicians in citrus disease and pest control, thereby reducing the difficulty of annotation and the professional requirements of the annotators. wherein, in addition to images of diseases and pests of the selected plants, the images used in the first training data set can also include healthy images of the selected plants.

[0093] The second training dataset is a small-scale, high-precision training dataset. This dataset consists entirely of citrus image samples, each of which has been annotated by citrus pest and disease control professionals. In addition to images of citrus pests and diseases, the citrus image samples in this second training dataset also include images of healthy citrus plant parts. The annotations detail the symptom location, symptom morphology, and color of the symptom location displayed in the corresponding image (for healthy images, the overall morphological characteristics and color of the location are described in detail).

[0094] After preparing the training dataset, the first training dataset is used for coarse-grained pre-training. The disease symptom recognition neural network is trained on the first training dataset until convergence, resulting in an initial disease symptom recognition model for citrus. This leads to the next stage of fine-tuning training. The model parameters in the initial disease symptom recognition model are fine-tuned using the second training dataset. The model is then trained on the second training dataset until convergence, completing model training. Finally, the disease symptom recognition model for citrus is obtained, effectively achieving the target disease symptom recognition model.

[0095] For example, when annotating images of citrus, suppose an image is an image representing citrus diseases, and the image is an image of a citrus leaf. The leaves in the image have morphological characteristics of unevenness, bumps, and swollen veins, and the leaves are yellow. In this way, when the annotator describes the symptoms of the image, it can be described as "uneven yellowing of leaves", "bumps and unevenness of leaves", and "swollen veins of leaves". Based on the symptom description, the symptom location labels are extracted as "leaves" and "veins", the symptom morphology labels are "uneven", "bumps and unevenness", and "swollen", and the color description is "yellow". The above 6 labels are all labels for this image. Here, when annotating images, the image can be annotated based on the description of the symptom node in the knowledge graph, so that the preset keywords can be automatically extracted as labels for the image based on keyword extraction technology.

[0096] When annotating the images of the first training dataset and the second training dataset, three symptom description labels may be included, namely, symptom location labels, symptom morphology labels, and symptom color labels. Each symptom description label includes at least one label.

[0097] It should be noted that when performing model training based on the first training data set and the second training data set, for any one of the first training data set and the second training data set, during training, the training data set can be divided into training, verification, and test data sets in a certain proportion.

[0098] In another embodiment of the present application, the disease symptom recognition neural network is pre-trained using the first training data set until the disease symptom recognition neural network converges to obtain an initial disease symptom recognition model, including: inputting each training sample image in the first training data set into the pre-trained deep residual neural network for image feature processing to determine the feature sequence of each training sample image; the image feature processing includes at least one of the following processing methods: feature extraction processing, dimensionality conversion processing, global average pooling processing, feature decoupling processing and feature splicing processing; using the feature sequence of each training sample image as input feature and the disease symptom label of each training sample image as output feature, training the self-attention neural network until the self-attention neural network converges to obtain an initial disease symptom recognition model; wherein, each training sample image has multiple disease symptom labels.

[0099] This step is the pre-training process for building a disease symptom recognition model, specifically comprising the following steps: First, each training sample image in the first training dataset is input into a pre-trained deep residual neural network in the disease symptom recognition neural network. The deep residual neural network then performs image feature processing on each image in the first training dataset input thereto. Here, the image feature processing includes at least one processing method: feature extraction processing, dimensionality conversion processing, global average pooling processing, feature decoupling processing, and feature splicing processing. After image feature processing, an image feature sequence for each training image in the training set can be obtained. Then, for each training image, the determined image feature sequence of the training image is input as the input feature into the self-attention neural network, and the disease symptom label of the training image is used as the output feature. The self-attention neural network is trained until the self-attention neural network converges to obtain an initial disease symptom recognition model. In particular, each training sample image used in the training process has multiple disease symptom labels, thereby ensuring that the disease symptom recognition model that is successfully trained subsequently has the function of simultaneously recognizing multiple symptoms and outputting multiple disease symptom labels.

[0100] Here, in the process of training the self-attention neural network with the feature sequence of the training sample image as the input feature and the disease symptom label of the training sample image as the output feature, spatial self-attention and global hierarchical self-attention can be applied to the input feature sequence respectively, so that the same pixel area on the input image can have multiple label attributes.

[0101] For example, when a leaf has three labels: "yellow," "bumpy," and "uneven color," spatial self-attention enables the self-attention neural network to focus primarily on the "leaf" region in each feature map. Furthermore, for labels like "yellow," which are described by patches of color, global hierarchical self-attention enables the self-attention neural network to focus more on the top-level feature maps of the backbone network. For labels like "bumpy" and "uneven color," which are described more by texture structures, global hierarchical self-attention directs the network to focus more on the lower-level feature maps of the backbone network. This empowers the multi-label image classifier to better identify complex diseases.

[0102] In addition, the fine-tuning process is similar to the pre-training process in terms of training method. The only difference is that the initial training objects and the ending conditions may be different, so we will not go into details here.

[0103] In another embodiment of the present application, each training sample image in the first training data set is input into the pre-trained deep residual neural network for image feature processing, and a feature sequence of each training sample image is determined, including: for each training sample image, performing feature extraction processing on the training sample image input into the first target convolution layer in the deep residual neural network to obtain a first feature map; for each training sample image, performing feature extraction processing on the training sample image input into the second target convolution layer in the deep residual neural network, and performing dimensionality conversion processing on the feature map after feature extraction processing to obtain a second feature map; for each training sample image, performing feature extraction processing on the training sample image input into the third target convolution layer in the deep residual neural network The method comprises the following steps: extracting the first feature map, performing dimension conversion on the feature map after feature extraction, and obtaining a third feature map; performing global average pooling on the first feature map, the second feature map and the third feature map of each training sample image, respectively, to determine the first eigenvector, the second eigenvector and the third eigenvector of each training sample image; decoupling the first feature map, the second feature map and the third feature map of each training sample image in the spatial dimension, respectively, to determine the first feature tensor, the second feature tensor and the third feature tensor of each training sample image; for each training sample image, performing feature splicing processing based on the first eigenvector, the second eigenvector, the third eigenvector, the first feature tensor, the second feature tensor and the third feature tensor of the training sample image, and obtaining a feature sequence of the training sample image.

[0104] For examples, see Figure 2 , Figure 2 Schematic diagram of the structure of the neural network for disease symptom recognition, such as Figure 2As shown in , A represents the convolutional neural network. The convolutional neural network selected here is the deep residual neural network ResNet. A shows the internal structure of part of ResNet. The selected ResNet has 5 convolutional layers. Res5 is determined as the first target convolutional layer, Res4 is determined as the second target convolutional layer, and Res2 is determined as the third target convolutional layer. The dimension of the feature map output by each layer is different. B represents the self-attention neural network ViT. Figure 2 As shown, through the first target convolution layer Res5, the first feature map F1 is obtained. Assuming that its dimension is 1024×14×14, the first feature map F1 is globally averaged pooled to obtain a first feature vector g1 of 1024×1×1 dimensions. The first feature map F1 is decoupled in the spatial dimension to obtain a 1024×14 2 ×1-dimensional feature tensor K1; the feature map output by the second target convolution layer Res4 is transformed in dimension to obtain the second feature map F2, whose dimension is 1024×28×28. The second feature map F2 is globally averaged pooled to obtain a second feature vector g2 of 1024×1×1 dimension. The second feature map F2 is decoupled in the spatial dimension to obtain a 1024×28 2 ×1-dimensional feature tensor K2; the feature map output by the third target convolution layer Res2 is transformed to obtain a third feature map F3 with a dimension of 1024×56×56. The third feature map F3 is globally averaged pooled to obtain a third feature vector g3 with a dimension of 1024×1×1. The third feature map F3 is decoupled in the spatial dimension to obtain a 1024×56 2 ×1-dimensional feature tensor K3, and then perform feature splicing on the determined g1, g2, g3, K1, K2 and K3 to obtain the feature sequence X of the training sample image, whose dimension is 1024×N×1, where N=3+14 2 +28 2 +56 2 In this way, the feature sequence of each training sample image can be determined. Here, the reason for the dimension conversion process is to ensure the dimension is unified for feature splicing.

[0105] After determining the feature sequence X, the disease symptom recognition model can be constructed. The feature sequence X is input into ViT, and the spatial class-related attention score of the feature sequence X is calculated.

[0106]

[0107] Among them, x k ∈X,x k is the feature tensor, whose dimension is 1024×1×1; m iis the parameter of the i-th classifier; T is the hyperparameter for multi-head self-attention training. In this scheme, the number of attention heads can be selected as 8, and the value of T is: T1~T7 is 1, 2, 3, 4, 5, 6, 7, T8=1000.

[0108] Furthermore, the spatial self-attention feature a and the global hierarchical self-attention feature g of the i-th classification label are determined:

[0109]

[0110]

[0111] Furthermore, the i-th type of spatial self-attention feature representation of each input training sample image is obtained:

[0112] f i,T =g i,T +λa i,T

[0113] Here, λ is a scaling factor between 0 and 1, and its value is determined by parameter adjustment during training. Then, through the activation layer (using the sigmoid activation function), the multi-label classification probability is output, and the model parameters are updated using the labels of the training sample images, thus obtaining a disease symptom recognition model.

[0114] It should be noted that while the non-hybrid ViT model has powerful hierarchical feature expression capabilities for multi-label classification tasks, its hierarchical feature extraction mechanism is unexplainable and difficult to optimize because it does not use a convolutional network for hierarchical image feature extraction. Furthermore, the model requires a large amount of training data, and in practical applications, it must be trained using a "pre-training + fine-tuning" approach. Citrus pest and disease identification is a highly specialized task that moves from segmented scenarios. Images of pest and disease symptoms and related parts differ significantly from images in industrial-grade general-purpose image datasets, making it difficult to achieve good performance using a "pre-training + fine-tuning" approach for model training. This solution, however, employs a hierarchical hybrid ViT neural network model. By extracting feature maps from different layers of the backbone CNN network to form a feature sequence, and applying spatial self-attention and global hierarchical self-attention to the feature sequence of each feature map, the same pixel region in the input image can have multiple label attributes, giving the multi-label image classifier a stronger ability to express complex disease features. Furthermore, since this solution does not directly map input images to pest and disease types, but instead uses an intuitive description of the image features of disease symptoms, annotators do not need to use professional knowledge to interpret disease data images. Instead, they only need to describe the intuitive information contained in the image, such as location, spots, stripes, and color, which reduces the professional requirements of annotators. This data annotation method also makes it possible to produce large-scale low-precision datasets in the field of plant pest and disease identification. Therefore, this solution can adopt a training method of "large-scale low-precision dataset pre-training + small-scale high-precision dataset fine-tuning", thereby reducing data annotation costs and expanding the source of training data samples.

[0115] In addition, the fine-tuning process is similar to the pre-training process in terms of training method. The only difference is that the initial training objects and the ending conditions may be different, so we will not go into details here.

[0116] In another embodiment of the present application, the step of inputting the image to be detected into a pre-trained disease symptom recognition model to determine at least one target symptom description label of the image to be detected includes: inputting the image to be detected into a pre-trained disease symptom recognition model, and determining the feature sequence to be detected of the image to be detected through a pre-trained deep residual neural network in the disease symptom recognition model; identifying the feature sequence to be detected of the image to be detected through a self-attention neural network in the disease symptom recognition model, and determining the probability value of each symptom description label of the image to be detected; and determining the symptom description label having a probability value greater than a preset probability as the target symptom description label of the image to be detected.

[0117] In another embodiment of the present application, the image to be detected is input into a pre-trained disease symptom recognition model, and the feature sequence to be detected of the image to be detected is determined by the pre-trained deep residual neural network in the disease symptom recognition model, including: performing feature extraction processing on the image to be detected input into the first target convolution layer of the deep residual neural network to obtain a first feature map; performing feature extraction processing on the image to be detected input into the second target convolution layer of the deep residual neural network, and performing dimension conversion processing on the feature map after feature extraction processing to obtain a second feature map; performing feature extraction processing on the image to be detected input into the third target convolution layer of the deep residual neural network, and performing dimension conversion processing on the feature map after feature extraction processing to obtain a second feature map. The feature map after extraction processing is subjected to dimension conversion processing to obtain a third feature map; the first feature map, the second feature map and the third feature map of the image to be detected are subjected to global average pooling processing respectively to determine the first eigenvector, the second eigenvector and the third eigenvector of the image to be detected; the first feature map, the second feature map and the third feature map of the image to be detected are decoupled in the spatial dimension respectively to determine the first feature tensor, the second feature tensor and the third feature tensor of the image to be detected; feature splicing processing is performed based on the first eigenvector, the second eigenvector, the third eigenvector, the first feature tensor, the second feature tensor and the third feature tensor of the image to be detected to determine the feature sequence to be detected of the image to be detected.

[0118] Here, the process of determining the feature sequence to be detected of the image to be detected is substantially the same as the process of determining the feature sequence of the training sample image, and therefore will not be described in detail here.

[0119] Here, the preset probability may be selected according to the applicability, and the preset probability value corresponding to each identified tag may be the same.

[0120] S103. Based on the at least one target symptom description tag, determine at least one disease suffered by the target plant by searching the disease knowledge graph of the plant species to which the target plant belongs; the disease includes at least one of a disease or an insect pest.

[0121] In this step, after determining at least one target symptom description label of the image to be detected of the target plant based on the trained disease symptom recognition model, the symptom node consistent with the target symptom description label is determined according to the symptom node in the disease knowledge graph of the plant species to which the target plant belongs. In this way, the disease suffered by the target plant can be determined based on the disease node above the symptom node, and multiple diseases at the same location of the target plant can be diagnosed simultaneously.

[0122] For example, when the target plant is an orange plant, the plant species to which the target plant belongs may be a citrus plant or a citrus plant.

[0123] In one embodiment of the present application, a disease knowledge graph of the plant species to which the target plant belongs is constructed by the following steps: determining the plant species to which the target plant belongs as a category node of the disease knowledge graph to be constructed; determining at least one disease suffered by the plant species to which the target plant belongs, and determining each disease as a disease node under the category node; for each disease, determining each symptom description of the disease as a symptom node under the disease node corresponding to the disease; connecting the category node with each disease node, and connecting each disease node with its corresponding symptom node, to construct a disease knowledge graph of the plant species to which the target plant belongs. The symptom description includes a symptom location description, a symptom morphology description, and a symptom color description.

[0124] Here, a disease node represents a disease or pest that the target plant species can suffer from, and all disease nodes represent all diseases and pests that the target plant species can suffer from. All symptom nodes constitute all symptom descriptions of all diseases and pests that the target plant species can suffer from. Here, a disease corresponds to at least three symptom descriptions, including a symptom location description, a symptom morphology description, and a symptom color description, with each symptom description having at least one. In other words, a disease node is connected to at least three symptom nodes, and different disease nodes can be connected to the same symptom node.

[0125] For examples, see Figure 3 , Figure 3 Schematic diagram of the disease knowledge graph of the plant species to which the target plant belongs. Figure 3 The "plant" shown in the figure represents the target plant species. The disease knowledge graph includes n possible diseases, including disease 1, disease 2, and disease n. It also includes s possible symptom manifestations, including symptom 1 and symptom s. a represents the symptom location node, b represents the symptom morphology node, and c represents the symptom color node.

[0126] It should be noted that the labels of samples in the first and second training datasets can be selected and added based on the symptom nodes in the disease knowledge graph. For example, if the symptom node in a disease knowledge graph is "swelling", and a symptom description label in the first and second training datasets is also labeled "swelling" and should not be labeled "hypertrophy", then the target plant can be automatically determined to have at least one disease based on the target symptom description label and the disease knowledge graph of the plant species to which the target plant belongs.

[0127] Among them, the method for automatically determining at least one disease suffered by the target plant based on the target symptom description label and the disease knowledge graph of the plant species to which the target plant belongs can be, based on at least one target symptom description label, generating a query statement for querying the disease from the knowledge graph, and then searching according to the query statement to determine at least one disease suffered by the target plant.

[0128] In another embodiment of the present application, based on the at least one target symptom description label, at least one disease suffered by the target plant is determined by searching the disease knowledge graph of the plant species to which the target plant belongs, including: for each target symptom description label, determining the symptom node corresponding to the target symptom description label in the disease knowledge graph; based on the symptom node corresponding to each target symptom description label of the image to be detected, determining at least one disease node on the symptom node; and determining at least one disease suffered by the target plant according to the disease name of each disease node.

[0129] Here, the target symptom description labels for the image to be detected, as determined by the disease symptom recognition model, are generally at least three, including a symptom location description label, a symptom morphology description label, and a symptom color description label. For each target symptom description label, the symptom node corresponding to the target symptom description label in the disease knowledge graph is determined. Specifically, the symptom node corresponding to the target symptom description label is determined. When a symptom location node, symptom morphology node, and symptom color node simultaneously correspond to a disease, the disease is determined to be a disease of the target plant, thereby enabling diagnosis of multiple diseases in the same part of the target plant.

[0130] For examples, please see Figure 3 As shown, when the target symptom description labels of the image to be detected correspond to the symptom 1 node, symptom m+1 node, symptom m+d+1 node and symptom m+d+2 node in the symptom node group in the knowledge graph, it can be determined that the diseases suffered by the target plant are disease 1 and disease 2.

[0131] Here, in the process of determining the diseases suffered by the target plant based on the target symptom description label, a query statement can be first generated based on the target symptom description label, and then the corresponding disease node can be searched from the knowledge graph through the query statement, so as to determine at least one disease suffered by the target plant based on the disease node.

[0132] For example, if the generated target symptom description tags are leaves, spots, stripes, and red, the corresponding query statement can be: match {location_on:{leaves}, has_symptom:{spots, stripes}, has_color:{red}}, which will return query results for the target plant diseases based on this query statement. It should be noted that the specific query statement is determined by the graph database software used to build the knowledge graph.

[0133] An embodiment of the present application provides a method for identifying plant diseases and pests, which includes: obtaining an image to be detected of a target plant; inputting the image to be detected into a pre-trained disease symptom recognition model to determine at least one target symptom description label of the image to be detected; based on the at least one target symptom description label, determining at least one disease suffered by the target plant by searching a disease knowledge graph of the plant species to which the target plant belongs; the disease includes at least one of a disease or a pest.

[0134] In this way, this application combines image multi-label classification technology with knowledge graph technology, obtains detailed symptom description information through multi-label description of pest and disease parts, and then connects symptoms with pest and disease entities through knowledge graphs, so that this solution has the ability to identify complex pest and disease symptoms, especially complex pest and disease symptoms occurring in the same part. Secondly, by adopting a hierarchical hybrid self-attention neural network model, by extracting feature maps of different layers of the deep residual neural network, and by performing image feature processing on the extracted feature maps, the feature sequence corresponding to the feature map is determined, and self-attention is applied to the feature sequence so that the same pixel area on the input image can have multiple label attributes, so that the disease symptom recognition model has a stronger ability to express complex disease characteristics.

[0135] Furthermore, because this solution doesn't directly map input images to citrus pest and disease types, but instead uses intuitive descriptions of disease symptom image features, labelers don't need to use specialized knowledge to interpret disease data images. Instead, they only need to describe intuitive information contained in the image, such as location, spots, stripes, and color, reducing the professional requirements for labelers. This data labeling approach also makes it possible to produce large-scale, low-precision datasets for plant pest and disease identification. Therefore, this solution can adopt a training method of "large-scale, low-precision dataset pre-training + small-scale, high-precision dataset fine-tuning," reducing data labeling costs and expanding the source of training data samples.

[0136] See also Figure 4 、 Figure 5 , Figure 4 This is one of the structural diagrams of a plant disease and insect pest identification device provided in an embodiment of the present application. Figure 5 This is a second structural diagram of a plant pest identification device provided in an embodiment of the present application. Figure 4 As shown in , the identification device 400 includes:

[0137] An acquisition module 410 is used to acquire an image of a target plant to be detected;

[0138] The label determination module 420 is used to input the image to be detected into a pre-trained disease symptom recognition model to determine at least one target symptom description label of the image to be detected;

[0139] The disease determination module 430 is used to determine at least one disease suffered by the target plant based on the at least one target symptom description tag by searching the disease knowledge graph of the plant species to which the target plant belongs; the disease includes at least one of a disease or an insect pest.

[0140] Optional, such as Figure 5 As shown, the recognition device 400 further includes a model training module 440, which is used to:

[0141] Obtaining a first training data set and a second training data set; the number of samples in the first training data set is greater than the number of samples in the second training data set, and the sample accuracy in the first training data set is lower than the sample accuracy in the second training data set;

[0142] Pre-training a disease symptom recognition neural network using the first training data set until the disease symptom recognition neural network converges to obtain an initial disease symptom recognition model;

[0143] The model parameters in the initial disease symptom recognition model are fine-tuned using the second training data set until the initial disease symptom recognition model converges to obtain a disease symptom recognition model.

[0144] Optionally, the disease symptom recognition neural network is a self-attention neural network with a hybrid structure, and the self-attention neural network with a hybrid structure includes a self-attention neural network and a deep residual neural network.

[0145] Optionally, when the model training module 440 is used to pre-train the disease symptom recognition neural network using the first training data set until the disease symptom recognition neural network converges and an initial disease symptom recognition model is obtained, the model training module 440 is used to:

[0146] Inputting each training sample image in the first training data set into the pre-trained deep residual neural network to perform image feature processing to determine a feature sequence for each training sample image; the image feature processing includes at least one of the following processing methods: feature extraction processing, dimensionality conversion processing, global average pooling processing, feature decoupling processing, and feature splicing processing;

[0147] The feature sequence of each training sample image is used as an input feature, and the disease symptom label of each training sample image is used as an output feature. The self-attention neural network is trained until the self-attention neural network converges to obtain an initial disease symptom recognition model; wherein each training sample image has multiple disease symptom labels.

[0148] Optionally, the recognition device 400 further includes a knowledge graph construction module 450, and the knowledge graph construction module 450 is used to:

[0149] Determine the plant species to which the target plant belongs as a category node of the disease knowledge graph to be constructed;

[0150] Determining at least one disease suffered by the plant species to which the target plant belongs, and determining each disease as a disease node under a category node;

[0151] For each disease, each symptom description of the disease is determined as a symptom node under the disease node corresponding to the disease;

[0152] The category node is connected to each disease node, and each disease node is connected to its corresponding symptom node to construct a disease knowledge graph of the plant species to which the target plant belongs.

[0153] Optionally, the symptom description includes a description of the symptom location, a description of the symptom morphology, and a description of the symptom color.

[0154] Optionally, when the label determination module 420 is used to input the image to be detected into a pre-trained disease symptom recognition model to determine at least one target symptom description label of the image to be detected, the label determination module 420 is used to:

[0155] Inputting the image to be detected into a pre-trained disease symptom recognition model, and determining a feature sequence to be detected of the image to be detected through a pre-trained deep residual neural network in the disease symptom recognition model;

[0156] Identifying a feature sequence to be detected of the image to be detected by using the self-attention neural network in the disease symptom recognition model, and determining a probability value of each symptom description label of the image to be detected;

[0157] The symptom description label with a probability value greater than a preset probability is determined as the target symptom description label of the image to be detected.

[0158] Optionally, when the label determination module 420 is used to input the image to be detected into a pre-trained disease symptom recognition model and determine the feature sequence to be detected of the image to be detected through a deep residual neural network pre-trained in the disease symptom recognition model, the label determination module 420 is used to:

[0159] Performing feature extraction processing on the image to be detected input into the first target convolutional layer of the deep residual neural network to obtain a first feature map;

[0160] Performing feature extraction processing on the image to be detected input into the second target convolutional layer of the deep residual neural network, and performing dimension conversion processing on the feature map after the feature extraction processing to obtain a second feature map;

[0161] Performing feature extraction processing on the image to be detected input into the third target convolutional layer of the deep residual neural network, and performing dimensionality conversion processing on the feature map after the feature extraction processing to obtain a third feature map;

[0162] Performing global average pooling processing on the first feature map, the second feature map, and the third feature map of the image to be detected, respectively, to determine the first eigenvector, the second eigenvector, and the third eigenvector of the image to be detected;

[0163] Decoupling the first feature map, the second feature map, and the third feature map of the image to be detected in spatial dimensions respectively, and determining a first feature tensor, a second feature tensor, and a third feature tensor of the image to be detected;

[0164] Feature splicing processing is performed based on the first eigenvector, the second eigenvector, the third eigenvector, the first eigentensor, the second eigentensor and the third eigentensor of the image to be detected to determine a feature sequence to be detected of the image to be detected.

[0165] See also Figure 6 , Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 6 As shown in FIG, the electronic device 600 includes a processor 610 , a memory 620 and a bus 630 .

[0166] The memory 620 stores machine-readable instructions executable by the processor 610. When the electronic device 600 is running, the processor 610 communicates with the memory 620 via the bus 630. When the machine-readable instructions are executed by the processor 610, the above-mentioned Figures 1 to 3 The specific implementation of the steps in the method embodiment shown can be found in the method embodiment and will not be repeated here.

[0167] The embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the computer program can execute the above-mentioned Figures 1 to 3 The specific implementation of the steps in the method embodiment shown can be found in the method embodiment and will not be repeated here.

[0168] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0169] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interface, the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.

[0170] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0171] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0172] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0173] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The scope of protection of the present application is not limited thereto. Although the present application has been described in detail with reference to the above-mentioned embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-mentioned embodiments within the technical scope disclosed in the present application, or perform equivalent replacements for some of the technical features thereof. However, these modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A method for identifying plant diseases and insect pests, characterized in that: The identification method comprises: Acquire an image of the target plant to be detected; The image to be detected is input into a pre-trained disease symptom recognition model to determine multiple target symptom description labels of the image to be detected; the multiple target symptom description labels include at least one description label of the symptom site, at least one description label of the symptom morphology and at least one description label of the symptom color; the training method of the disease symptom recognition model is: using a large-scale, low-precision first training data set to pre-train the disease symptom recognition neural network, and using a small-scale, high-precision second training data set to fine-tune the pre-trained neural network; when annotating the images of the first training data set and the second training data set, the annotation is based on the description of the symptom node in the disease knowledge graph; the disease symptom recognition neural network is a self-attention neural network with a hybrid structure, and the self-attention neural network with a hybrid structure includes a self-attention neural network and a deep residual neural network; the deep residual neural network includes three target convolution layers, and the dimension of the feature map output by each target convolution layer is different; Based on the multiple target symptom description tags, determining at least one disease suffered by the target plant by searching the disease knowledge graph of the plant species to which the target plant belongs; the disease includes at least one of a disease or an insect pest; The multiple target symptom description labels of the image to be detected are determined by a feature sequence to be detected determined by identifying the image to be detected according to a disease symptom recognition model; Determine the feature sequence to be detected of the image to be detected by the following steps: Performing feature extraction processing on the image to be detected in the first target convolutional layer of the deep residual neural network input into the disease symptom recognition model to obtain a first feature map; Performing feature extraction processing on the image to be detected input into the second target convolutional layer of the deep residual neural network, and performing dimension conversion processing on the feature map after the feature extraction processing to obtain a second feature map; Performing feature extraction processing on the image to be detected input into the third target convolutional layer of the deep residual neural network, and performing dimensionality conversion processing on the feature map after the feature extraction processing to obtain a third feature map; Performing global average pooling processing on the first feature map, the second feature map, and the third feature map of the image to be detected, respectively, to determine the first eigenvector, the second eigenvector, and the third eigenvector of the image to be detected; Decoupling the first feature map, the second feature map, and the third feature map of the image to be detected in spatial dimensions respectively, and determining a first feature tensor, a second feature tensor, and a third feature tensor of the image to be detected; Feature splicing processing is performed based on the first eigenvector, the second eigenvector, the third eigenvector, the first eigentensor, the second eigentensor and the third eigentensor of the image to be detected to determine a feature sequence to be detected of the image to be detected.

2. The identification method according to claim 1, characterized in that The disease symptom recognition model is trained by the following steps: Obtaining a first training data set and a second training data set; the number of samples in the first training data set is greater than the number of samples in the second training data set, and the sample accuracy in the first training data set is lower than the sample accuracy in the second training data set; Pre-training a disease symptom recognition neural network using the first training data set until the disease symptom recognition neural network converges to obtain an initial disease symptom recognition model; The model parameters in the initial disease symptom recognition model are fine-tuned using the second training data set until the initial disease symptom recognition model converges to obtain a disease symptom recognition model.

3. The identification method according to claim 2, characterized in that Pre-training a disease symptom recognition neural network using the first training data set until the disease symptom recognition neural network converges to obtain an initial disease symptom recognition model, including: Inputting each training sample image in the first training data set into the pre-trained deep residual neural network to perform image feature processing to determine a feature sequence for each training sample image; the image feature processing includes at least one of the following processing methods: feature extraction processing, dimensionality conversion processing, global average pooling processing, feature decoupling processing, and feature splicing processing; The feature sequence of each training sample image is used as an input feature, and the disease symptom label of each training sample image is used as an output feature. The self-attention neural network is trained until the self-attention neural network converges to obtain an initial disease symptom recognition model; wherein each training sample image has multiple disease symptom labels.

4. The identification method according to claim 1, wherein: The disease knowledge graph of the plant species to which the target plant belongs is constructed by the following steps: Determine the plant species to which the target plant belongs as a category node of the disease knowledge graph to be constructed; Determining at least one disease suffered by the plant species to which the target plant belongs, and determining each disease as a disease node under a category node; For each disease, each symptom description of the disease is determined as a symptom node under the disease node corresponding to the disease; The category node is connected to each disease node, and each disease node is connected to its corresponding symptom node to construct a disease knowledge graph of the plant species to which the target plant belongs.

5. The identification method according to claim 2, characterized in that Inputting the image to be detected into a pre-trained disease symptom recognition model to determine multiple target symptom description labels of the image to be detected includes: Inputting the image to be detected into a pre-trained disease symptom recognition model, and determining a feature sequence to be detected of the image to be detected through a pre-trained deep residual neural network in the disease symptom recognition model; Identifying a feature sequence to be detected of the image to be detected by using the self-attention neural network in the disease symptom recognition model, and determining a probability value of each symptom description label of the image to be detected; The symptom description label with a probability value greater than a preset probability is determined as the target symptom description label of the image to be detected.

6. A device for identifying plant diseases and insect pests, characterized in that: The identification device comprises: An acquisition module, used for acquiring an image of a target plant to be detected; A label determination module is used to input the image to be detected into a pre-trained disease symptom recognition model to determine multiple target symptom description labels of the image to be detected; the multiple target symptom description labels include at least one description label of the symptom site, at least one description label of the symptom morphology, and at least one description label of the symptom color; the training method of the disease symptom recognition model is: using a large-scale, low-precision first training data set to pre-train the disease symptom recognition neural network, and using a small-scale, high-precision second training data set to fine-tune the pre-trained neural network; when annotating the images of the first training data set and the second training data set, the annotation is based on the description of the symptom node in the disease knowledge graph; the disease symptom recognition neural network is a self-attention neural network with a hybrid structure, and the self-attention neural network with a hybrid structure includes a self-attention neural network and a deep residual neural network; the deep residual neural network includes three target convolution layers, and the dimension of the feature map output by each target convolution layer is different; a disease determination module, configured to determine at least one disease suffered by the target plant by searching a disease knowledge graph of the plant species to which the target plant belongs based on the multiple target symptom description tags; the disease includes at least one of a disease or an insect pest; The multiple target symptom description labels of the image to be detected are determined by a feature sequence to be detected determined by identifying the image to be detected according to a disease symptom recognition model; The label determination module is further configured to determine a sequence of features to be detected of the image to be detected by performing the following steps: Performing feature extraction processing on the image to be detected in the first target convolutional layer of the deep residual neural network input into the disease symptom recognition model to obtain a first feature map; Performing feature extraction processing on the image to be detected input into the second target convolutional layer of the deep residual neural network, and performing dimension conversion processing on the feature map after the feature extraction processing to obtain a second feature map; Performing feature extraction processing on the image to be detected input into the third target convolutional layer of the deep residual neural network, and performing dimensionality conversion processing on the feature map after the feature extraction processing to obtain a third feature map; Performing global average pooling processing on the first feature map, the second feature map, and the third feature map of the image to be detected, respectively, to determine the first eigenvector, the second eigenvector, and the third eigenvector of the image to be detected; Decoupling the first feature map, the second feature map, and the third feature map of the image to be detected in spatial dimensions respectively, and determining a first feature tensor, a second feature tensor, and a third feature tensor of the image to be detected; Feature splicing processing is performed based on the first eigenvector, the second eigenvector, the third eigenvector, the first eigentensor, the second eigentensor and the third eigentensor of the image to be detected to determine a feature sequence to be detected of the image to be detected.

7. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and the machine-readable instructions are executed by the processor to execute the steps of the identification method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the identification method according to any one of claims 1 to 5 are executed.

Images