Method for training plant disease recognition model and related device

Abstract

The invention relates to a method for training a plant disease recognition model, the plant disease recognition model comprises a feature extraction network, a gating unit network and a feature coding network, the feature coding network comprises a plurality of expert network modules, and when the plant disease recognition model is trained, the expert network modules are connected with the feature extraction network. The original plant image is input into the feature extraction network to obtain an original feature map, the original feature map is input into the gating unit network to obtain at least two probability distributions, each probability distribution corresponds to one expert network module, and the at least two probability distributions are input into the corresponding expert network modules to obtain a plurality of intermediate feature maps; and splicing the plurality of intermediate feature maps into a target feature map, and using the target feature map to train a plant disease recognition model. Therefore, when the plant disease identification model is trained, each expert network module focuses on learning one disease feature, so that the obtained plant disease identification model can accurately identify the plant disease, and the accuracy of plant disease identification is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of plant disease recognition, and in particular to a method for training a plant disease recognition model, a plant disease recognition method, an electronic device and a storage medium. Background Art

[0002] Kiwifruit is a fruit with high nutritional value and great economic value, and is widely grown around the world. However, kiwifruit is susceptible to a variety of fungal, bacterial and viral diseases during its growth, such as gray mold, canker and leaf spot, which not only affect the quality and yield of the fruit, but can even cause the death of the entire fruit tree in severe cases. Traditional disease identification mainly relies on manual observation, which is time-consuming and labor-intensive, and requires high professional skills. It is difficult to quickly and accurately identify diseases in the early stages, and the best time for prevention and control is often missed.

[0003] In recent years, with the development of computer vision technology and deep learning algorithms, automated crop disease recognition systems have gradually become a research hotspot. Crop disease recognition systems can automatically identify disease types by analyzing photographed plant leaf or fruit images, providing a scientific basis for precise pesticide application and disease prevention and control. However, existing crop disease recognition systems have been applied to crop disease recognition, but there are still problems such as low accuracy in disease recognition and weak generalization ability. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a method for training a plant disease recognition model, a plant disease recognition method, an electronic device and a storage medium. The trained plant disease recognition model can accurately identify plant diseases, reduce the impact of unbalanced learning of plant disease samples, improve the accuracy of plant disease recognition, and improve the stability of practical applications.

[0005] To solve the above technical problems, the embodiments of the present invention provide the following technical solutions:

[0006] In a first aspect, an embodiment of the present invention provides a method for training a plant disease recognition model, wherein the plant disease recognition model includes a feature extraction network, a gating unit network, a feature encoding network, and a feature decoding network, wherein the feature encoding network includes a plurality of expert network modules;

[0007] The method comprises:

[0008] Acquire a training set, wherein the training set includes a plurality of original plant images containing diseases, the original plant images have real disease category labels of the diseases, and the diseases include multiple types;

[0009] Inputting the original plant image into the feature extraction network to obtain an original feature map;

[0010] Inputting the original feature map into the gated unit network to obtain at least two probability distributions, each of which corresponds to one of the expert network modules;

[0011] Inputting the at least two probability distributions into corresponding expert network modules respectively to obtain corresponding intermediate feature maps, and splicing the intermediate feature maps into a target feature map;

[0012] Inputting the target feature map into the feature decoding network to obtain a predicted disease category label of the disease;

[0013] The loss between the true disease category label and the predicted disease category label is calculated based on the loss function, and the plant disease recognition model is iteratively trained according to the loss until the plant disease recognition model converges to obtain a trained plant disease recognition model.

[0014] In some embodiments, the step of inputting the original plant image into the feature extraction network to obtain an original feature map includes:

[0015] Inputting the original plant image into the feature extraction network, equally dividing the original plant image to obtain a preset number of sub-original plant images, wherein both the original plant image and the sub-original plant images are three-dimensional images;

[0016] Converting a preset number of the sub-original plant images into a preset number of two-dimensional images;

[0017] The preset number of two-dimensional images are subjected to dimensionality reduction processing based on the following formula to obtain the original feature map:

[0018] E M*N =Image M*L *W L*N

[0019] Among them, Em M*N Represents the original feature map, Image M*L represents the preset number of two-dimensional images, W L*N Represents the weight parameters of the feature extraction network.

[0020] In some embodiments, before inputting the original plant image into the feature extraction network to obtain the original feature map, the method further includes:

[0021] Normalizing the original plant image to obtain a normalized original plant image;

[0022] The step of inputting the original plant image into the feature extraction network to obtain an original feature map comprises:

[0023] The normalized original plant image is input into the feature extraction network to obtain an original feature map.

[0024] In some embodiments, the gating unit network includes at least two of the following mathematical models, which are used to obtain the probability distribution:

[0025] G σ (x) = Softmax(x*W g +Softplus(x*W noise ))

[0026] Among them, G σ (x) represents the probability distribution, Softplus() and Softmax() both represent activation functions, x represents the original feature map, W g and W noise Represents the weight parameters of the gating unit network.

[0027] In some embodiments, the expert network module includes a multi-head attention mechanism layer and a forward layer, and the probability distribution is input into the expert network module to obtain the intermediate feature map, including:

[0028] Inputting the probability distribution into the multi-head attention mechanism layer to obtain a corresponding preliminary feature map;

[0029] The preliminary feature map is input into the forward layer to obtain the corresponding intermediate feature map.

[0030] In some embodiments, the feature encoding network further includes a summation and normalization layer, and the step of concatenating the plurality of intermediate feature maps into a target feature map includes:

[0031] Inputting the plurality of intermediate feature maps into the summing and normalization layer, summing the intermediate feature maps, and obtaining a concatenated feature map;

[0032] The concatenated feature map is normalized to obtain the target feature map.

[0033] In some embodiments, the model formula of the feature decoding network is:

[0034]

[0035] in, represents the predicted disease category label, Softmax() and s() both represent activation functions, the s() function is any one of the sigmoid function and the tanh function, h represents the number of hidden layers in the feature decoding network, W i and bi Respectively represent the weight and bias parameters of the i-th hidden layer, W 1 and b 1 represent the weights and bias parameters of the input layer in the feature decoding network, respectively, Represents the target feature map.

[0036] In some embodiments, the loss function is:

[0037]

[0038] Where y represents the real disease category label, represents the predicted disease category label, and They represent the probability distributions corresponding to the i-th and j-th expert network modules respectively, and K represents the number of types of the diseases.

[0039] In a second aspect, an embodiment of the present invention provides a plant disease identification method, comprising:

[0040] Acquire the plant image to be processed;

[0041] Inputting the to-be-processed plant image into a plant disease recognition model to obtain a predicted disease category label of the disease in the to-be-processed plant image, wherein the plant disease recognition model is trained using any one of the methods for training a plant disease recognition model proposed in the first aspect;

[0042] The type of disease in the plant image to be processed is obtained according to the predicted disease category label.

[0043] In a third aspect, an embodiment of the present invention provides an electronic device, including:

[0044] A processor and a memory communicatively connected to the processor;

[0045] The memory stores computer program instructions executable by the processor, and when the computer program instructions are called by the processor, the processor executes any one of the methods for training a plant disease recognition model proposed in the first aspect, or executes any one of the plant disease recognition methods proposed in the second aspect.

[0046] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which is stored computer program instructions executable by a processor, and when the computer program instructions are called by the processor, the processor executes any one of the methods for training a plant disease recognition model proposed in the first aspect, or executes any one of the plant disease recognition methods proposed in the second aspect.

[0047] Beneficial effects of the embodiments of the present invention: Different from the prior art, the method for training a plant disease recognition model provided by the embodiments of the present invention comprises a feature extraction network, a gating unit network, a feature encoding network and a feature decoding network, wherein the feature encoding network comprises several expert network modules; the method comprises: obtaining a training set, wherein the training set comprises a plurality of original plant images containing diseases, the original plant images have real disease category labels of the diseases, and the diseases comprise multiple types; inputting the original plant images into the feature extraction network to obtain an original feature map; inputting the original feature map into the gating unit network to obtain at least two probability distributions, each probability distribution corresponding to an expert network module; inputting at least two probability distributions into the corresponding expert network modules respectively to obtain several corresponding intermediate feature maps, and splicing the several intermediate feature maps into a target feature map; inputting the target feature map into the feature decoding network to obtain a predicted disease category label of the disease; calculating the loss between the real disease category label and the predicted disease category label based on a loss function, and iteratively training the plant disease recognition model according to the loss until the plant disease recognition model converges to obtain a trained plant disease recognition model.

[0048] When training a plant disease recognition model, the embodiment of the present invention inputs the original plant image into the feature extraction network to obtain the original feature map, then inputs the original feature map into the gated unit network to obtain at least two probability distributions, each probability distribution corresponds to an expert network module, and the at least two probability distributions are respectively input into the corresponding expert network modules to obtain several intermediate feature maps, and the several intermediate feature maps are spliced ​​into a target feature map, and the target feature map is used to train the plant disease recognition model. Thus, when training the plant disease recognition model, more attention is paid to allowing each expert network module to focus on learning a disease feature without interfering with each other, thereby improving the accuracy of plant disease recognition. In addition, the imbalance of feature learning during the model training process caused by the imbalance of disease image samples can be reduced, so that the model can evenly learn samples of various plant diseases during the training process, thereby improving the accuracy of plant disease recognition with a small number of samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art description. Obviously, the drawings described below only illustrate certain embodiments of the present invention and should not be regarded as limiting the scope of protection. For those of ordinary skill in the art, other related drawings can also be obtained based on these drawings without creative work.

[0050] Figure 1 It is a schematic diagram of application scenarios of plant disease recognition model training and plant disease recognition provided by some embodiments of the present invention;

[0051] Figure 2 is a schematic diagram of the network structure of a plant disease recognition model provided by some embodiments of the present invention;

[0052] Figure 3 is a schematic diagram of the structure of an electronic device provided by some embodiments of the present invention;

[0053] Figure 4 is a flow chart of a method for training a plant disease recognition model provided by some embodiments of the present invention;

[0054] Figure 5 yes Figure 4 A schematic diagram of a sub-process of step S400 in the method for training a plant disease recognition model shown in the embodiment;

[0055] Figure 6 It is a schematic diagram of the flow chart of the plant disease identification method provided by some embodiments of the present invention. DETAILED DESCRIPTION

[0056] In order to make the purpose and advantages of the embodiments of the present invention easier to understand, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The detailed description of the embodiments of the present invention in the drawings below does not limit the scope of protection claimed by the present invention, but only represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0057] It should be noted that, if no conflict is constituted, the various technical features involved in the embodiments of the present invention described below can be combined with each other, and are all within the scope of protection of the present invention. In addition, although the functional modules are divided in the device or structural schematic diagram, and the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in a different order than the module division in the device, or in a different order than in the flow chart. In addition, the "first", "second", "third" and other similar expressions used herein do not limit the data and execution order, but are only for the purpose of convenience of explanation and to distinguish between the same or similar items with basically the same functions and effects, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features.

[0058] Unless otherwise defined, the technical terms and scientific terms used in this specification have the same meanings as those generally understood by those of ordinary skill in the art of the present invention. The terms used in this specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. It should be understood that the term "and / or" used in this specification includes any and all combinations of one or more of the related listed items.

[0059] To facilitate understanding of the method provided in the embodiment of the present invention, the terms involved in the embodiment of the present invention are first introduced:

[0060] 1. Neural Networks

[0061] A neural network can be composed of neural units, which can be specifically understood as a neural network with an input layer, a hidden layer, and an output layer. Generally speaking, the first layer is the input layer, the last layer is the output layer, and the layers in between are all hidden layers. Among them, a neural network with many hidden layers is called a deep neural network (DNN). The work of each layer in a neural network can be described by the mathematical expression y=a(W·x+b). From a physical level, the work of each layer in a neural network can be understood as completing the transformation from input space to output space (i.e., the row space to column space of a matrix) through five operations on the input space (a set of input vectors). These five operations include: 1. Dimensionality increase / reduction; 2. Zoom in / out; 3. Rotation; 4. Translation; 5. "Bending". Among them, operations 2 and 3 are completed by "W·x", operation 4 is completed by "+b", and operation 5 is implemented by "a()". The word "space" is used here because the classified object is not a single thing, but a class of things. Space refers to the collection of all individuals of this type of thing. Among them, W is the weight matrix of each layer of the neural network. Each value in the matrix represents the weight value of a neuron in this layer. The matrix W determines the spatial transformation from the input space to the output space described above, that is, the W of each layer of the neural network controls how to transform the space. The purpose of training a neural network is to eventually obtain the weight matrix of all layers of the trained neural network. Therefore, the training process of a neural network is essentially to learn how to control spatial transformation, and more specifically to learn the weight matrix.

[0062] It should be noted that in the embodiments of the present invention, the models used in the machine learning tasks are essentially neural networks. Common components in neural networks include convolutional layers, pooling layers, normalization layers, and inverse convolutional layers. By assembling these common components in neural networks, a model is designed. When the model parameters (i.e., the weight matrices of each layer) are determined so that the model error meets the preset conditions or the number of model parameters is adjusted to reach a preset threshold, the model converges.

[0063] The convolution layer is configured with multiple convolution kernels, each of which is set with a corresponding step size to perform convolution operations on the image. The purpose of the convolution operation is to extract different features of the input image. The first convolution layer may only extract some low-level features, such as edges, lines, and corners. Deeper convolution layers can iteratively extract more complex features from low-level features.

[0064] The deconvolution layer is used to map a low-dimensional space to a high-dimensional space while maintaining the connection relationship / pattern between them. The connection relationship here refers to the connection relationship during convolution. The deconvolution layer is configured with multiple convolution kernels, each of which is set with a corresponding step size to perform deconvolution operations on the image. Generally, the framework library for designing neural networks (such as the PyTorch library) has a built-in upsumple() function, which can be called to achieve low-dimensional to high-dimensional spatial mapping.

[0065] Pooling layers imitate the human visual system and can reduce the dimension of data or represent images with higher-level features. Common operations of pooling layers include: maximum pooling, mean pooling, random pooling, median pooling, and combined pooling. Generally speaking, pooling layers are periodically inserted between convolutional layers of neural networks to achieve dimensionality reduction.

[0066] The normalization layer is used to normalize all neurons in the intermediate layer to prevent gradient explosion and gradient disappearance.

[0067] 2. Loss Function

[0068] In the process of training a neural network, because we want the output of the neural network to be as close as possible to the value we really want to predict, we can compare the current predicted value of the network with the target value we really want, and then update the weight matrix of each layer of the neural network according to the difference between the two (there is usually an initialization process before the first update, that is, pre-configuring parameters for each layer in the neural network). For example, if the predicted value of the network is high, adjust the weight matrix to make it predict lower, and keep adjusting until the neural network can predict the target value we really want. Therefore, it is necessary to predefine "how to compare the difference between the predicted value and the target value", which is the loss function or objective function, which are important equations used to measure the difference between the predicted value and the target value. Among them, taking the loss function as an example, the higher the output value (loss) of the loss function, the greater the difference, so the training of the neural network becomes a process of minimizing this loss as much as possible.

[0069] 3. Expert Network Module

[0070] The expert network module is used to learn a feature in a targeted manner and is good at processing a specific type of data or a specific task. During the model training process, the input image data is assigned to different expert network modules by the gated unit network. The expert network modules selected by the gated unit network will generate corresponding outputs for the input image data. The outputs of each expert network module will finally be weighted and combined with the weights assigned to the ability of each expert network module to process the feature to form the final prediction result. The expert network module can be a small MLP or a complex LLM. During the model training process, the gated unit network is used to achieve "teaching students in accordance with their aptitude", and then the expert network modules are "learning from each other" during the reasoning process, which improves the diversity and flexibility of the model. By utilizing the fact that each expert network module can specialize in processing different tasks or different parts of the data, each expert network module can model different data distributions and patterns, thereby improving the accuracy and generalization ability of the model, and the model can better adapt to the complexity of the task.

[0071] The following describes an exemplary application of an electronic device for training a plant disease recognition model or for plant disease recognition provided by an embodiment of the present invention. It should be understood that the electronic device provided by an embodiment of the present invention can be a smart terminal of various suitable types, such as a laptop computer, a desktop computer, or a mobile device, with certain computing and control capabilities. In some embodiments, the electronic device can also be a server, for example, a server deployed in the cloud.

[0072] See also Figure 1 , Figure 1 The schematic diagram of the application scenario of plant disease recognition model training and plant disease recognition in some embodiments of the present invention is schematically shown. In this application scenario, the terminal 10 is connected to the server 20 via a network, and the network can be the Internet, an intranet, a local area network, a mobile communication network, or a combination thereof.

[0073] Specifically, when the terminal 10 is used to train the plant disease recognition model, the terminal 10 can be used to obtain training image data and build a plant disease recognition model. For example, a person skilled in the art can download the prepared training image data and build a network structure of the plant disease recognition model on the terminal 10, and can train the plant disease recognition model when the training image data is obtained. The training image data includes a plurality of original plant images containing diseases.

[0074] In some embodiments, the terminal 10 can locally execute the method for training a plant disease recognition model provided in an embodiment of the present invention to complete iterative training of the designed plant disease recognition model using training image data, determine the final model parameters, and thus the plant disease recognition model configures the final model parameters to obtain the plant disease recognition model.

[0075] In some embodiments, the terminal 10 can be connected to the server 20 through a network, and send the training image data and the constructed plant disease recognition model stored on the terminal 10 by a person skilled in the art to the server 20. The server 20 receives the training image data and the plant disease recognition model, and iteratively trains the plant disease recognition model using the training image data to determine the final model parameters, and then sends the final model parameters to the terminal 10. The terminal 10 receives and saves the final model parameters, so that the plant disease recognition model configures the final model parameters, that is, the plant disease recognition model is obtained. It is easy to understand that the aforementioned network can be a wide area network or a local area network, or a combination of the two.

[0076] When the terminal 10 is used for plant disease identification, the terminal 10 can be used to obtain image data to be identified or image data to be processed. For example, a person skilled in the art packages the image data to be identified or image data to be processed and sends it to the terminal 10 through the network, so that the terminal 10 obtains the image data to be identified or image data to be processed. For another example, a person skilled in the art can download the prepared image data to be identified or image data to be processed on the terminal 10 and store it in the disease image library of the terminal 10. The terminal 10 can select the image data to be identified or image data to be processed from the disease image library. In some embodiments, after the terminal 10 obtains the image data to be identified or image data to be processed, it sends it to the controller ( Figure 1 (not shown), thus, the controller uses the built-in plant disease recognition model to perform plant disease recognition on the image data to be recognized or the image data to be processed to obtain the plant disease recognition result.

[0077] In some embodiments, the terminal 10 locally executes the plant disease identification method provided by the embodiment of the present invention, provides plant disease identification services for users, inputs the image data to be identified or the image data to be processed into a trained plant disease identification model, and obtains a plant disease identification result.

[0078] In some embodiments, the terminal 10 may also send the image data to be identified or the image data to be processed to the server 20 through the network. After receiving the image data to be identified or the image data to be processed, the server 20 inputs the image data to be identified or the image data to be processed into the trained plant disease recognition model to obtain the plant disease recognition result, and sends the plant disease recognition result to the terminal 10. It is easy to understand that the aforementioned network may be a wide area network or a local area network, or a combination of the two.

[0079] It should be understood that Figure 1In the application scenario shown, the electronic device 100 is a desktop computer or a server, which does not impose any restrictions on the structure, type, quantity, etc. of the electronic device 100 in other application scenarios. For example, in some other application scenarios, the electronic device 100 can also be a vehicle terminal, a cloud server, a tablet computer, or other suitable types of devices.

[0080] See also Figure 2 , Figure 2 The network structure diagram of the plant disease recognition model in some embodiments of the present invention is schematically shown. The plant disease recognition model includes a feature extraction network, a gate control unit network, a feature encoding network and a feature decoding network. The feature encoding network includes several expert network modules (i.e. Figure 2 Modules X1, X2, ..., Xn).

[0081] Specifically, after obtaining the training image data, the original plant image in the training image data is input into the feature extraction network, and the features in the original plant image are extracted through the feature extraction network to obtain the original feature map. Then the original feature map is input into the gated unit network, and at least two probability distributions are obtained through calculation by the gated unit network, and each probability distribution corresponds to an expert network module. Then, at least two probability distributions are respectively input into the corresponding expert network modules, and several corresponding intermediate feature maps are obtained through learning of the expert network modules, and several intermediate feature maps are spliced ​​into a target feature map. After obtaining the target feature map, the target feature map is input into the feature decoding network, and the predicted disease category label f(y) of the disease is obtained through decoding and prediction by the feature decoding network.

[0082] In some embodiments, the expert network module includes: a multi-head attention mechanism layer (i.e., Figure 2 The network layer A shown in Figure 2 The feature encoding network also includes a summation and normalization layer (i.e. Figure 2 The network layer C shown in FIG. 1 ). For each expert network module, the probability distribution is input into the multi-head attention mechanism layer of the expert network module to obtain the corresponding preliminary feature map, and then the preliminary feature map is input into the forward layer of the expert network module to obtain the corresponding intermediate feature map. After obtaining the intermediate feature maps corresponding to several expert network modules, the several intermediate feature maps are input into the summation and normalization layer, and the concatenated feature map is obtained by summing the summation and normalization layer, and then the summation and normalization layer normalizes the concatenated feature map to obtain the target feature map.

[0083] In some embodiments, the original plant image has a real disease category label of the disease. After the predicted disease category label f(y) of the disease is predicted, the real disease category label and the predicted disease category label f(y) are substituted into the loss function, and the loss value between the real disease category label and the predicted disease category label f(y) is calculated. The plant disease recognition model is iteratively trained according to the loss value until the plant disease recognition model converges, thereby obtaining a trained plant disease recognition model. The trained plant disease recognition model can accurately identify plant diseases and improve the accuracy of plant disease recognition.

[0084] In order to facilitate understanding of the method for training a plant disease recognition model and the plant disease recognition method provided by the embodiment of the present invention, the electronic device provided by the embodiment of the present invention is first introduced in detail.

[0085] See also Figure 3 , Figure 3 The structural diagram of the electronic device 100 provided in some embodiments of the present invention is schematically shown.

[0086] like Figure 3 As shown, the electronic device 100 includes at least one processor 110 and a memory 120 ( Figure 3 In the example of a bus system connection and a processor, the components in the electronic device 100 are coupled together through the bus system 130, and the bus system 130 is used to realize the connection and communication between these components. It is easy to understand that the bus system 130 includes not only a data bus, but also a power bus, a control bus, and a status signal bus. However, for the sake of clarity and brevity, Figure 3 In the figure, various buses are labeled as bus system 130. It can be understood that Figure 3 The structure shown in the embodiment is only for illustration and does not impose any limitation on the structure of the electronic device 100. For example, the electronic device 100 may further include: Figure 3 More or fewer components as shown, or with Figure 3 Different configurations are shown.

[0087] Specifically, the processor 110 is used to provide computing and control capabilities to control the electronic device 100 to perform corresponding tasks. For example, the electronic device 100 is controlled to perform any of the methods for training a plant disease recognition model and the plant disease recognition method provided in the embodiments of the present invention. Those skilled in the art will appreciate that the processor 110 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0088] The memory 120, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer executable programs, instructions and modules, for example, the method for training a plant disease recognition model and the program, instruction and module corresponding to the plant disease recognition method in the embodiment of the present invention. In some embodiments, the memory 120 may include a program storage area and a data storage area, and the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the processor 110, etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the non-transitory software programs, instructions and modules stored in the memory 120, so as to implement any method for training a plant disease recognition model and a plant disease recognition method provided in the embodiment of the present invention. In some embodiments, the memory 120 may include a high-speed random access memory, and may also include a non-transitory memory. For example, at least one disk storage device, a flash memory device or other non-transitory solid-state storage device. In some embodiments, the memory 120 may also include a memory remotely arranged relative to the processor 110, and these remote memories may be connected to the processor 110 via a communication network. It can be understood that examples of the above-mentioned communication network include but are not limited to the Internet, corporate intranet, local area network, mobile communication network and combinations thereof.

[0089] It is easy to understand that the implementation subject of any of the methods for training plant disease recognition models and plant disease recognition methods provided in the embodiments of the present invention can be any suitable type of electronic device with certain computing and control capabilities. In some possible implementations, any of the methods for training plant disease recognition models and plant disease recognition methods provided in the embodiments of the present invention can be implemented by a processor calling computer program instructions stored in a memory.

[0090] It can be understood from the above that the method for training a plant disease recognition model and the method for identifying a plant disease provided by the embodiments of the present invention can be implemented by various suitable types of electronic devices with certain computing and control capabilities, for example, they can be implemented by the above electronic devices. The method for training a plant disease recognition model and the method for identifying a plant disease provided by the embodiments of the present invention will be described in detail below in conjunction with the exemplary application and implementation of the electronic device provided by the embodiments of the present invention.

[0091] First, a method for training a plant disease recognition model provided by an embodiment of the present invention is described in detail.

[0092] See also Figure 4 , Figure 4 The flowchart of the method for training a plant disease recognition model provided by some embodiments of the present invention is schematically shown.

[0093] Those skilled in the art will appreciate that the method for training a plant disease recognition model provided in the embodiment of the present invention can be applied to the above-mentioned electronic device (e.g., terminal 10 or server 20). Specifically, the execution subject of the method for training a plant disease recognition model is one or at least two processors of the electronic device.

[0094] like Figure 4 As shown, the method for training a plant disease recognition model includes but is not limited to the following steps S100-S600:

[0095] S100: Obtain a training set, wherein the training set includes a plurality of original plant images containing diseases, the original plant images have real disease category labels of the diseases, and the diseases include multiple types.

[0096] Specifically, a user or a trainer can collect multiple different original plant images from various data sources, and combine the multiple original plant images into an image data set for training a plant disease recognition model. The multiple original plant images are images of the same plant. It can be understood that the method for training a plant disease recognition model is applicable to identifying diseases in any original plant image, especially for identifying diseases of the same plant. Obviously, each original plant image includes one or more types of diseases, and the diseases in the original plant image can be one or more.

[0097] It should be understood that in order to improve the efficiency of model training and the accuracy of the model output prediction disease category label, multiple training sets can be collected, each training set includes an original plant image, that is, multiple original plant images in each training set belong to the same plant. In any training set, the multiple original plant images in the training set include multiple types of diseases.

[0098] After obtaining the original plant image data for training the plant disease recognition model, the original plant images are annotated according to the diseases in the original plant images, and each original plant image is annotated with a real disease category label. The real disease category labels are manually classified and annotated by professionals according to the characteristics of plant diseases to ensure the accuracy and reliability of the annotations. In some embodiments, automatic annotation technology can also be used to annotate the original plant images with real disease category labels.

[0099] After annotating the original plant images with true disease category labels, the original plant images are assigned a specific category or label to identify the category or type of the disease in the original plant images, so that each original plant image in the training set has a corresponding true disease category label.

[0100] Obviously, when the original plant image data is obtained as a training set, the real disease category labels annotated on the original plant images are all the real labels of the diseases in the original plant images, and the original plant images are input into the plant disease recognition model, and the recognition results obtained are the predicted disease category labels output after the disease recognition of the original plant images.

[0101] S200: Input the original plant image into the feature extraction network to obtain the original feature map.

[0102] Among them, the feature extraction network is a deep convolutional neural network (for example, ResNet, VGG, Inception and other networks) that is specifically used to extract useful features from the input image. Convolutional neural network is a deep learning network architecture suitable for image processing tasks. It consists of multiple convolutional layers and pooling layers. By using convolutional operations such as convolutional layers and pooling layers, it can effectively extract local and global features in the original plant image. In the disease recognition task, the convolutional neural network, as a feature extractor, can learn different feature information at different levels in the original plant image to capture the important feature information of the original plant image.

[0103] Specifically, after obtaining the original plant image, since the original plant image collected from the data source is usually a real plant image that has not been processed or modified, the obtained original plant image needs to be preprocessed and cleaned, including operations such as resizing, data enhancement, pixel value normalization and standardization processing, to ensure that the original plant image meets the input requirements of the feature extraction network.

[0104] The processed original plant image is then input into the feature extraction network so that the feature extraction network can learn abstract feature representations from the original plant image and output an original feature map corresponding to the original plant image.

[0105] It is easy to understand that in order to extract the features of the original plant image, the feature extraction network can use multiple convolution kernels and pooling kernels to capture the details and overall feature information of the original plant image at different levels through multiple convolution, pooling and activation operations, and gradually abstract the features of the original plant image into a high-level representation, thereby extracting the original feature map of the original plant image and outputting the original feature map.

[0106] In some embodiments, after the original plant image is acquired, since the original plant image collected from the data source is usually a real plant image that has not been processed or modified, the acquired original plant image needs to be preprocessed and cleaned, including operations such as resizing, data enhancement, and normalization, to ensure that the original plant image meets the input requirements of the feature extraction network before the original plant image is input into the feature extraction network to obtain the original feature map.

[0107] In some embodiments, before the original plant image is input into the feature extraction network to obtain the original feature map, the method for training the plant disease recognition model further includes but is not limited to the following steps S201:

[0108] S201: normalizing the original plant image to obtain a normalized original plant image.

[0109] Among them, in order to speed up the convergence of the plant disease recognition model, the original plant images obtained need to be normalized.

[0110] Specifically, the original plant image is read from the training set, wherein the original plant image is usually stored in the form of pixel values, each pixel value is between 0 and 255.

[0111] First, the original plant image is enlarged, reduced or cropped to set the size of the original plant image to 64*64. Then the pixel values ​​of all pixels in the original plant image are converted to values ​​between 0 and 1 to enhance the training effect and stability of the model, thereby obtaining the normalized original plant image.

[0112] In some embodiments, the pixel values ​​of the pixels of the original plant image may be converted to between 0 and 1 by the following formula:

[0113]

[0114] Among them, n i represents the normalized pixel value of the ith pixel, x i represents the pixel value of the i-th pixel in the original plant image, and max(x) and min(x) represent the maximum and minimum pixel values ​​of all pixels in the original plant image, respectively. Substituting the pixel value of each pixel into the above formula, the pixel value of each pixel can be converted to between 0 and 1, thereby obtaining the normalized original plant image.

[0115] Correspondingly, in some embodiments, the original plant image is input into the feature extraction network to obtain the original feature map, which specifically includes but is not limited to the following steps S202:

[0116] S202: Input the normalized original plant image into a feature extraction network to obtain an original feature map.

[0117] Specifically, the normalized original plant image is used as input, that is, the normalized original plant image is input into the feature extraction network. Then, the feature extraction network processes the normalized original plant image, extracts and learns the high-level features of the original plant image layer by layer, and finally generates the corresponding original feature map, and outputs the original feature map of the original plant image.

[0118] In some embodiments, the dimension of the original plant image is higher than the dimension specified by the feature extraction network, so the original plant image needs to be segmented, transformed and dimensionally reduced to obtain the original feature map.

[0119] Exemplarily, in some embodiments, the original plant image is input into the feature extraction network to obtain the original feature map, which specifically includes but is not limited to the following steps S210-S230:

[0120] S210: Input the original plant image into a feature extraction network, divide the original plant image equally, and obtain a preset number of sub-original plant images, where both the original plant image and the sub-original plant images are three-dimensional images.

[0121] Specifically, after the normalized original plant image is input into the feature extraction network, the normalized original plant image is equally divided using the split_image function to obtain a preset number of sub-original plant images. It should be understood that each sub-original plant image is a part of the original plant image, and the sub-original plant images have the same size, and the original plant image and the sub-original plant images are both three-dimensional images, that is, the original plant image and the sub-original plant images both include three dimensions: height, width and channel.

[0122] In some embodiments, the preset number is 16, that is, the normalized original plant image is equally divided into 16 sub-original plant images. The size of the normalized original plant image is 64*64, each normalized original plant image can be represented as 64*64*3, and each sub-original plant image obtained by equal division can be represented as 16*16*3. It can be understood that those skilled in the art can adjust the preset number of equal divisions according to actual needs. For example, the preset number can also be 4, and the embodiment of the present invention does not impose any limitation on this.

[0123] S220: Convert a preset number of sub-original plant images into a preset number of two-dimensional images.

[0124] Specifically, for each sub-original plant image, the sub-original plant image including height, width and channel is converted into a two-dimensional image including height and width, that is, the information of the channel dimension is fused or compressed onto a two-dimensional plane to obtain a two-dimensional image, thereby converting a preset number of sub-original plant images one by one to obtain a preset number of two-dimensional images.

[0125] In some embodiments, for each sub-original plant image, the channel dimension information of the sub-original plant image can be fused or compressed into a two-dimensional plane using an average value, maximum value or flattening operation to obtain a two-dimensional image corresponding to the sub-original plant image.

[0126] As in the above embodiment, each sub-original plant image is 16*16*3. After dimensional conversion, one two-dimensional image is represented as 1*768, and 16 two-dimensional images can be represented as 16*768.

[0127] S230: Perform dimensionality reduction processing on a preset number of two-dimensional images based on the following formula to obtain an original feature map:

[0128] E M*N =Image M*L *W L*N

[0129] Among them, Em M*N Represents the original feature map, Image M*LRepresents a preset number of two-dimensional images, W L*N represents the weight parameter of the feature extraction network, W L*N Related to the preset number of two-dimensional images, the model will be initialized before training. L*N , and as the training progresses, W L*N Changes will also be updated until an effective plant disease recognition model is trained.

[0130] Specifically, the obtained preset number of two-dimensional images are input into the above formula. After the calculation and dimensionality reduction processing of the formula, the preset number of two-dimensional images are mapped to the dimension specified by the feature extraction network. Then, the feature extraction network processes the preset number of two-dimensional images mapped to the specified dimension, extracts and learns the high-level features of the preset number of two-dimensional images layer by layer, generates the corresponding original feature map, and outputs the original feature map corresponding to the original plant image.

[0131] As in the above embodiment, 16 two-dimensional images are 16*768, that is, Image M*L Image 16*768 , and assume that the dimension specified by the feature extraction network is 16*96, that is, the original feature map Em M*N For Em 16*96 , then initialize the setting W L*N W 768*96 The feature extraction network maps the image to the specified dimension. 16*768 Process, extract and learn Image layer by layer 16*768 High-level features of the corresponding original feature map Em 16*96 .

[0132] It should be understood that other methods (e.g., principal component analysis (PCA)) may be used to reduce the dimension of the two-dimensional image to obtain the original feature map. Those skilled in the art may select any suitable method, approach or other technology to reduce the dimension of the two-dimensional image to obtain the original feature map according to actual needs, and the embodiment of the present invention does not impose any limitation on this.

[0133] S300: Inputting the original feature map into the gated unit network to obtain at least two probability distributions, each probability distribution corresponding to an expert network module.

[0134] Among them, the gated unit network includes several fully connected layers (or dense layers). The gated unit network is used to generate the probability distribution of different expert network modules according to the input original feature map, and the probability distribution is used to guide the subsequent expert network module selection and feature processing. In some embodiments, the gated unit network models the probability distribution through the Softmax() gating function to obtain the probability distribution corresponding to different expert network modules.

[0135] Specifically, the original feature map output from the feature extraction network is input into the gated unit network. The fully connected layer of the gated unit network gradually processes and transforms the input original feature map. After each fully connected layer, a nonlinear activation function (e.g., ReLU function) is applied to introduce nonlinear characteristics.

[0136] The last layer of the gated unit network is a Softmax layer. After being processed by several fully connected layers and activation functions, the Softmax layer converts the features of the processed output into at least two probability distributions, each of which corresponds to an expert network module, and each probability distribution represents the probability of an expert network module being selected. The sum of at least two probability distributions is 1.

[0137] For example, in some embodiments, the feature encoding network includes N expert network modules, then the gating unit network will output N probability distributions, the sum of the N probability distributions is 1, and the N probability distributions correspond one-to-one to the N expert network modules.

[0138] In some embodiments, the gating unit network includes: at least two first mathematical models, the first mathematical models are used to obtain probability distribution, and each first mathematical model is used to obtain the probability distribution corresponding to an expert network module.

[0139] Among them, the formula of the first mathematical model is:

[0140] G σ (x) = Softmax(x*W g +Softplus(x*W noise ))

[0141] Among them, G σ (x) represents the probability distribution, x represents the original feature map of the input, Softplus() and Softmax() both represent activation functions used in neural networks and machine learning, and the formula of the Softplus() function is Softplus(y)=log(1+e y ), y represents the input of the Softplus() function. The Softmax() function is an activation function used for multi-classification problems. It accepts a feature vector and converts the feature vector into a probability distribution, where the sum of all probability distributions obtained by the Softmax() function is 1.

[0142] W g and W noise represents the weight parameter of the gated unit network. Specifically, W g represents the selection weight of the gate control unit network for each expert network module, W noiserepresents the learning weight of the gating unit network for each type or category of disease features learned by each expert network module, where W noise The initial value of can be determined according to the distribution law of diseases in the original plant image.

[0143] Specifically, the original feature map is input into the gated unit network, and the input original feature map x is combined with the weight matrix W g Multiply to get the first linear transformation result. Similarly, the original feature map x is multiplied by the weight matrix W noise The second linear transformation result is obtained by multiplication, and the third linear transformation result is obtained by using the Softplus() function to calculate the second linear transformation result. The first linear transformation result is added to the third linear transformation result to obtain the fourth linear transformation result, and then the fourth linear transformation result is converted into a probability distribution using the Softmax() function.

[0144] S400: Inputting at least two probability distributions into corresponding expert network modules respectively, obtaining a corresponding number of intermediate feature maps, and splicing the several intermediate feature maps into a target feature map.

[0145] Each expert network module is an independent neural network that focuses on extracting and learning the characteristics of a type or category of plant diseases.

[0146] Specifically, the at least two probability distributions obtained are respectively input into the corresponding expert network modules, each expert network module processes the input probability distribution, learns the characteristics of a plant disease corresponding to the expert network module in the original feature map, generates an intermediate feature map, and thus obtains several intermediate feature maps corresponding to the expert network modules, that is, obtains several intermediate feature maps. Among them, each expert network module corresponds to an intermediate feature map, and the obtained several intermediate feature maps represent the feature extraction results of the original feature map in several different expert network modules.

[0147] After obtaining several intermediate feature maps, the intermediate feature maps are connected according to the channel dimension, and each channel is the superposition of the corresponding channels of each intermediate feature map. That is, the intermediate feature maps generated by several expert network modules are spliced ​​in the depth direction (channel dimension), and the corresponding channels of each intermediate feature map are connected to form a target feature map with richer and more comprehensive feature representation.

[0148] In some embodiments, the expert network module includes a multi-head attention mechanism layer and a forward layer. The multi-head attention mechanism layer is used to preliminarily extract features to obtain a preliminary feature map of each expert network module, and the forward layer is used to further extract features to obtain an intermediate feature map of each expert network module.

[0149] In some embodiments, for each expert network module, the probability distribution is input into the expert network module to obtain an intermediate feature map, which specifically includes but is not limited to the following steps S401-S402:

[0150] S401: Input the probability distribution into the multi-head attention mechanism layer to obtain the corresponding preliminary feature map.

[0151] Specifically, for each expert network module, the probability distribution is input into the multi-head attention mechanism layer of the expert network module. The multi-head attention mechanism layer performs weighted calculation on the input probability distribution to generate attention outputs under different heads. The multi-head attention mechanism layer splices the outputs of different heads to obtain a preliminary feature map.

[0152] In some embodiments, the model formula of the multi-head attention mechanism layer is:

[0153] Q=G σ (x)*W Q

[0154] K=G σ (x)*W K

[0155] V=G σ (x)*W V

[0156]

[0157] Among them, G σ (x) represents the probability distribution of the input, W Q , W K , W V Denote the weight matrices of the query vector, key vector, and value vector of the multi-head attention mechanism layer, Q, K, and V denote the query vector, key vector, and value vector, respectively. Softmax() denotes the activation function. k represents the dimension of the key vector, and X represents the preliminary feature map to be obtained.

[0158] For each expert network module, after the probability distribution is input into the multi-head attention mechanism layer of the expert network module, the preliminary feature map corresponding to the expert network module is obtained through calculation of the multi-head attention mechanism layer.

[0159] S402: Input the preliminary feature map into the forward layer to obtain the corresponding intermediate feature map.

[0160] Specifically, for each expert network module, after the probability distribution is input into the multi-head attention mechanism layer of the expert network module to obtain the preliminary feature map, the preliminary feature map is input into the forward layer of the expert network module. The forward layer performs further linear transformation and activation function processing on the preliminary feature map, learns and extracts higher-level features in the preliminary feature map, and outputs the final intermediate feature map.

[0161] In some embodiments, the model formula of the forward layer is:

[0162] FFN i (X) = τ i (max(0,X*W1+b1)*W2+b2+X)

[0163] Among them, X represents the preliminary feature map of the input, W1 and W2 represent the weight matrix of the forward layer, b1 and b2 represent the bias parameters of the forward layer, and τ i represents the weight of the i-th expert network module, max() represents the maximum function, K represents the number of disease types in the original plant image, FFN i (X) represents the intermediate feature map corresponding to the i-th expert network module.

[0164] The forward layer consists of two fully connected layers and a residual connection. The residual connection adds the input directly to the output of the forward layer. The fully connected layer includes a linear transformation and an activation function, where the linear transformation multiplies the input preliminary feature map X by a weight matrix W1 and adds a bias parameter b1. The result is then input to the activation function ReLU for nonlinear transformation.

[0165] For each expert network module, after the preliminary feature map is input into the forward layer of the expert network module, the intermediate feature map corresponding to the expert network module is obtained through calculation of the forward layer.

[0166] In some embodiments, the feature encoding network further includes a summation and normalization layer, which is used to sum and normalize each intermediate feature map of several expert network modules to obtain a target feature map.

[0167] See also Figure 5 , Figure 5 A sub-flow diagram of step S400 in the plant disease identification method provided by some embodiments of the present invention is schematically shown.

[0168] like Figure 5 As shown, several intermediate feature maps are spliced ​​into a target feature map, specifically including but not limited to the following steps S410-S420:

[0169] S410: Input several intermediate feature maps into the summing and normalization layer, sum each intermediate feature map, and obtain a concatenated feature map.

[0170] Specifically, several intermediate feature maps are input to the summation and normalization layer, which sums each intermediate feature map and merges all the intermediate feature maps. That is, the elements at the same position of all the intermediate feature maps are added together to obtain a concatenated feature map.

[0171] In some embodiments, the summation formula for the summation and normalization layers is:

[0172]

[0173] Among them, FFN i (X) represents the intermediate feature map corresponding to the i-th expert network module, K represents the number of disease types in the original plant image, and y1 represents the obtained spliced ​​feature map.

[0174] After several intermediate feature maps are input into the summation and normalization layer of the feature encoding network, a concatenated feature map is obtained after calculation of the summation and normalization layer.

[0175] It should be understood that those skilled in the art may also adopt any other suitable summation formula to sum each intermediate feature map according to actual needs, and the embodiments of the present invention do not impose any limitation on this.

[0176] S420: Normalize the concatenated feature map to obtain a target feature map.

[0177] Specifically, after the spliced ​​feature map is obtained by summing, the spliced ​​feature map is normalized to normalize the feature value of the spliced ​​feature map within a certain range, for example, between 0 and 1.

[0178] In some embodiments, the normalization formula for the summation and normalization layer is:

[0179]

[0180] in, represents the target feature map to be obtained, y1 represents the input concatenated feature map, and y 1min and 1max They respectively represent the minimum and maximum eigenvalues ​​in the concatenated feature map y1.

[0181] After several intermediate feature maps are input into the summation and normalization layer of the feature encoding network, a concatenated feature map is obtained after calculation of the summation and normalization layer.

[0182] It should be understood that those skilled in the art may also adopt any other suitable normalization formula to normalize the spliced ​​feature map according to actual needs, and the embodiments of the present invention do not impose any limitation on this.

[0183] S500: Input the target feature map into the feature decoding network to obtain the predicted disease category label of the disease.

[0184] Among them, the feature decoding network is used to decode the target feature map into a specific disease category label, and the feature decoding network includes multiple convolutional layers and fully connected layers, which are used to gradually reduce the dimension of the feature map and convert high-level features into classification results. In some embodiments, the feature decoding network also includes a deconvolution layer (transposed convolution layer) or an upsampling layer, which is used to gradually restore the resolution of the target feature map through operations such as deconvolution, upsampling, and skip connections, while retaining the semantic feature information, and outputting the predicted disease category label of the disease in the original plant image.

[0185] Specifically, after obtaining the target feature map, the target feature map is input into the feature decoding network. After being processed by the feature decoding network, it is gradually restored and combined into high-level semantic features, and a fixed-length feature vector is generated through a fully connected layer or a global average pooling layer. Then, the feature vector is input into a fully connected layer, and the number of neurons output is equal to the number of disease categories or types. Finally, a softmax layer is used to convert the feature vector into a probability distribution, representing the probability of each disease category, and the category with the highest probability is selected as the predicted disease category label.

[0186] In some embodiments, the feature decoding network includes an input layer, multiple hidden layers and an output layer, wherein the input layer includes: N neural units, the hidden layer includes: Q neural units, and the output layer includes: K neural units, where K is the number of disease types in the original plant image.

[0187] In some embodiments, the model formula of the feature decoding network is:

[0188]

[0189] in, represents the predicted disease category label, Softmax() and s() both represent activation functions, s() function is any one of sigmoid function and tanh function, h represents the number of hidden layers in the feature decoding network, W i and b i Represent the weight and bias parameters of the i-th hidden layer, W 1 and b 1 They represent the weights and bias parameters of the input layer in the feature decoding network, Represents the target feature map.

[0190] Specifically, the target feature map After inputting into the feature decoding network, the input layer calculates Then the calculation result z of the input layer 1 Apply activation function s() to calculate a 1 =s(z 1 ), the s() function can be any of the sigmoid function and the tanh function. Then, each hidden layer i calculates z i =b i +W i *a i-1 , and calculate the result z i Apply activation function s() to calculate a i =s(z i ), and get the output a of each hidden layer h .

[0191] Sum the outputs of all hidden layers, that is, calculate And the summation result z sum Apply the Softmax() function to calculate Get the final predicted disease category label.

[0192] S600: Calculate the loss between the real disease category label and the predicted disease category label based on the loss function, and iteratively train the plant disease recognition model according to the loss until the plant disease recognition model converges to obtain a trained plant disease recognition model.

[0193] Among them, the loss function includes cross-entropy loss, which is particularly suitable for classification problems. Cross-entropy loss can measure the difference between the true disease category label and the predicted disease category label.

[0194] Specifically, the predicted disease category label of the original plant image output by the plant disease recognition model is compared with the real disease category label, that is, the loss between the predicted disease category label and the real disease category label is calculated through a preset loss function. Then, according to the calculated loss function value between the predicted disease category label and the real disease category label, an optimization algorithm (such as gradient descent) is used to perform multiple optimization iterative training on the parameters of the plant disease recognition model, and the neural network parameters in the plant disease recognition model are continuously adjusted to reduce the loss function value, so that the plant disease recognition model is closer to the real disease category label and outputs a more accurate predicted disease category label.

[0195] As the iterative training proceeds, the plant disease recognition model will gradually converge, that is, the predicted disease category label of the plant disease recognition model gradually approaches the real disease category label. Obviously, when the plant disease recognition model reaches a certain convergence state (for example, the loss function value no longer decreases significantly), it can be considered that the model has good generalization ability on the original plant image data used for training, and can be applied to plant image data that has not been encountered, that is, a trained and effective plant disease recognition model is obtained.

[0196] In some embodiments, a cross entropy loss function or a Dice loss function can be used to calculate the loss value between the real disease category label and the predicted disease category label. Among them, the cross entropy loss function measures the difference between the predicted probability distribution and the real disease category label, and the Dice loss function measures the similarity between the two sets. It can be understood that the smaller the calculated loss function value is, the closer the prediction result of the plant disease recognition model is to the real disease category label, and the more optimized the training process is. It is easy to understand that according to actual needs, any other suitable loss function can be used to calculate the loss between the real disease category label and the predicted disease category label, and the embodiment of the present invention does not impose any limitation on this.

[0197] In some special cases, if all the original plant images are passed to only a few popular expert network modules, the gated unit network tends to mainly activate the popular expert network modules, and the model training efficiency will be reduced. Since the popular expert network modules are trained faster, this situation will be self-reinforcing, so they are more likely to be selected. If the plant disease samples are unbalanced, the model will tend to learn samples with a large number of diseases during the training process, learn less or ignore samples with a small number of diseases, resulting in low accuracy in disease recognition for diseases with a small number of diseases.

[0198] To alleviate this problem, an auxiliary loss is introduced when designing the loss function to encourage giving equal importance to all expert network modules. The auxiliary loss ensures that all expert network modules receive roughly equal numbers of training samples, thereby balancing the selection between expert network modules. In some embodiments, such as in the transformers library, the auxiliary loss can be controlled by the aux_loss parameter.

[0199] In some embodiments, the loss function is:

[0200]

[0201] Among them, y represents the true disease category label of the original plant image, represents the predicted disease category label output by the plant disease recognition model, and They represent the probability distributions corresponding to the i-th and j-th expert network modules respectively, K represents the number of disease types in all original plant images, and the right half of the loss function formula represents the auxiliary loss.

[0202] In the process of training the plant disease recognition model, the original plant image is input into the plant disease recognition model to obtain the predicted disease category label output by the plant disease recognition model. The predicted disease category label is then compared with the true disease category label, and the loss function value is calculated through the loss function. It is easy to understand that the weight parameters of the relevant neural network in the plant disease recognition model can be adjusted through the back propagation algorithm using an optimization algorithm (e.g., Adam, stochastic gradient descent SGD, etc.) to reduce or lower the value of the loss function, thereby optimizing the prediction ability of the plant disease recognition model.

[0203] In some embodiments, the Adam optimization algorithm can be used to optimize the weight parameters of the plant disease recognition model, wherein the number of iterations is set to 100,000 times, the initialization learning rate is set to 0.001, the weight decay is set to 0.0005, and the learning rate decays to 1 / 10 of the original every 50 iterations until the plant disease recognition model converges.

[0204] After multiple iterations of training, the loss function value of the plant disease recognition model gradually decreases, and the consistency between the predicted disease category labels output by the plant disease recognition model and the actual disease category labels gradually improves until the plant disease recognition model converges. When the plant disease recognition model reaches a convergence state, it means that the plant disease recognition model has reached a relatively stable state, and more training will not bring significant improvements and enhancements. It can be considered that the plant disease recognition model already has good generalization capabilities on the original plant image data used for training, and can be applied to plant image data that has not been encountered, that is, a trained and effective plant disease recognition model is obtained.

[0205] In summary, the method for training a plant disease recognition model provided by an embodiment of the present invention comprises a feature extraction network, a gating unit network, a feature encoding network and a feature decoding network, wherein the feature encoding network comprises several expert network modules; the method comprises: obtaining a training set, wherein the training set comprises a plurality of original plant images containing diseases, the original plant images have real disease category labels of the diseases, and the diseases comprise multiple types; inputting the original plant images into the feature extraction network to obtain an original feature map; inputting the original feature map into the gating unit network to obtain at least two probability distributions, each probability distribution corresponding to an expert network module; inputting at least two probability distributions into corresponding expert network modules respectively to obtain several corresponding intermediate feature maps, and splicing the several intermediate feature maps into a target feature map; inputting the target feature map into the feature decoding network to obtain a predicted disease category label of the disease; calculating the loss between the real disease category label and the predicted disease category label based on a loss function, and iteratively training the plant disease recognition model according to the loss until the plant disease recognition model converges to obtain a trained plant disease recognition model.

[0206] When training a plant disease recognition model, the embodiment of the present invention inputs the original plant image into the feature extraction network to obtain the original feature map, then inputs the original feature map into the gated unit network to obtain at least two probability distributions, each probability distribution corresponds to an expert network module, and the at least two probability distributions are respectively input into the corresponding expert network modules to obtain several intermediate feature maps, and the several intermediate feature maps are spliced ​​into a target feature map, and the target feature map is used to train the plant disease recognition model. Thus, when training the plant disease recognition model, more attention is paid to allowing each expert network module to focus on learning a disease feature without interfering with each other, thereby improving the accuracy of plant disease recognition. In addition, the imbalance of feature learning during the model training process caused by the imbalance of disease image samples can be reduced, so that the model can evenly learn samples of various plant diseases during the training process, thereby improving the accuracy of plant disease recognition with a small number of samples.

[0207] Next, the plant disease identification method provided by the embodiment of the present invention is described in detail.

[0208] See also Figure 6 , Figure 6 The flowchart of the plant disease identification method provided by some embodiments of the present invention is schematically shown.

[0209] It is understandable that the plant disease identification method provided in the embodiment of the present invention can be applied to the above electronic device (eg, the terminal 10 or the server 20). Specifically, the execution subject of the plant disease identification method is one or at least two processors of the electronic device.

[0210] like Figure 6 As shown, the plant disease identification method includes but is not limited to the following steps S10-S30:

[0211] S10: Acquire the plant image to be processed.

[0212] Specifically, when applying the plant disease recognition model to the plant images to be processed for plant disease recognition, it is first necessary to obtain the plant images to be processed. The plant images to be processed for image processing or computer vision tasks can be obtained by the user or operator from various data sources. The obtained plant images to be processed provide a reliable data source for subsequent image data preprocessing, recognition detection, feature extraction, plant disease recognition and other operations.

[0213] It can be understood that the plant image to be processed can be pixel data represented in digital form, or a plant image file in a format such as JPEG, PNG, etc., or a real-time plant image obtained by an image acquisition device or sensor such as a camera or scanner.

[0214] For example, in some embodiments, a high-resolution camera (such as a SLR camera, a mirrorless camera, or a professional plant disease detection camera, etc.) is used to photograph plants to obtain images of the plants to be processed, or a drone (equipped with a high-resolution camera) can be used to photograph plants from the air to obtain images of the plants to be processed, or a scanner or special plant scanning equipment can be used to scan the leaves or other parts of the plants to obtain high-resolution images of the plants to be processed, or a fixed sensor installed in a farmland or greenhouse can be used to regularly photograph plants to obtain images of the plants to be processed.

[0215] Of course, there may be any other suitable way to obtain the plant image to be processed, and those skilled in the art may adopt a suitable way according to actual needs, and the embodiment of the present invention does not impose any limitation on this.

[0216] S20: Inputting the plant image to be processed into a plant disease recognition model to obtain a predicted disease category label of the disease in the plant image to be processed, wherein the plant disease recognition model is trained using any one of the methods for training a plant disease recognition model provided in an embodiment of the present invention.

[0217] Among them, the plant disease recognition model in this embodiment is trained by any of the methods for training plant disease recognition models in the aforementioned embodiments. It has the same structure and function as the plant disease recognition model in the aforementioned embodiments, and will not be described one by one here.

[0218] Specifically, after obtaining the plant image to be processed, the obtained plant image to be processed is loaded as input data into the plant disease recognition model. The plant image to be processed is forward propagated through the plant disease recognition model, and the model classifies the diseases that may exist in the plant image to be processed into corresponding disease categories, thereby obtaining the predicted disease category labels of the diseases in the plant image to be processed. It can be understood that there may be more than one disease in the plant image to be processed, so the predicted disease category labels output by the plant disease recognition model may be more than one, wherein each predicted disease category label corresponds to a disease determined by identifying the plant image to be processed.

[0219] During the forward propagation process, each pixel in the plant image to be processed is input into the plant disease recognition model, and after a series of convolution, pooling, feature extraction, feature analysis and synthesis operations of the neural network in the plant disease recognition model, a multi-level feature representation is extracted from the input plant image to be processed. Based on the feature representation and weight parameters learned in the training phase, the plant disease recognition model outputs the corresponding predicted disease category label for each disease that may exist in the plant image to be processed, thereby obtaining one or more predicted disease category labels corresponding to the disease in the plant image to be processed.

[0220] S30: Obtain the type of disease in the plant image to be processed according to the predicted disease category label.

[0221] A mapping table is established in advance to match the predicted disease category labels with specific disease types one by one. The mapping table can be a dictionary, a database, or other suitable data structures. For example, in some embodiments, the mapping table is: "0-healthy", "1-gray mold", "2-ulcer disease" and "3-leaf spot disease".

[0222] Specifically, a predicted disease category label is extracted from the output of the plant disease recognition model. The predicted disease category label can be the name or code of each type of disease predicted by the plant disease recognition model. Then, the disease type corresponding to the predicted disease category label is searched in a mapping table established in advance. For example, if the predicted disease category label output by the plant disease recognition model is 1, the disease type found in the mapping table is "botrytis cinerea". After finding the specific disease type according to the mapping relationship in the mapping table, the disease type is used as the output result, that is, the disease type existing in the plant image to be processed is output, indicating that botrytis cinerea exists in the plant image to be processed.

[0223] To summarize, the plant disease identification method provided in the embodiment of the present invention includes: obtaining a plant image to be processed; inputting the plant image to be processed into a plant disease identification model to obtain a predicted disease category label of the disease in the plant image to be processed, wherein the plant disease identification model is trained using any one of the methods for training a plant disease identification model provided in the embodiment of the present invention; and obtaining the type of disease in the plant image to be processed based on the predicted disease category label.

[0224] In the embodiment of the present invention, plant diseases are identified using the trained plant disease recognition model, which can accurately identify plant diseases, reduce the impact of unbalanced learning of plant disease samples, improve the accuracy of plant disease recognition, and improve the stability of practical applications.

[0225] An embodiment of the present invention provides a computer-readable storage medium, on which computer program instructions executable by a processor are stored. When the computer program instructions are called by the processor, the processor executes any method for training a plant disease recognition model provided by the embodiment of the present invention, or executes any plant disease recognition method provided by the embodiment of the present invention.

[0226] In some embodiments, the storage medium may be a flash memory, a hard disk, an optical disk, a register, a magnetic surface storage, a removable disk, a CD-ROM, a random access memory (RAM), a read-only memory (ROM), an electrically programmable ROM, and an electrically erasable programmable ROM, or any other form of storage medium known in the technical field, or various devices including one or any combination of the above storage media.

[0227] In some embodiments, computer program instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0228] As an example, computer program instructions may, but do not necessarily, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, for example, in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0229] As an example, the computer program instructions may be deployed to be executed on a computing device (including devices such as smart terminals and servers), or on multiple computing devices located at one location, or on multiple computing devices distributed at multiple locations and interconnected by a communication network. It is easy to understand that all or part of the steps of the method described in the embodiments of the present invention may be directly implemented using electronic hardware or processor executable computer program instructions, or a combination of the two.

[0230] Those skilled in the art will appreciate that the embodiments provided by the present invention are only illustrative, and the order of writing the steps in the method of the embodiment does not mean a strict execution order and does not constitute any limitation on the implementation process. The order can be adjusted, merged and deleted according to actual needs, and the modules or sub-modules, units or sub-units in the device or system of the embodiment can be merged, divided and deleted according to actual needs. For example, the division of units is only a logical function division, and there can be other division methods in actual implementation. For another example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0231] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a general hardware platform, and of course can also be implemented by hardware. Those skilled in the art can understand that all or part of the processes in the above embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0232] It should be noted that the above embodiments are intended to illustrate the technical concept and features of the present invention, and their purpose is to enable people familiar with this technology to understand the content of the present invention and implement it accordingly, and cannot be used to limit the scope of protection of the present invention. Ordinary technicians in this field can understand that all or part of the processes of the above embodiments can be implemented according to the technical solutions recorded in the embodiments of the present invention, or some of the technical features can be equivalently replaced. It can be understood that these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and should be regarded as equal changes and modifications based on the embodiments of the present invention, and should all fall within the scope of the claims of the present invention.

Claims

1. A method for training a plant disease recognition model, characterized in that: The plant disease recognition model includes a feature extraction network, a gating unit network, a feature encoding network and a feature decoding network, wherein the feature encoding network includes several expert network modules; The method comprises: Acquire a training set, wherein the training set includes a plurality of original plant images containing diseases, the original plant images have real disease category labels of the diseases, and the diseases include multiple types; Inputting the original plant image into the feature extraction network to obtain an original feature map; Inputting the original feature map into the gated unit network to obtain at least two probability distributions, each of which corresponds to one of the expert network modules; Inputting the at least two probability distributions into corresponding expert network modules respectively to obtain corresponding intermediate feature maps, and splicing the intermediate feature maps into a target feature map; Inputting the target feature map into the feature decoding network to obtain a predicted disease category label of the disease; The loss between the true disease category label and the predicted disease category label is calculated based on the loss function, and the plant disease recognition model is iteratively trained according to the loss until the plant disease recognition model converges to obtain a trained plant disease recognition model.

2. The method according to claim 1, characterized in that The step of inputting the original plant image into the feature extraction network to obtain an original feature map comprises: Inputting the original plant image into the feature extraction network, equally dividing the original plant image to obtain a preset number of sub-original plant images, wherein both the original plant image and the sub-original plant images are three-dimensional images; Converting a preset number of the sub-original plant images into a preset number of two-dimensional images; The preset number of two-dimensional images are subjected to dimensionality reduction processing based on the following formula to obtain the original feature map: Em M*N =Image M*L *W L*N Among them, Em M*N Represents the original feature map, Image M*L represents the preset number of two-dimensional images, W L*N Represents the weight parameters of the feature extraction network.

3. The method according to claim 2, characterized in that Before inputting the original plant image into the feature extraction network to obtain the original feature map, the method further includes: Normalizing the original plant image to obtain a normalized original plant image; The step of inputting the original plant image into the feature extraction network to obtain an original feature map comprises: The normalized original plant image is input into the feature extraction network to obtain an original feature map.

4. The method according to claim 1, characterized in that: The gating unit network includes at least two of the following mathematical models, which are used to obtain the probability distribution: G σ (x)=Softmax(x*W g +Softplus(x*W noise )) Among them, G σ (x) represents the probability distribution, Softplus() and Softmax() both represent activation functions, x represents the original feature map, W g and W noise Represents the weight parameters of the gating unit network.

5. The method according to any one of claims 1 to 4, characterized in that: The expert network module includes a multi-head attention mechanism layer and a forward layer, and the probability distribution is input into the expert network module to obtain the intermediate feature map, including: Inputting the probability distribution into the multi-head attention mechanism layer to obtain a corresponding preliminary feature map; The preliminary feature map is input into the forward layer to obtain the corresponding intermediate feature map.

6. The method according to claim 5, characterized in that The feature encoding network further includes a summation and normalization layer, and the step of concatenating the plurality of intermediate feature maps into a target feature map includes: Inputting the plurality of intermediate feature maps into the summing and normalization layer, summing the intermediate feature maps, and obtaining a concatenated feature map; The concatenated feature map is normalized to obtain the target feature map.

7. The method according to any one of claims 1 to 4, characterized in that: The model formula of the feature decoding network is: in, represents the predicted disease category label, Softmax() and s() both represent activation functions, the s() function is any one of the sigmoid function and the tanh function, h represents the number of hidden layers in the feature decoding network, W i and b i Respectively represent the weight and bias parameters of the i-th hidden layer, W 1 and b 1 represent the weights and bias parameters of the input layer in the feature decoding network, respectively, Represents the target feature map.

8. A method for identifying plant diseases, characterized in that: include: Acquire the plant image to be processed; Inputting the to-be-processed plant image into a plant disease recognition model to obtain a predicted disease category label of the disease in the to-be-processed plant image, wherein the plant disease recognition model is trained using the method for training a plant disease recognition model according to any one of claims 1 to 7; The type of disease in the plant image to be processed is obtained according to the predicted disease category label.

9. An electronic device, characterized in that: include: A processor and a memory communicatively connected to the processor; The memory stores computer program instructions executable by the processor, and when the computer program instructions are called by the processor, the processor executes the method for training a plant disease recognition model as described in any one of claims 1 to 7, or executes the plant disease recognition method as described in claim 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions executable by a processor, and when the computer program instructions are called by the processor, the processor executes the method for training a plant disease identification model as described in any one of claims 1 to 7, or executes the plant disease identification method as described in claim 8.

Images