Crop leaf disease degree diagnosis method and system

Abstract

The invention discloses a crop leaf disease degree diagnosis method and system, and relates to the technical field of deep learning and agricultural information, and the method comprises the steps: obtaining a crop leaf image data set, carrying out the marking and preprocessing of the crop leaf image data set, and obtaining a processed crop leaf image data set; inputting the crop leaf image data set into a pre-established multi-scale double-attention mechanism U-Net model for training, and obtaining a trained multi-scale double-attention mechanism U-Net model; obtaining a to-be-recognized crop leaf image, inputting the to-be-recognized crop leaf image into the trained multi-scale double-attention mechanism U-Net model, outputting the classification probability of the obtained crop leaf image, and recognizing the crop leaf disease category according to the classification probability of the crop leaf image; and for the crop leaf image with the identified disease category, performing disease degree diagnosis calculation according to the percentage of the total area of leaf disease spots in the image to the total area of leaves to obtain a disease degree grade.

Description

Technical Field

[0001] The present invention relates to the field of deep learning and agricultural information technology, and specifically to a method and system for diagnosing the degree of crop leaf disease. Background Art

[0002] Crops are susceptible to a series of diseases during their growth. Leaf diseases of crops seriously affect crop yields. If not treated in time, they will reduce or even fail to produce, causing significant losses to the agricultural economy. Therefore, accurate diagnosis of the extent of crop diseases is the key to the prevention and control of crop diseases, and is of great significance to ensuring the quality and yield of crops. Traditional methods for diagnosing the extent of crop diseases mainly rely on manual observation and the experience of agricultural experts. These methods include: field observation, laboratory testing, empirical judgment, chemical control, etc. Although traditional methods are effective to a certain extent, they have obvious limitations, including being susceptible to subjective judgment, low efficiency, and difficulty in achieving rapid diagnosis over a large area.

[0003] In recent years, automated diagnosis methods based on image processing and deep learning have been increasingly widely used. There are currently two main methods: target-based region detection and semantic-based segmentation. Target detection focuses on the regional location of the disease, while semantic segmentation is a binary classification of pixels in the image to distinguish the edges of real and non-real target areas. From the perspective of labeling time, target detection takes less time, but it is difficult to accurately describe the actual area of ​​the target disease. With the development of smart agriculture, more sophisticated disease prevention and control measures are needed. Although segmentation labels are time-consuming, accurate segmentation results can spatially quantify the specific area of ​​the disease. However, traditional segmentation networks cannot solve the irregular and variable shapes of crop leaf diseases. In the process of model encoding, there is a lack of local information interaction, and the ability to extract features such as image edges and geometric shapes is weak, resulting in low segmentation accuracy when the model is segmenting leaves and lesions. And it is impossible to quantitatively assess the severity of the disease. Summary of the invention

[0004] In order to solve the deficiencies mentioned in the above background technology, the purpose of the present invention is to provide a method and system for diagnosing the degree of crop leaf disease, which realizes the recognition and degree diagnosis of crop leaf disease by constructing a multi-scale dual attention mechanism U-Net.

[0005] In a first aspect, the purpose of the present invention can be achieved by the following technical solution: a method for diagnosing the degree of disease on crop leaves, the method comprising the following steps:

[0006] Acquire a crop leaf image dataset, annotate and preprocess the crop leaf image dataset, and obtain a processed crop leaf image dataset;

[0007] The crop leaf image dataset is input into a pre-established multi-scale dual attention mechanism U-Net model for training, and a trained multi-scale dual attention mechanism U-Net model is obtained;

[0008] Obtain the crop leaf image to be identified, input the crop leaf image to be identified into the trained multi-scale dual-attention mechanism U-Net model, output the classification probability of the crop leaf image, and identify the crop leaf disease category according to the classification probability of the crop leaf image;

[0009] For crop leaf images with identified disease categories, the disease severity is diagnosed and calculated based on the percentage of the total area of ​​leaf lesions in the image to the total leaf area to obtain the disease severity level.

[0010] In combination with the first aspect, in certain implementations of the first aspect, the method further includes: the crop leaf image dataset is screened from three public plant disease datasets: Plant Village, Plant Doc, and Dataset of Tomato Leaves, wherein the crop leaf image dataset includes images with various diseases and images without diseases.

[0011] In combination with the first aspect, in some implementations of the first aspect, the method further includes: the process of labeling and preprocessing the crop leaf image dataset:

[0012] Labeling process: Use Labelme or other labeling software to perform pixel-level labeling on the leaf images in the crop leaf image dataset. During the labeling process, the background, leaf, and diseased pixels in the image are marked as 0, 1, and 2, respectively;

[0013] Preprocessing process: The leaf images in the crop leaf image dataset are enhanced through five techniques: image translation, rotation, flipping, brightness or contrast adjustment, and noise addition.

[0014] In combination with the first aspect, in certain implementations of the first aspect, the method further includes: the pre-established multi-scale dual attention mechanism U-Net model includes an encoder and a decoder;

[0015] In the encoder, each stage consists of two 3x3 convolutional layers, followed by ReLU activation and 2x2 maximum pooling operations for downsampling, and the number of feature channels is doubled in each downsampling process. In the decoder, each stage includes upsampling, 2x2 convolution, halving the number of feature channels, concatenating with the corresponding cropped feature map in the encoder, and then two 3x3 convolutions, each followed by a ReLU activation function.

[0016] In combination with the first aspect, in some implementations of the first aspect, the method further includes: the encoder performs multi-layer convolution and downsampling operations, adopts a skip connection strategy, directly passes a part of the feature map to a higher layer, and directly passes the other part to the decoder part through the skip connection as a partial input of the upsampling operation, and in the output stage, fuses the feature images of different scales to form a final result, and the output of the multi-scale feature fusion is shown in the following formula:

[0017] Z(x)=Conv([Z0,Z1,Z2,Z3,Z4])

[0018] Where conv() represents the convolution operation, Z(x) is the output after feature fusion, and Z0, Z1, Z2, Z3, and Z4 are the output feature maps sampled at different scales.

[0019] In combination with the first aspect, in some implementations of the first aspect, the method further includes: executing a position attention module after upsampling operation of each layer in the decoding stage of the decoder, and executing a channel attention module before skip connection of each layer;

[0020] Position attention module structure: The position feature map P∈R^(H×W×C) is reshaped to obtain PB∈R^((H×W)×C) and PC∈R^(C×(H×W)), PB and PC are obtained by matrix multiplication to obtain PS∈R^((H×W)×(H×W)), PF∈R^(H×W×C) is obtained by matrix multiplication and reshaping of PS and PB, and finally PF and P are subjected to matrix addition and batch normalization operations to obtain the feature map PE∈R^(H×W×C);

[0021] Channel attention module structure: The channel feature map C∈R^(H×W×C) is reshaped to obtain CB∈R^((H×W)×C) and CC∈R^(C×(H×W)), CB and CC are obtained by matrix multiplication to obtain CS∈R^(C×C), CF∈R^(H×W×C) is obtained by matrix multiplication and reshaping of CS and CB, and finally CF and C are subjected to matrix addition and batch normalization operations to obtain the final feature map CE∈R^(H×W×C);

[0022] Concatenate the feature maps PE and CE as the input features of the next layer in the decoding stage;

[0023] The bottleneck layer contains two convolutional layers, each of which has multiple convolution kernels. Each convolutional layer is a convolutional layer that expands the dimension of the input image. A BN layer is added after each convolutional layer, and a Dropout layer is added after the second BN layer and before upsampling to regularize the output image.

[0024] In combination with the first aspect, in certain implementations of the first aspect, the method further includes: the calculation formula for diagnosing the disease degree based on the percentage of the total area of ​​leaf spots to the total area of ​​the leaf is as follows:

[0025]

[0026] Where, M is the percentage of the total area of ​​lesions to the total area of ​​leaves, N lesion Represented as the number of diseased spot pixels in the image, N leaf Represented as the number of pixels of crop leaves in the image.

[0027] In a second aspect, in order to achieve the above-mentioned object, the present invention discloses a crop leaf disease degree diagnosis system, comprising:

[0028] An image processing module is used to obtain a crop leaf image dataset, annotate and preprocess the crop leaf image dataset, and obtain a processed crop leaf image dataset;

[0029] A model training module is used to input the crop leaf image dataset into a pre-established multi-scale dual-attention mechanism U-Net model for training, thereby obtaining a trained multi-scale dual-attention mechanism U-Net model;

[0030] The disease recognition module is used to obtain the crop leaf image to be identified, input the crop leaf image to be identified into the trained multi-scale dual-attention mechanism U-Net model, output the classification probability of the crop leaf image, and identify the crop leaf disease category according to the classification probability of the crop leaf image;

[0031] The disease diagnosis module is used to diagnose and calculate the degree of disease based on the percentage of the total area of ​​leaf spots to the total area of ​​the leaf according to the harmful leaves obtained according to the crop leaf image category, and obtain the disease degree level.

[0032] In conjunction with the second aspect, in certain implementations of the second aspect, the system further includes: the crop leaf image dataset of the image processing module is screened from three public plant disease datasets: Plant Village, Plant Doc, and Dataset of Tomato Leaves, wherein the crop leaf image dataset includes images with various diseases and images without diseases;

[0033] The image processing module labels and preprocesses the crop leaf image dataset:

[0034] Labeling process: Use Labelme or other labeling software to perform pixel-level labeling on the leaf images in the crop leaf image dataset. During the labeling process, the background, leaf, and diseased pixels in the image are marked as 0, 1, and 2, respectively;

[0035] Preprocessing process: The leaf images in the crop leaf image dataset are enhanced by using five techniques: image translation, rotation, flipping, brightness or contrast adjustment, and noise addition;

[0036] The pre-built multi-scale dual-attention mechanism U-Net model of the model training module includes an encoder and a decoder;

[0037] In the encoder, each stage consists of two 3x3 convolutional layers followed by ReLU activation and 2x2 max pooling for downsampling, doubling the number of feature channels in each downsampling process, and in the decoder, each stage consists of upsampling, 2x2 convolution to halve the number of feature channels, concatenated with the corresponding cropped feature map in the encoder, followed by two 3x3 convolutions, each followed by ReLU activation.

[0038] The encoder of the model training module performs multi-layer convolution and downsampling operations, and adopts the skip connection strategy to directly pass part of the feature map to the higher layer, and the other part is directly passed to the decoder part through the skip connection as part of the input of the upsampling operation. In the output stage, the feature images of different scales are fused to form the final result. The output of multi-scale feature fusion is shown in the following formula:

[0039] Z(x)=Conv([Z0,Z1,Z2,Z3,Z4])

[0040] In the formula, conv() represents the convolution operation, Z(x) is the output after feature fusion, and Z0, Z1, Z2, Z3, and Z4 are the output feature maps sampled at different scales respectively;

[0041] The position attention module is executed after the upsampling operation of each layer in the decoding stage of the decoder of the model training module, and the channel attention module is executed before the skip connection of each layer;

[0042] Position attention module structure: The position feature map P∈R^(H×W×C) is reshaped to obtain PB∈R^((H×W)×C) and PC∈R^(C×(H×W)), PB and PC are obtained by matrix multiplication to obtain PS∈R^((H×W)×(H×W)), PF∈R^(H×W×C) is obtained by matrix multiplication and reshaping of PS and PB, and finally PF and P are subjected to matrix addition and batch normalization operations to obtain the feature map PE∈R^(H×W×C);

[0043] Channel attention module structure: The channel feature map C∈R^(H×W×C) is reshaped to obtain CB∈R^((H×W)×C) and CC∈R^(C×(H×W)), CB and CC are obtained by matrix multiplication to obtain CS∈R^(C×C), CF∈R^(H×W×C) is obtained by matrix multiplication and reshaping of CS and CB, and finally CF and C are subjected to matrix addition and batch normalization operations to obtain the final feature map CE∈R^(H×W×C);

[0044] Concatenate the feature maps PE and CE as the input features of the next layer in the decoding stage;

[0045] The bottleneck layer contains two convolutional layers, each of which has multiple convolution kernels. Each convolutional layer is a convolutional layer that expands the dimension of the input image. A BN layer is added after each convolutional layer. A Dropout layer is added after the second BN layer and before upsampling to regularize the output image.

[0046] The disease diagnosis module uses the following calculation formula to diagnose the degree of disease based on the percentage of the total area of ​​leaf spots to the total area of ​​the leaf:

[0047]

[0048] Where, M is the percentage of the total area of ​​lesions to the total area of ​​leaves, N lesion Represented as the number of diseased spot pixels in the image, N leaf Represented as the number of pixels of crop leaves in the image.

[0049] In another aspect of the present invention, in order to achieve the above-mentioned purpose, a terminal device is disclosed, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the memory stores a computer program capable of running on the processor, and when the processor loads and executes the computer program, a method for diagnosing the degree of crop leaf disease as described above is adopted.

[0050] Beneficial effects of the present invention:

[0051] The present invention introduces the dual attention mechanism operations of channel attention (CA) and position attention (PA), and adds batch normalization (BN) and dropout layers to the bottleneck layer of the network, to construct a multi-scale dual attention mechanism U-Net to realize crop leaf disease recognition and degree diagnosis. It can effectively identify various diseases, improve the processing efficiency and the accuracy of disease type recognition, and can realize disease degree diagnosis with high accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0053] Figure 1 It is a schematic flow chart of the method of the present invention;

[0054] Figure 2 It is a schematic diagram of the model principle of the present invention;

[0055] Figure 3 It is a schematic diagram of the structure of the position attention module of the present invention;

[0056] Figure 4 Schematic diagram of the channel attention module structure of the present invention;

[0057] Figure 5 This is a schematic diagram of the bottleneck layer structure principle of the present invention;

[0058] Figure 6 It is a schematic diagram of the system structure of the present invention. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] Embodiment 1:

[0061] like Figure 1 As shown, a method for diagnosing the degree of crop leaf disease comprises the following steps:

[0062] S101: Acquire a crop leaf image dataset, annotate and preprocess the crop leaf image dataset, and obtain a processed crop leaf image dataset;

[0063] Crop leaf images can be selected from three public plant disease datasets: Plant Village, Plant Doc, and Dataset of Tomato Leaves, or they can be taken by yourself. Crop leaf images should include images with various diseases and images without diseases. 636 crop leaf disease images with simple and complex backgrounds were selected from the above datasets. Then the image size was uniformly adjusted to 256 pixels × 256 pixels by image scaling.

[0064] Label the crop leaf image dataset: Use Labelme or other labeling software to label the leaf images at the pixel level. During the labeling process, the background, leaf, and diseased pixels in the image are marked as 0, 1, and 2, respectively.

[0065] Crop leaf image data preprocessing.

[0066] The leaf images were enhanced by five techniques: image translation, rotation, flipping, brightness or contrast adjustment, and noise addition. After enhancement, the number of images in the dataset increased to 3,800.

[0067] S102: inputting the crop leaf image dataset into a pre-established multi-scale dual attention mechanism U-Net model for training, thereby obtaining a trained multi-scale dual attention mechanism U-Net model;

[0068] like Figure 2 The multi-scale dual attention mechanism U-Net structure consists of two parts: encoder and decoder. The encoder gradually reduces the spatial dimension of the image while increasing the number of feature channels to capture the contextual information of the image. The decoder gradually restores the spatial dimension of the image while reducing the number of feature channels for precise positioning.

[0069] In the encoder, each stage consists of two 3x3 convolutional layers followed by ReLU activation and 2x2 max pooling for downsampling, doubling the number of feature channels in each downsampling process. In the decoder, each stage consists of upsampling, a 2x2 convolution to halve the number of feature channels, concatenation with the corresponding cropped feature map from the encoder, and then two 3x3 convolutions, each followed by a ReLU activation function.

[0070] The preprocessed leaf image first undergoes multi-layer convolution and downsampling operations in the encoder part. Using the skip connection strategy, part of the feature map is directly passed to the higher level, while the other part is directly passed to the decoder part through the skip connection as part of the input of the upsampling operation. In the output stage, the feature images of different scales are fused to form the final result. The output of multi-scale feature fusion is shown in the following formula:

[0071] Z(x)=Conv([Z0,Z1,Z2,Z3,Z4])

[0072] Where conv() represents the convolution operation, Z(x) is the output after feature fusion, and Z0, Z1, Z2, Z3, and Z4 are the output feature maps sampled at different scales.

[0073] The position attention module is executed after the upsampling operation of each layer in the decoding stage, and the channel attention module is executed before the skip connection of each layer.

[0074] The structure of the position attention module is as follows Figure 3 As shown in the figure, the position feature map P∈R^(H×W×C) is reshaped to obtain PB∈R^((H×W)×C) and PC∈R^(C×(H×W)), PB and PC are obtained by matrix multiplication to obtain PS∈R^((H×W)×(H×W)), PF∈R^(H×W×C) is obtained by matrix multiplication and reshaping of PS and PB, and finally PF and P are subjected to matrix addition and batch normalization operations to obtain the feature map PE∈R^(H×W×C).

[0075] The channel attention module structure is as follows Figure 4 As shown in the figure, the channel feature map C∈R^(H×W×C) is reshaped to obtain CB∈R^((H×W)×C) and CC∈R^(C×(H×W)), CB and CC are obtained by matrix multiplication to obtain CS∈R^(C×C), CF∈R^(H×W×C) is obtained by matrix multiplication and reshaping of CS and CB, and finally CF and C are subjected to matrix addition and batch normalization operations to obtain the final feature map CE∈R^(H×W×C).

[0076] The feature maps PE and CE are concatenated and used as the input features of the next layer in the decoding stage.

[0077] The bottleneck layer contains two convolutional layers, each of which has 1024 convolution kernels, and each convolutional layer is a convolutional layer that expands the dimension of the input image. The specific structure is as follows Figure 5 As shown in the figure, a BN layer is added after each 3×3 convolution layer. A Dropout layer is added after the second BN layer and before upsampling to regularize the output image.

[0078] Model training process: Use Adam optimizer to update parameters, the initial learning rate of training is 0.001, the decay value is 0.0001, the momentum coefficient is 0.9, the batch-size is set to 4, the epoch size is set to 100, and the Dropout value is set to 0.2.

[0079] S103: obtaining a crop leaf image to be identified, inputting the crop leaf image to be identified into a trained multi-scale dual-attention mechanism U-Net model, outputting a classification probability of the crop leaf image, and identifying the crop leaf disease category according to the classification probability of the crop leaf image;

[0080] S104: For the crop leaf images with identified disease categories, a disease degree diagnosis calculation is performed based on the percentage of the total area of ​​leaf lesions to the total area of ​​the leaf to obtain a disease degree grade.

[0081] The calculation formula for diagnosing the degree of disease based on the percentage of the total area of ​​leaf spots to the total area of ​​the leaf is as follows:

[0082]

[0083] Where, M is the percentage of the total area of ​​lesions to the total area of ​​leaves, N lesion Represented as the number of diseased spot pixels in the image, N leaf Represented as the number of pixels of crop leaves in the image.

[0084] According to the M value, the disease grade is outputted by referring to Table 1, that is, the disease degree diagnosis is realized.

[0085] Table 1 Classification table of crop leaf diseases

[0086]

[0087] In order to evaluate the performance of this method, a typical crop tomato leaf disease image was taken as an example, and the mean pixel accuracy (MPA), mean intersection over Union (MIoU), accuracy (Acc), and running time evaluation indicators were used to evaluate the method.

[0088] Table 2 is a comparison list of disease type identification of this method and several other currently popular methods. It can be seen that the evaluation indicators of this method, average pixel accuracy, average intersection-over-union ratio, and accuracy, are better than those of the other compared methods. And the running time is the shortest among the several methods. This shows that this method effectively identifies various diseases and improves the processing efficiency and the accuracy of disease type identification.

[0089] Table 2 Performance comparison of this method and different models

[0090]

[0091] In order to test the diagnostic effect of this method on the degree of disease, the tomato leaf disease image was taken as an example to compare the disease degree level obtained by this method with the actual disease degree level. Through corresponding calculations, the tomato leaf disease degree obtained by this method achieved a high accuracy rate at each degree level, with an average accuracy rate of 94.53%.

[0092] Embodiment 2: In the second aspect, as Figure 6As shown, in order to achieve the above-mentioned purpose, the present invention discloses a crop leaf disease degree diagnosis system, comprising:

[0093] The image processing module 11 is used to obtain a crop leaf image dataset, annotate and preprocess the crop leaf image dataset to obtain a processed crop leaf image dataset, and divide the processed crop leaf image dataset into a training set, a verification set, and a test set;

[0094] A model training module 12 is used to input the training set into a pre-established multi-scale dual attention mechanism U-Net model for training, so as to obtain a trained multi-scale dual attention mechanism U-Net model;

[0095] The disease recognition module 13 is used to obtain the crop leaf image to be identified, input the crop leaf image to be identified into the trained multi-scale dual-attention mechanism U-Net model, output the classification probability of the crop leaf image, and identify the crop leaf disease category according to the classification probability of the crop leaf image;

[0096] The disease diagnosis module 14 is used to diagnose and calculate the degree of disease based on the percentage of the total area of ​​leaf spots to the total area of ​​the leaf according to the harmful leaves obtained from the crop leaf image categories, and obtain the disease degree level.

[0097] In conjunction with the second aspect, in some implementations of the second aspect, the system further includes: the crop leaf image dataset of the image processing module 11 is screened from three public plant disease datasets: Plant Village, Plant Doc, and Dataset of Tomato Leaves, wherein the crop leaf image dataset includes images with various diseases and images without diseases;

[0098] The image processing module 11 performs labeling and preprocessing on the crop leaf image dataset:

[0099] Labeling process: Use Labelme or other labeling software to perform pixel-level labeling on the leaf images in the crop leaf image dataset. During the labeling process, the background, leaf, and diseased pixels in the image are marked as 0, 1, and 2, respectively;

[0100] Preprocessing process: The leaf images in the crop leaf image dataset are enhanced by using five techniques: image translation, rotation, flipping, brightness or contrast adjustment, and noise addition;

[0101] The pre-established multi-scale dual attention mechanism U-Net model of the model training module 12 includes an encoder and a decoder;

[0102] In the encoder, each stage consists of two 3x3 convolutional layers followed by ReLU activation and 2x2 max pooling for downsampling, doubling the number of feature channels in each downsampling process, and in the decoder, each stage consists of upsampling, 2x2 convolution to halve the number of feature channels, concatenated with the corresponding cropped feature map in the encoder, followed by two 3x3 convolutions, each followed by ReLU activation.

[0103] The encoder of the model training module 12 performs multi-layer convolution and downsampling operations, and adopts a skip connection strategy to directly pass part of the feature map to a higher level, and the other part is directly passed to the decoder part through a skip connection as part of the input of the upsampling operation. In the output stage, feature images of different scales are fused to form the final result. The output of multi-scale feature fusion is shown in the following formula:

[0104] Z(x)=Conv([Z0,Z1,Z2,Z3,Z4])

[0105] In the formula, conv() represents the convolution operation, Z(x) is the output after feature fusion, and Z0, Z1, Z2, Z3, and Z4 are the output feature maps sampled at different scales respectively;

[0106] The position attention module is executed after the upsampling operation of each layer in the decoding stage of the decoder of the model training module 12, and the channel attention module is executed before the jump connection of each layer;

[0107] Position attention module structure: The position feature map P∈R^(H×W×C) is reshaped to obtain PB∈R^((H×W)×C) and PC∈R^(C×(H×W)), PB and PC are obtained by matrix multiplication to obtain PS∈R^((H×W)×(H×W)), PF∈R^(H×W×C) is obtained by matrix multiplication and reshaping of PS and PB, and finally PF and P are subjected to matrix addition and batch normalization operations to obtain the feature map PE∈R^(H×W×C);

[0108] Channel attention module structure: The channel feature map C∈R^(H×W×C) is reshaped to obtain CB∈R^((H×W)×C) and CC∈R^(C×(H×W)), CB and CC are obtained by matrix multiplication to obtain CS∈R^(C×C), CF∈R^(H×W×C) is obtained by matrix multiplication and shaping of CS and CB, and finally CF and C are subjected to matrix addition and batch normalization operations to obtain the final feature map CE∈R^(H×W×C).

[0109] Concatenate the feature maps PE and CE as the input features of the next layer in the decoding stage;

[0110] The bottleneck layer contains two convolutional layers, each of which has multiple convolution kernels. Each convolutional layer is a convolutional layer that expands the dimension of the input image. A BN layer is added after each convolutional layer. A Dropout layer is added after the second BN layer and before upsampling to regularize the output image.

[0111] The disease diagnosis module 14 performs disease degree diagnosis calculation according to the percentage of the total area of ​​leaf spots to the total area of ​​the leaf using the following calculation formula:

[0112]

[0113] Where, M is the percentage of the total area of ​​lesions to the total area of ​​leaves, N lesion Represented as the number of diseased spot pixels in the image, N leaf Represented as the number of pixels of crop leaves in the image.

[0114] Based on the same inventive concept, the present invention also provides a computer device, which includes: one or more processors, and a memory for storing one or more computer programs; the program includes program instructions, and the processor is used to execute the program instructions stored in the memory. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is used to implement one or more instructions, specifically for loading and executing one or more instructions in a computer storage medium to implement the above method.

[0115] It needs to be further explained that, based on the same inventive concept, the present invention also provides a computer storage medium, on which a computer program is stored, and the computer program is executed by a processor to execute the above method. The storage medium can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electrical, magnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0116] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0117] The above shows and describes the basic principles, main features and advantages of the present disclosure. Those skilled in the art should understand that the present disclosure is not limited by the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present disclosure. Without departing from the spirit and scope of the present disclosure, the present disclosure may have various changes and improvements, and these changes and improvements fall within the scope of the present disclosure to be protected.

Claims

1. A method for diagnosing the degree of crop leaf disease, characterized in that: The method comprises the following steps: Acquire a crop leaf image dataset, annotate and preprocess the crop leaf image dataset, and obtain a processed crop leaf image dataset; The crop leaf image dataset is input into a pre-established multi-scale dual attention mechanism U-Net model for training, and a trained multi-scale dual attention mechanism U-Net model is obtained; Obtain the crop leaf image to be identified, input the crop leaf image to be identified into the trained multi-scale dual-attention mechanism U-Net model, output the classification probability of the crop leaf image, and identify the crop leaf disease category according to the classification probability of the crop leaf image; For crop leaf images with identified disease categories, the disease severity is diagnosed and calculated based on the percentage of the total area of ​​leaf lesions in the image to the total leaf area to obtain the disease severity level.

2. A method for diagnosing the degree of crop leaf disease according to claim 1, characterized in that: The crop leaf image dataset is screened from three public plant disease datasets: Plant Village, Plant Doc and Dataset of Tomato Leaves, wherein the crop leaf image dataset includes images with various diseases and images without diseases.

3. A method for diagnosing the degree of crop leaf disease according to claim 1, characterized in that: The process of labeling and preprocessing the crop leaf image dataset: Labeling process: Use Labelme or other labeling software to perform pixel-level labeling on the leaf images in the crop leaf image dataset. During the labeling process, the background, leaf, and diseased pixels in the image are marked as 0, 1, and 2, respectively; Preprocessing process: The leaf images in the crop leaf image dataset are enhanced through five techniques: image translation, rotation, flipping, brightness or contrast adjustment, and noise addition.

4. A method for diagnosing the degree of crop leaf disease according to claim 1, characterized in that: The pre-established multi-scale dual attention mechanism U-Net model includes an encoder and a decoder; In the encoder, each stage consists of two 3x3 convolutional layers, followed by ReLU activation and 2x2 max pooling for downsampling, doubling the number of feature channels in each downsampling process, and in the decoder, each stage consists of upsampling, 2x2 convolution to halve the number of feature channels, concatenated with the corresponding cropped feature map in the encoder, followed by two 3x3 convolutions, each followed by a ReLU activation function.

5. A method for diagnosing the degree of crop leaf disease according to claim 4, characterized in that: The encoder performs multi-layer convolution and downsampling operations, and adopts a skip connection strategy to directly pass part of the feature map to a higher level, and the other part is directly passed to the decoder part through a skip connection as part of the input of the upsampling operation. At the output stage, feature images of different scales are fused to form the final result. The output of multi-scale feature fusion is shown in the following formula: Z(x)=Conv([Z0,Z1,Z2,Z3,Z4]) Where conv() represents the convolution operation, Z(x) is the output after feature fusion, and Z0, Z1, Z2, Z3, and Z4 are the output feature maps sampled at different scales.

6. A method for diagnosing the degree of crop leaf disease according to claim 5, characterized in that: The decoder performs a position attention module after upsampling operation of each layer in the decoding stage, and performs a channel attention module before the skip connection of each layer; Position attention module: The position feature map P∈R^(H×W×C) is reshaped to obtain PB∈R^((H×W)×C) and PC∈R^(C×(H×W)), PB and PC are matrix multiplied to obtain PS∈R^((H×W)×(H×W)), PF∈R^(H×W×C) is obtained by matrix multiplication and shaping of PS and PB, and finally PF and P are matrix added and batch normalized to obtain the feature map PE∈R^(H×W×C); Channel attention module: The channel feature map C∈R^(H×W×C) is reshaped to obtain CB∈R^((H×W)×C) and CC∈R^(C×(H×W)), CB and CC are obtained by matrix multiplication to obtain CS∈R^(C×C), CF∈R^(H×W×C) is obtained by matrix multiplication and reshaping of CS and CB, and finally CF and C are subjected to matrix addition and batch normalization operations to obtain the final feature map CE∈R^(H×W×C); Concatenate the feature maps PE and CE as the input features of the next layer in the decoding stage; The bottleneck layer contains two convolutional layers, each of which has multiple convolution kernels. Each convolutional layer is a convolutional layer that expands the dimension of the input image. A BN layer is added after each convolutional layer, and a Dropout layer is added after the second BN layer and before upsampling to regularize the output image.

7. A method for diagnosing the degree of crop leaf disease according to claim 1, characterized in that: The calculation formula for diagnosing the degree of disease based on the percentage of the total area of ​​leaf spots to the total area of ​​leaves is as follows: Where, M is the percentage of the total area of ​​lesions to the total area of ​​leaves, N lesion Represented as the number of diseased spot pixels in the image, N leaf Represented as the number of pixels of crop leaves in the image.

8. A crop leaf disease degree diagnosis system, characterized in that: include: An image processing module is used to obtain a crop leaf image dataset, annotate and preprocess the crop leaf image dataset, and obtain a processed crop leaf image dataset; A model training module is used to input the crop leaf image dataset into a pre-established multi-scale dual-attention mechanism U-Net model for training, thereby obtaining a trained multi-scale dual-attention mechanism U-Net model; The disease recognition module is used to obtain the crop leaf image to be identified, input the crop leaf image to be identified into the trained multi-scale dual-attention mechanism U-Net model, output the classification probability of the crop leaf image, and identify the crop leaf disease category according to the classification probability of the crop leaf image; The disease diagnosis module is used to diagnose and calculate the degree of disease for crop leaf images with identified disease categories based on the percentage of the total area of ​​leaf spots in the image to the total area of ​​the leaf, and obtain the disease degree level.

9. A crop leaf disease degree diagnosis system according to claim 8, characterized in that: The crop leaf image dataset of the image processing module is screened from three public plant disease datasets: Plant Village, Plant Doc and Dataset of Tomato Leaves, wherein the crop leaf image dataset includes images with various diseases and images without diseases; The image processing module labels and preprocesses the crop leaf image dataset: Labeling process: Use Labelme or other labeling software to perform pixel-level labeling on the leaf images in the crop leaf image dataset. During the labeling process, the background, leaf, and diseased pixels in the image are marked as 0, 1, and 2, respectively; Preprocessing process: leaf images are enhanced by five techniques: image translation, rotation, flipping, brightness or contrast adjustment, and noise addition; The pre-built multi-scale dual-attention mechanism U-Net model of the model training module includes an encoder and a decoder; In the encoder, each stage consists of two 3x3 convolutional layers followed by ReLU activation and 2x2 max pooling for downsampling, doubling the number of feature channels in each downsampling process. In the decoder, each stage consists of upsampling, 2x2 convolution to halve the number of feature channels, concatenated with the corresponding cropped feature map in the encoder, and then two 3x3 convolutions, each followed by ReLU activation. The encoder of the model training module performs multi-layer convolution and downsampling operations, and adopts the skip connection strategy to pass part of the feature map directly to the higher layer, while the other part is directly passed to the decoder part through the skip connection as part of the input of the upsampling operation. In the output stage, the feature images of different scales are fused to form the final result. The output of multi-scale feature fusion is shown in the following formula: Z(x)=Conv([Z0,Z1,Z2,Z3,Z4]) In the formula, conv() represents the convolution operation, Z(x) is the output after feature fusion, and Z0, Z1, Z2, Z3, and Z4 are the output feature maps sampled at different scales respectively; The position attention module is executed after the upsampling operation of each layer in the decoding stage of the model training module, and the channel attention module is executed before the skip connection of each layer; Position attention module: The position feature map P∈R^(H×W×C) is reshaped to obtain PB∈R^((H×W)×C) and PC∈R^(C×(H×W)), PB and PC are matrix multiplied to obtain PS∈R^((H×W)×(H×W)), PF∈R^(H×W×C) is obtained by matrix multiplication and shaping of PS and PB, and finally PF and P are matrix added and batch normalized to obtain the feature map PE∈R^(H×W×C); Channel attention module: The channel feature map C∈R^(H×W×C) is reshaped to obtain CB∈R^((H×W)×C) and CC∈R^(C×(H×W)), CB and CC are obtained by matrix multiplication to obtain CS∈R^(C×C), CF∈R^(H×W×C) is obtained by matrix multiplication and reshaping of CS and CB, and finally CF and C are subjected to matrix addition and batch normalization operations to obtain the final feature map CE∈R^(H×W×C); Concatenate the feature maps PE and CE as the input features of the next layer in the decoding stage; The bottleneck layer contains two convolutional layers, each of which has multiple convolution kernels. Each convolutional layer is a convolutional layer that expands the dimension of the input image. A BN layer is added after each convolution layer. A Dropout layer is added after the second BN layer and before upsampling to regularize the output image. The disease diagnosis module uses the following calculation formula to diagnose the degree of disease based on the percentage of the total area of ​​leaf spots to the total area of ​​the leaf: Where, M is the percentage of the total area of ​​lesions to the total area of ​​leaves, N lesion Represented as the number of diseased spot pixels in the image, N leaf Represented as the number of pixels of crop leaves in the image.

10. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that: The memory stores a computer program that can be run on the processor. When the processor loads and executes the computer program, a method for diagnosing the degree of crop leaf disease according to any one of claims 1 to 7 is adopted.

Images