Crop disease segmentation method, system and device based on generative adversarial semi-supervised learning, and medium

Abstract

A crop disease segmentation method, system, device and medium based on generative adversarial semi-supervised learning, the method takes Deeplabv3 + as a basic framework, introduces an adversarial training mechanism of a generative adversarial network, the Deeplabv3 + is used as a segmentation network to generate a prediction image of a disease area, and the prediction image is used as a prediction image of the disease area. The discrimination network is responsible for distinguishing the generated prediction image and the real annotation image, and the prediction accuracy is improved through adversarial learning of the two images; by introducing a semi-supervised loss function and combining standard cross entropy loss and adversarial loss, model performance is optimized under limited annotation data, and the crop disease segmentation capability is improved; a simulation test is combined to prove the effectiveness of the generative adversarial semi-supervised segmentation model (semi-i) in the aspect of training by using unlabeled data, relatively high segmentation performance can be still kept under the condition of limited labeled data, and the method has the advantages of high precision and low manpower consumption.

Description

Technical Field

[0001] The present invention relates to the technical field of crop disease segmentation, and in particular to a crop disease segmentation method, system, device and medium based on generative adversarial semi-supervised learning. Background Art

[0002] Crop disease image segmentation is an important step in disease detection and identification, and its segmentation effect directly affects the subsequent diagnosis results. Since the shapes and colors of crop diseases are complex and changeable, image segmentation technology can effectively restore the irregular distribution area of ​​the disease, providing a reliable basis for subsequent disease type identification. In the past, crop disease detection mainly relied on manual detection, which not only consumed a lot of manpower and material resources, but was also easily affected by the subjective factors of the inspectors, resulting in low efficiency. Therefore, the automation of disease detection has important research value and broad application prospects.

[0003] As an important branch of machine learning, deep learning has shown its great potential and value in the field of crop disease segmentation. Some classic semantic segmentation networks such as SegNet, U-Net and Deeplab series have been successively applied to crop disease segmentation.

[0004] Generative adversarial networks (GAN) are adversarial models composed of a generator and a discriminator. The generator tries to generate samples similar to real samples, while the discriminator tries to distinguish between generated samples and real samples. The adversarial process between the generator and the discriminator is used for training to generate more realistic data samples.

[0005] Semi-supervised learning is a learning method that combines labeled data and unlabeled data. Usually, in practical applications, the cost of obtaining labeled data is high, while unlabeled data is easy to obtain. Semi-supervised learning uses a small amount of labeled data and a large amount of unlabeled data to improve the generalization ability of the model by learning the potential structure or distribution of unlabeled data. It is usually based on supervised learning, through self-training, generative models or graphical models, so that the model can achieve better performance on limited labeled data.

[0006] Afzaal (Afzaal U, Bhattai B, Pandeya YR, et al. An instance segmentation model for strawberry diseases based on Mask R-CNN[J]. Sensors, 2021, 21: 6565.) proposed a segmentation model based on the Mask R-CNN architecture; by replacing the backbone network with ResNet and adding data enhancement methods, the segmentation of strawberry diseases under 7 complex background conditions was achieved, and the final average accuracy reached 82.43%; however, this method requires a large amount of labeled data sets, which is manpower and time-consuming.

[0007] Guo Xiaoqing (Guo Xiaoqing, Fan Taojie, Shu Xin. Tomato leaf disease image recognition based on improved Multi-Scale AlexNet [J]. Transactions of the Chinese Society of Agricultural Engineering, 2019, 35(13): 162-169.) proposed a multi-receptive field recognition model based on MultiScale Alex Net. By adding convolution kernels of different sizes to the Alex Net model, multiple local features can be extracted simultaneously. The average recognition accuracy of tomato leaf diseases and eight disease severity levels reached 92.7%. However, the network algorithm of this method is too complex, and the number of parameters of some networks is relatively high.

[0008] The patent application document with publication number CN116543282A discloses a crop disease segmentation method, system, equipment and medium based on multi-scale fusion and CBAM-ResNet50. On the basis of the standard U-Net model, by changing the model feature extraction network and designing a multi-scale fusion module, the crop disease segmentation accuracy is improved, the segmentation speed is increased, and it also has a good segmentation effect on small target crops and complex background conditions; however, the model has a complex structure, there is a gradient vanishing problem in the extremely deep network, and it is highly dependent on labeled data. Summary of the invention

[0009] In order to overcome the defects of the above-mentioned prior art, the purpose of the present invention is to provide a crop disease segmentation method, system, device and medium based on generative adversarial semi-supervised learning. The method is based on Deeplabv3+ and introduces an adversarial training mechanism of a generative adversarial network, wherein Deeplabv3+ is used as a segmentation network to generate a predicted image of the diseased area, and the discriminant network is responsible for distinguishing the generated predicted image from the real labeled image, and the prediction accuracy is improved through the adversarial learning of the two; at the same time, by introducing a semi-supervised loss function and combining the standard cross entropy loss and adversarial loss, the model performance is optimized under limited labeled data, thereby improving the segmentation ability of crop diseases; it has the advantages of high precision and low manpower consumption.

[0010] In order to achieve the above object, the technical solution adopted by the present invention is:

[0011] A crop disease segmentation method based on generative adversarial semi-supervised learning includes the following steps:

[0012] Step 1: Collect crop disease images in different environments and growth stages as a data set, annotate the data set, and indicate the category to which it belongs;

[0013] Step 2: Use Deeplabv3+ as the segmentation network, send the dataset in step 1 and the labeled dataset to the segmentation network for preliminary training, and then introduce the discriminant network to perform adversarial training on the segmentation network;

[0014] Step 3: Introduce the loss function to jointly affect the training process in step 2;

[0015] Step 4: By continuously iterating the training process in steps 2 and 3, a trained generative adversarial semi-supervised segmentation model (semi) is obtained;

[0016] Step 5: Input the crop disease image collected in step 1 into the generative adversarial semi-supervised segmentation model (semi) trained in step 4 to obtain the crop disease segmentation result.

[0017] Furthermore, the step 2 is specifically as follows:

[0018] Step 2.1: Use Deeplabv3+ as the segmentation network instead of the generative network of the Generative Adversarial Network (GAN);

[0019] Step 2.2: Divide the labeled data set in step 1 into a training set and a test set, and use the training set and the corresponding labels as the labeled data (X L ), using unlabeled original crop disease images as unlabeled data (X U );

[0020] Step 2.3: Transform the labeled data (X) in step 2.2 into L ) and unlabeled data (X U ) are respectively input into the segmentation network in step 2.1 for training, and the corresponding prediction result graph f(X L )、f(X U );

[0021] Step 2.4: Map the prediction result graph f(X L )、f(X U ) is sent to the discriminant network to obtain the confidence map of the discriminant network;

[0022] Step 2.5: The discriminant network marks the preliminary segmentation results of the segmentation network according to the confidence map in step 2.4, generates pseudo labels, and guides the further training of the segmentation network.

[0023] Furthermore, the step 3 is specifically as follows:

[0024] Step 3.1: When using labeled data (X L ) is trained, the segmentation network is trained based on the true label (Y L )’s standard cross entropy loss (L ce ) supervised training, adversarial loss from the discriminative network (L adv ) supervised training;

[0025] Step 3.2: When using unlabeled data (X U ) is trained, the segmentation network is subject to a semi-supervised loss (L semi ) supervised training.

[0026] Furthermore, the step 3.1 is specifically as follows:

[0027] Step 3.1.1: Based on the true label (Y L )’s standard cross entropy loss (L ce ), which is used to measure the difference between the class probability map generated by the segmentation network and the actual diseased area; by minimizing the standard cross entropy loss (L ce ), the segmentation network can learn to accurately predict the diseased area; the standard cross entropy loss (L ce ) is calculated as:

[0028]

[0029] Where: Y n represents the true label image; P n Represents the prediction result image; h, w, c represent the height, width and number of channels of the image respectively;

[0030] Step 3.1.2: Adversarial loss from the discriminative network (L adv ), which is used to guide the training of the segmentation network; for the samples generated by the segmentation network, the discriminant network will discriminate and give the probability that the generated samples are real samples; the adversarial loss (L adv ) is calculated as:

[0031]

[0032] Where: P nRepresents the prediction result image; h, w, c represent the height, width and number of channels of the image respectively; I represents the indicator function, which returns a specific value, usually 1 or 0, according to the input conditions.

[0033] Furthermore, the step 3.2 is specifically as follows:

[0034] Step 3.2.1: Use the segmentation network to generate a pseudo image, input the generated pseudo image into the discriminant network, and calculate the confidence of the discriminant network, which is the output of the discriminant network;

[0035] Step 3.2.2: Use the probability threshold to take the pseudo-images corresponding to the output of the discriminant network that are higher than the threshold as pseudo-labels, input the pseudo-labels and the unlabeled images corresponding to the pseudo-labels into the segmentation network, and calculate the output of the segmentation network for the unlabeled image data;

[0036] Step 3.2.3: Calculate the semi-supervised loss using the cross entropy loss function between the pseudo-label and the segmentation network output; the semi-supervised loss (L semi ) is calculated as:

[0037]

[0038] in: Represents a pseudo image generated by the segmentation network; I represents an indicator function, which returns a specific value, usually 1 or 0, according to the input conditions; P n Represents the prediction result image.

[0039] Furthermore, the segmentation network in step 2 adopts an encoder-decoder structure; the encoder is responsible for converting the input image into a high-dimensional feature vector, while the decoder is responsible for mapping the high-dimensional feature vector back to the pixel space, and finally obtaining the prediction result.

[0040] Furthermore, the encoder uses dilated convolution to extract features calculated by deep convolutional neural networks at any resolution, and by applying dilated convolutions with different rates to the spatial pyramid ASPP module, combined with image-level features, convolutional features can be obtained at multiple scales, thereby extracting high-level features in the image;

[0041] The decoder performs an upsample operation on the feature map obtained by the ASPP module, restores it to a feature map of the same size as the original input image, and fuses the shallow feature map.

[0042] A crop disease segmentation system based on generative adversarial semi-supervised learning, including:

[0043] Data collection module: collects crop disease images in different environments and growth stages as data sets, labels the data sets, and indicates the categories to which they belong;

[0044] Adversarial training module: Deeplabv3+ is used as the segmentation network. The data set and the labeled data set are respectively sent to the segmentation network for preliminary training, and then the discriminant network is introduced to perform adversarial training on the segmentation network.

[0045] Loss-supervised training module: introduces the loss function to jointly act against the training process of the training module;

[0046] Model building module: Through the continuous iteration of the training process of the adversarial training module and the loss supervision training module, a trained generative adversarial semi-supervised segmentation model (semi) is obtained;

[0047] Disease segmentation module: The collected crop disease images are input into the trained generative adversarial semi-supervised segmentation model (semi) to obtain the crop disease segmentation results.

[0048] A crop disease segmentation device based on generative adversarial semi-supervised learning, comprising:

[0049] Memory: used to store a computer program to implement the crop disease segmentation method based on generative adversarial semi-supervised learning as described above;

[0050] Processor: used to implement the crop disease segmentation method based on generative adversarial semi-supervised learning as described above when executing the computer program.

[0051] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a crop disease segmentation method based on generative adversarial semi-supervised learning.

[0052] Compared with the prior art, the present invention has the following beneficial effects:

[0053] 1. The present invention gradually optimizes the segmentation ability of the segmentation network and improves the accuracy and stability of segmentation through adversarial training of the segmentation network and the discriminant network; generative adversarial training enables the model to capture image details, edge information and tiny structures, which has the advantage of improving the accuracy of crop disease segmentation.

[0054] 2. The present invention introduces a semi-supervised loss function and combines it with standard cross entropy loss and adversarial loss, so that the model can still generate accurate diseased area segmentation images under limited labeled data, which has the advantage of reducing the time consumption of the data labeling process.

[0055] In summary, the present invention takes Deeplabv3+ as the basic architecture and introduces the discriminant network. The segmentation network is optimized through the adversarial training strategy between the segmentation network and the discriminant network. At the same time, a semi-supervised training method is used to enable the model to optimize its own performance under limited labeled data, thereby improving the segmentation ability of crop diseases. The present invention has the advantages of high precision and low manpower consumption, and can be widely used in the field of disease segmentation technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 This is a flow chart of the crop disease segmentation method based on generative adversarial semi-supervised learning of the present invention.

[0057] Figure 2 are images of three different diseases of the present invention; wherein, Figure 2 (a) is an image of northern leaf blight; Figure 2 (b) is an image of northern leaf spot; Figure 2 (c) in the figure is an image of gray leaf spot.

[0058] Figure 3 This is a label diagram of three different diseases in the present invention; wherein, Figure 3 (a) in the figure is a label image of northern leaf blight; Figure 3 (b) in the figure is a label image of northern leaf spot; Figure 3 (c) in the figure is the label image of gray leaf spot disease.

[0059] Figure 4 A framework diagram of the adversarial semi-supervised disease segmentation model (Semi) generated for the present invention.

[0060] Figure 5 This is the AdvSeg disease segmentation model of the present invention.

[0061] Figure 6 The segmentation results of three different diseases input into the Deeplabv3+ segmentation model when the present invention uses 1 / 2 of the training set; among them, Figure 6 (a) shows the segmentation result of the image of northern leaf blight input into the Deeplabv3+ segmentation model when using 1 / 2 of the training set; Figure 6 (b) shows the segmentation result of the image of northern leaf spot disease input into the Deeplab v3+ segmentation model when using 1 / 2 of the training set; Figure 6 (c) in the figure is the segmentation result of the gray leaf spot image input into the Deeplabv3+ segmentation model when using 1 / 2 of the training set.

[0062] Figure 7 The segmentation results of three different diseases input into the AdvSeg disease segmentation model when the present invention uses 1 / 2 of the training set; wherein, Figure 7(a) shows the segmentation result of the image of northern leaf blight input into the AdvSeg disease segmentation model when using 1 / 2 of the training set; Figure 7 (b) shows the segmentation result of the image of northern leaf spot disease input into the AdvSeg disease segmentation model when using 1 / 2 of the training set; Figure 7 (c) in the figure is the segmentation result of the gray leaf spot image input into the AdvSeg disease segmentation model when using 1 / 2 of the training set.

[0063] Figure 8 When the present invention uses 1 / 2 of the training set, three different disease inputs generate segmentation results in the adversarial semi-supervised segmentation model; wherein, Figure 8 (a) shows the segmentation result of the adversarial semi-supervised segmentation model generated by the image input of northern leaf blight when using 1 / 2 of the training set; Figure 8 (b) shows the segmentation result of the adversarial semi-supervised segmentation model generated by the northern leaf spot image input when using 1 / 2 of the training set; Figure 8 (c) in the figure shows the segmentation result of the adversarial semi-supervised segmentation model generated by the gray leaf spot image input when using 1 / 2 of the training set. DETAILED DESCRIPTION

[0064] The present invention is further described in detail below with reference to the accompanying drawings and embodiments:

[0065] See also Figure 1 , a crop disease segmentation method based on generative adversarial semi-supervised learning, comprising the following steps:

[0066] Step 1: Collect crop disease images in different environments and growth stages as a data set, annotate the data set, and indicate the category to which it belongs;

[0067] The disease data comes from the public dataset CD&S, which was created by Aanis Ahmad et al. and contains 1,575 disease images, including three types of diseases: Figure 2 (a) in the figure is northern leaf blight (NLB); Figure 2 (b) in the figure is northern leaf spot (NLS); Figure 2 (c) in the figure is gray leaf spot (GLS).

[0068] The dataset was annotated using the Labelme image annotation tool to draw the disease boundary and mark the diseased area. The image size was 512×512 pixels. Figure 3 Label images after marking three different diseases; Figure 3 (a) in the figure is a label image of northern leaf blight; Figure 3 (b) in the figure is a label image of northern leaf spot; Figure 3(c) in the figure is the label image of gray leaf spot disease.

[0069] Step 2: Use Deeplabv3+ as the segmentation network, send the dataset in step 1 and the labeled dataset into the segmentation network for preliminary training, and then introduce the discriminant network to perform adversarial training on the segmentation network; through the adversarial training strategy between the two, the segmentation ability of the segmentation network is gradually optimized, and the accuracy and stability of the segmentation are improved.

[0070] The step 2 is specifically as follows:

[0071] Step 2.1: Use Deeplabv3+ as the segmentation network instead of the generative network of the Generative Adversarial Network (GAN);

[0072] The typical GAN ​​architecture consists of two sub-networks: the generator (G) and the discriminator (D), which interact through the maximum-minimum game mechanism during the training process. Specifically, the task of G is to receive random noise or other types of inputs and convert them into samples similar to the training data, so as to produce outputs with actual data characteristics. D is responsible for judging whether these generated samples come from real data sets or are generated by G, so as to identify the authenticity of the samples. Through this competitive and collaborative training method, GAN can continuously improve the quality of generated samples and the identification ability of the discriminator.

[0073] Step 2.2: Divide the labeled data set in step 1 into a training set and a test set, and use the training set and the corresponding labels as the labeled data (X L ), using unlabeled original crop disease images as unlabeled data (X U );

[0074] Step 2.3: Transform the labeled data (X) in step 2.2 into L ) and unlabeled data (X U ) are respectively input into the segmentation network in step 2.1 for training, and the corresponding prediction result graph f(X L )、f(X U );

[0075] Step 2.4: Map the prediction result graph f(X L )、f(X U ) is sent to the discriminant network to obtain the confidence map of the discriminant network;

[0076] Step 2.5: The discriminant network marks the preliminary segmentation results of the segmentation network according to the confidence map in step 2.4, generates pseudo labels, and guides the further training of the segmentation network.

[0077] The above discriminant network only participates in training and does not participate in the actual segmentation task. The discriminant network is trained only with labeled images because the discriminant network needs to use real labels to distinguish whether the prediction results are real, thereby providing effective adversarial losses to the segmentation network. In this way, the segmentation network and the discriminant network compete and cooperate with each other during the training process to jointly improve performance and achieve more accurate semantic segmentation.

[0078] The segmentation network in step 2 adopts an encoder-decoder structure; the encoder is responsible for converting the input image into a high-dimensional feature vector, while the decoder is responsible for mapping the high-dimensional feature vector back to the pixel space, and finally obtaining the prediction result.

[0079] The encoder is first a pre-trained deep convolutional neural network that can accurately capture features such as texture, shape, and color in an image. The encoder uses dilated convolutions to extract features calculated by a deep convolutional neural network at any resolution. By applying dilated convolutions with different rates to the spatial pyramid ASPP module, combined with image-level features, convolutional features can be obtained at multiple scales, thereby extracting high-level features in the image.

[0080] The decoder performs an upsampling operation on the feature map obtained by the ASPP module, restores it to a feature map of the same size as the original input image, and fuses the shallow feature map. This can retain high-level semantic information while obtaining a more refined segmentation result.

[0081] Step 3: Introduce the loss function to optimize the training process in step 2; by using the semi-supervised loss (L semi ), standard cross entropy loss (L ce ) and adversarial loss (L adv ) The three loss functions act together in the training process in step 2, so that the segmentation network can generate images that accurately predict the diseased area, and the discriminant network can accurately judge the authenticity of the generated image data. The semi-supervised learning framework can utilize the mutual cooperation of labeled data and unlabeled data in the training process, fully explore the potential information of the data, and improve the generalization ability and performance of the model.

[0082] The step 3 is specifically as follows:

[0083] Step 3.1: When using labeled data (X L ) is trained, the segmentation network is trained based on the true label (Y L )’s standard cross entropy loss (L ce ) supervised training, adversarial loss from the discriminative network (L adv ) supervised training;

[0084] The step 3.1 is specifically as follows:

[0085] Step 3.1.1: Based on the true label (Y L )’s standard cross entropy loss (L ce ), which is used to measure the difference between the class probability map generated by the segmentation network and the actual diseased area; by minimizing the standard cross entropy loss (L ce ), the segmentation network can learn to accurately predict the diseased area; the standard cross entropy loss (L ce ) is calculated as:

[0086]

[0087] Where: Y n represents the true label image; P n Represents the prediction result image; h, w, c represent the height, width and number of channels of the image respectively;

[0088] Step 3.1.2: Adversarial loss from the discriminative network (L adv ), which is used to guide the training of the segmentation network; for the samples generated by the segmentation network, the discriminant network will discriminate and give the probability that the generated samples are real samples; the goal of the segmentation network is to increase the probability that the generated samples are discriminated as real samples as much as possible, thereby "cheating" the discriminant network; the adversarial loss (L adv ) is calculated as:

[0089]

[0090] Where: P n Represents the prediction result image; h, w, c represent the height, width and number of channels of the image respectively; I represents the indicator function, which returns a specific value, usually 1 or 0, according to the input conditions.

[0091] Step 3.2: When using unlabeled data (X U ) is trained, the segmentation network is subject to a semi-supervised loss (L semi ) supervised training.

[0092] The step 3.2 is specifically as follows:

[0093] Step 3.2.1: Use the segmentation network to generate a pseudo image, input the generated pseudo image into the discriminant network, and calculate the confidence of the discriminant network, which is the output of the discriminant network;

[0094] Step 3.2.2: Use the probability threshold to take the pseudo-images corresponding to the output of the discriminant network that are higher than the threshold as pseudo-labels, input the pseudo-labels and the unlabeled images corresponding to the pseudo-labels into the segmentation network, and calculate the output of the segmentation network for the unlabeled image data;

[0095] Step 3.2.3: Calculate the semi-supervised loss using the cross entropy loss function between the pseudo-label and the segmentation network output; the semi-supervised loss (L semi ) is calculated as:

[0096]

[0097] in: Represents a pseudo image generated by the segmentation network; I represents an indicator function, which returns a specific value, usually 1 or 0, according to the input conditions; P n Represents the prediction result image.

[0098] The above three loss functions satisfy the following relationship:

[0099] L seg =L ce +λ adv L adv +λ semi L semi (4)

[0100] Where: L ce represents the standard cross entropy loss; L adv represents adversarial loss; L semi represents the semi-supervised loss; λ adv and λ semi are two constants that balance multi-task training

[0101] Step 4: By continuously iterating the training process in steps 2 and 3, a trained generative adversarial semi-supervised segmentation model (semi) is obtained; Figure 4 This is a framework diagram of the generative adversarial semi-supervised disease segmentation model (Semi) of the present invention. The model combines the principles of generative adversarial networks (GANs) with the idea of ​​semi-supervised learning, aiming to improve the accuracy and stability of image segmentation.

[0102] Step 5: Input the crop disease image collected in step 1 into the generative adversarial semi-supervised segmentation model (semi) trained in step 4 to obtain the crop disease segmentation result.

[0103] A crop disease segmentation system based on generative adversarial semi-supervised learning, including:

[0104] Data collection module: collects crop disease images in different environments and growth stages as data sets, labels the data sets, and indicates the categories to which they belong;

[0105] Adversarial training module: Deeplabv3+ is used as the segmentation network. The data set and the labeled data set are respectively sent to the segmentation network for preliminary training, and then the discriminant network is introduced to perform adversarial training on the segmentation network.

[0106] Loss-supervised training module: introduces the loss function to jointly act against the training process of the training module;

[0107] Model building module: Through the continuous iteration of the training process of the adversarial training module and the loss supervision training module, a trained generative adversarial semi-supervised segmentation model (semi) is obtained;

[0108] Disease segmentation module: The collected crop disease images are input into the trained generative adversarial semi-supervised segmentation model (semi) to obtain the crop disease segmentation results.

[0109] A crop disease segmentation device based on generative adversarial semi-supervised learning, comprising:

[0110] Memory: used to store a computer program to implement the crop disease segmentation method based on generative adversarial semi-supervised learning as described above;

[0111] Processor: used to implement the crop disease segmentation method based on generative adversarial semi-supervised learning as described above when executing the computer program.

[0112] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a crop disease segmentation method based on generative adversarial semi-supervised learning.

[0113] The application effect of the present invention is described in detail below in conjunction with simulation experiments.

[0114] The experimental environment configuration is shown in Table 1.

[0115] Table 1 Experimental environment configuration

[0116]

[0117] This paper uses MIoU (Mean Intersection over Union, MIoU) and Dice coefficient as evaluation indicators. MIoU is the average of the IoU (Intersection over Union, IoU) of each category. It is calculated by dividing the intersection of the true label and the predicted result by their union. It is used to measure the segmentation accuracy of the model for each category. The larger the MIoU, the more accurate the segmentation result. The MIoU is:

[0118]

[0119] Where: n represents the number of categories, IoU i is the IoU of the i-th category.

[0120] The Dice coefficient is a commonly used indicator to measure the similarity between the generated segmented image and the manually annotated image. The Dice coefficient ranges from 0 to 1, and the closer it is to 1, the higher the similarity between the two images. It is calculated by dividing twice the intersection of the true label and the predicted result by the sum of the total number of pixels of the true label and the predicted result. The Dice coefficient is:

[0121]

[0122] In order to comprehensively evaluate the performance of the semi-supervised semantic segmentation network under different ratios of labeled data and unlabeled data, 1 / 2 (707), 1 / 4 (354), 1 / 8 (177), and 1 / 16 (89) samples of the training set and the corresponding labels were used as labeled data. With the same number of samples, only the original image data was used as unlabeled data to train the semi-supervised network. At the same time, the original supervised network was trained using the same ratio of labeled data, and no unlabeled data was used in this part of the training.

[0123] For semi-supervised training, we randomly mix labeled and unlabeled data for training, and jointly update the segmentation network and the discriminant network. In each iteration, we only use the batch containing real data to train the discriminant network, and set the hyperparameter λ adv is 0.01, λ semi is 0.1 and T semi When randomly sampling some labeled and unlabeled data from the dataset, we average multiple experimental results using different random seeds to ensure the robustness of the evaluation.

[0124] Figure 5It is the structure of the AdvSeg disease segmentation model, which is based on Deeplabv3+ and is generated by adversarial training of the segmentation network and the discriminant network; the generative adversarial semi-supervised segmentation model (semi) of the present invention combines the AdvSeg disease segmentation model and the semi-supervised learning mechanism (i.e., introducing a loss function for supervised training).

[0125] In order to verify the effectiveness of the algorithm in this paper, the Deeplabv3+ segmentation model, the AdvSeg disease segmentation model based on Deeplabv3+, and the generative adversarial semi-supervised segmentation model (semi) were selected to conduct experiments in the same experimental environment. The experimental results are shown in Table 2.

[0126] Table 2 Ablation experiment

[0127]

[0128] As can be seen from the table, as the amount of labeled data decreases, the performance of all models gradually decreases. By comparing the results of the generative adversarial semi-supervised segmentation model (semi) and the Deeplabv3+ model (without unlabeled data), it can be found that the performance of the generative adversarial semi-supervised segmentation model (semi) is slightly better than that of the Deeplabv3+ model at the same data ratio. When the data is 1 / 16, the generative adversarial semi-supervised segmentation model (semi) is compared with the Deeplabv3+ model. The MIoU is improved by up to 5.44%, and the Dice coefficient is improved by 4.51%. This shows that using unlabeled data for semi-supervised training does help improve the performance of the model. Compared with the Deeplabv3+ model, the AdvSeg model achieved a 5.11% improvement in MIoU. In addition, when the semi-supervised loss function is introduced, the model performance is further improved by 0.13%, which verifies the effectiveness of the adversarial training mechanism in improving the segmentation accuracy of the model.

[0129] When the above model uses 1 / 2 of the training set, the actual predictions obtained are as follows Figure 6-8 By comparison Figure 6 (a)-(c) Figure 7 (a)-(c) Figure 8 From (a)-(c) in the figure, we can see that the three models can basically segment the diseased areas in the dataset, but the generative adversarial semi-supervised segmentation model is closer to the true label in segmentation results and has more correct pixels segmented.

[0130] In order to verify the effectiveness of the AdvSeg segmentation model, three classic neural network models commonly used in image semantic segmentation tasks, SegNet, U-Net, and Deeplabv3+, were selected to compare the algorithm performance with the AdvSeg model. The comparison results are shown in Table 3.

[0131] Table 3 Comparison results of different semantic segmentation models

[0132]

[0133] As can be seen from Table 3, among all the models, the AdvSeg model performs best, with an MIoU of 76.73%, which is 3.44, 5.49, and 7.54 percentage points higher than the Deeplabv3+ model, U-net model, and SegNet model, respectively, and a Dice coefficient of 86.12%, which is 2.6, 8.54, and 11.53 percentage points higher than the Deeplabv3+ model, U-net model, and SegNet model, respectively. By analyzing the effectiveness of the AdvSeg segmentation model, the good performance of the generative adversarial semi-supervised segmentation model (semi) is further reflected.

[0134] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered within the protection scope of the present invention.

Claims

1. A crop disease segmentation method based on generative adversarial semi-supervised learning, characterized by: The following steps are involved: Step 1: Collect crop disease images in different environments and growth stages as a data set, annotate the data set, and indicate the category to which it belongs; Step 2: Use Deeplabv3+ as the segmentation network, send the dataset in step 1 and the labeled dataset to the segmentation network for preliminary training, and then introduce the discriminant network to perform adversarial training on the segmentation network; Step 3: Introduce the loss function to jointly affect the training process in step 2; Step 4: By continuously iterating the training process in steps 2 and 3, a trained generative adversarial semi-supervised segmentation model (semi) is obtained; Step 5: Input the crop disease image collected in step 1 into the generative adversarial semi-supervised segmentation model (semi) trained in step 4 to obtain the crop disease segmentation result.

2. The crop disease segmentation method based on generative adversarial semi-supervised learning according to claim 1, characterized in that: The step 2 is specifically as follows: Step 2.1: Use Deeplabv3+ as the segmentation network instead of the generative network of the Generative Adversarial Network (GAN); Step 2.2: Divide the labeled data set in step 1 into a training set and a test set, and use the training set and the corresponding labels as the labeled data (X L ), using unlabeled original crop disease images as unlabeled data (X U ); Step 2.3: Transform the labeled data (X) in step 2.2 into L ) and unlabeled data (X U ) are respectively input into the segmentation network in step 2.1 for training, and the corresponding prediction result graph f(X L )、f(X U ); Step 2.4: Map the prediction result graph f(X L )、f(X U ) is sent to the discriminant network to obtain the confidence map of the discriminant network; Step 2.5: The discriminant network marks the preliminary segmentation results of the segmentation network according to the confidence map in step 2.4, generates pseudo labels, and guides the further training of the segmentation network.

3. A crop disease segmentation method based on generative adversarial semi-supervised learning according to claim 1 or 2, characterized in that: The step 3 is specifically as follows: Step 3.1: When using labeled data (X L ) is trained, the segmentation network is trained based on the true label (Y L )’s standard cross entropy loss (L ce ) supervised training, adversarial loss from the discriminative network (L adv ) supervised training; Step 3.2: When using unlabeled data (X U ) is trained, the segmentation network is subject to a semi-supervised loss (L semi ) supervised training.

4. The crop disease segmentation method based on generative adversarial semi-supervised learning according to claim 3, characterized in that: The step 3.1 is specifically as follows: Step 3.1.1: Based on the true label (Y L )’s standard cross entropy loss (L ce ), which is used to measure the difference between the class probability map generated by the segmentation network and the actual diseased area; by minimizing the standard cross entropy loss (L ce ), the segmentation network can learn to accurately predict the diseased area; the standard cross entropy loss (L ce ) is calculated as: Where: Y n represents the true label image; P n Represents the prediction result image; h, w, c represent the height, width and number of channels of the image respectively; Step 3.1.2: Adversarial loss from the discriminative network (L adv ), which is used to guide the training of the segmentation network; for the samples generated by the segmentation network, the discriminant network will discriminate and give the probability that the generated samples are real samples; the adversarial loss (L adv ) is calculated as: Where: P n Represents the prediction result image; h, w, c represent the height, width and number of channels of the image respectively; I represents the indicator function, which returns a specific value, usually 1 or 0, according to the input conditions.

5. The crop disease segmentation method based on generative adversarial semi-supervised learning according to claim 3, characterized in that: The step 3.2 is specifically as follows: Step 3.2.1: Use the segmentation network to generate a pseudo image, input the generated pseudo image into the discriminant network, and calculate the confidence of the discriminant network, which is the output of the discriminant network; Step 3.2.2: Use the probability threshold to take the pseudo-images corresponding to the output of the discriminant network that are higher than the threshold as pseudo-labels, input the pseudo-labels and the unlabeled images corresponding to the pseudo-labels into the segmentation network, and calculate the output of the segmentation network for the unlabeled image data; Step 3.2.3: Calculate the semi-supervised loss using the cross entropy loss function between the pseudo-label and the segmentation network output; the semi-supervised loss (L semi ) is calculated as: in: Represents a pseudo image generated by the segmentation network; I represents an indicator function, which returns a specific value, usually 1 or 0, according to the input conditions; P n Represents the prediction result image.

6. The crop disease segmentation method based on generative adversarial semi-supervised learning according to claim 1, characterized in that: The segmentation network in step 2 adopts an encoder-decoder structure; the encoder is responsible for converting the input image into a high-dimensional feature vector, while the decoder is responsible for mapping the high-dimensional feature vector back to the pixel space, and finally obtaining the prediction result.

7. The crop disease segmentation method based on generative adversarial semi-supervised learning according to claim 6, characterized in that: The encoder uses dilated convolution to extract features calculated by deep convolutional neural networks at any resolution. By applying dilated convolutions with different rates to the spatial pyramid ASPP module and combining image-level features, convolutional features can be obtained at multiple scales, thereby extracting high-level features in the image. The decoder performs an upsample operation on the feature map obtained by the ASPP module, restores it to a feature map of the same size as the original input image, and fuses the shallow feature map.

8. A crop disease segmentation system based on generative adversarial semi-supervised learning, characterized by: include: Data collection module: collects crop disease images in different environments and growth stages as data sets, labels the data sets, and indicates the categories to which they belong; Adversarial training module: Use Deeplabv3+ as the segmentation network, send the data set and the labeled data set to the segmentation network for preliminary training, and then introduce the discriminant network to perform adversarial training on the segmentation network; Loss-supervised training module: introduces the loss function to jointly act against the training process of the training module; Model building module: Through the continuous iteration of the training process of the adversarial training module and the loss supervision training module, a trained generative adversarial semi-supervised segmentation model (semi) is obtained; Disease segmentation module: The collected crop disease images are input into the trained generative adversarial semi-supervised segmentation model (semi) to obtain the crop disease segmentation results.

9. A crop disease segmentation device based on generative adversarial semi-supervised learning, characterized by: include: Memory: used to store a computer program to implement a crop disease segmentation method based on generative adversarial semi-supervised learning as described in any one of claims 1 to 7; Processor: used to implement a crop disease segmentation method based on generative adversarial semi-supervised learning as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the crop disease segmentation method based on generative adversarial semi-supervised learning are implemented.

Images