Wheat stripe rust urediniospore monitoring method based on spin Unet++ network

Abstract

The present invention relates to a method for monitoring summer spores of wheat stripe rust based on a spin Unet++ network, including: obtaining microscopic images of summer spores; constructing a sample data set; dividing the sample data set into a training set, a test set, and a validation set; improving the Unet++ network model to obtain an improved Unet++ network model; inputting the images in the sample data set into the improved Unet++ network model for spin training to obtain a summer spore counting network model, and outputting the images in the sample data set with detection frames; training the summer spore counting network model; inputting the microscopic images of the summer spores to be detected into the trained summer spore counting network model, and outputting images with detection frames and counts. In the present invention, the counting accuracy of the summer spore counting network model is high, reaching 99.03%; the segmentation rate is relatively high, reaching 86.45%; the detection rate is relatively high, reaching 14 images per second, and the memory occupancy is small, only 46.8 MB.

Description

Technical Field

[0001] The present invention relates to the technical field of agricultural image processing, and in particular to a method for monitoring urediniospores of wheat stripe rust based on a spin Unet++ network. Background Art

[0002] Wheat stripe rust is one of the most serious diseases endangering wheat yield, seriously threatening the food production safety in China. This disease is mainly caused by Puccinia striiformis f. sp. tritici. When it occurs, a large number of uredinia will be generated on the front side of the leaves, and the leaves will wither and die in the later stage. As a large-scale epidemic airborne fungal disease, stripe rust has been widely distributed in the northwest, north China, the middle and lower reaches of the Yangtze River and other regions of China. Through recent research, it has been shown that the main transmission medium of wheat stripe rust is the stripe rust spores, and among them, the most influential is the urediniospores. The urediniospore source infects most of the wheat producing areas in China through air flow activities, thus forming large-scale outbreaks and epidemics. Therefore, how to quickly and accurately detect the early urediniospore source is crucial for the early prevention and control of wheat stripe rust, and is of great significance for reducing wheat yield losses and ensuring food security.

[0003] In recent years, many monitoring studies on fungal spores of diseases have emerged and good results have been achieved. Li Xiaolong et al. performed processing such as nearest neighbor interpolation, K-means clustering segmentation, morphological operation modification, and watershed segmentation on the microscopic images of wheat stripe rust urediniospores, realizing the automatic counting and marking of urediniospores. However, this method is relatively complex and the counting rate is low. Lei Yu et al. proposed a high-efficiency and high-precision microscopic image acquisition device for urediniospores based on an ARK-1123C type embedded industrial controller and a microscope CCD digital camera. This device can remotely and real-time collect microscopic images of urediniospores with a size of 4096×3288, but this device does not have the ability of automatic counting. Lei Yu et al. combined the morphological characteristics of urediniospores and improved the CenterNet network, achieving a detection accuracy of 98.77%, but the spore segmentation overlap degree of this algorithm is relatively low. Summary of the Invention

[0004] The purpose of the present invention is to provide a method for monitoring urediniospores of wheat stripe rust based on a spin Unet++ network, which solves the problem of accurate segmentation of microscopic images of urediniospores of wheat stripe rust and realizes the rapid and accurate detection of microscopic images of urediniospores of wheat stripe rust.

[0005] To achieve the above purpose, the present invention adopts the following technical solutions: A method for monitoring urediniospores of wheat stripe rust based on a spin Unet++ network, the method includes the following steps in sequence:

[0006] (1) Obtain urediniospore microscopic images: Use a spore trap to capture urediniospores of Puccinia striiformis f. sp. tritici in the air in the field, and collect microscopic images under a microscope to obtain urediniospore microscopic images;

[0007] (2) Construct a sample dataset: Use the LabelImg tool to annotate and save the obtained urediniospore microscopic images to obtain a sample dataset;

[0008] (3) Divide the sample dataset: Divide the sample dataset into a training set, a test set, and a validation set;

[0009] (4) Improve the Unet++ network model: Set the sizes and numbers of channels at the input and output ends of the Unet++ network model to be the same, and perform a spin connection, that is, connect the output end of the Unet++ network model to the input end, and increase the spin coefficient K for iterative training to obtain an improved Unet++ network model;

[0010] (5) Construct a urediniospore counting network model: Input the images in the sample dataset, as well as the positions, sizes, and rotation angles of the target information of the images, into the improved Unet++ network model for spin training to obtain a urediniospore counting network model, and output the corresponding heat map, elliptical center point, major and minor axes of the ellipse, and elliptical rotation angle information, as well as the images in the sample dataset with detection frames;

[0011] (6) Train the urediniospore counting network model: Input the training set into the urediniospore counting network model for training to obtain a trained urediniospore counting network model;

[0012] (7) Obtain the urediniospore microscopic image to be detected, and input the urediniospore microscopic image to be detected into the trained urediniospore counting network model to output an image with a detection frame and a count.

[0013] The specific content of step (1) is as follows: Use a TPBZ3 type spore trap to simulate the capture of urediniospores of Puccinia striiformis f. sp. tritici in the air in the field. In the wild wheat field, take out the glass slide carrier of the TPBZ3 type spore trap, and place a glass slide evenly coated with vaseline on the carrier. After capturing for one minute, take out the glass slide to obtain glass slides of urediniospores of Puccinia striiformis f. sp. tritici with capture times of 60, 120, 180, and 240 min respectively. Repeat multiple times to obtain glass slides with different spore densities as required;

[0014] Use a BX52 type inverted microscope to observe and photograph the glass slide, with a magnification of 10×20. Under the microscope, collect microscopic images of each glass slide, randomly select 5 fields of view for each slide to take pictures to obtain urediniospore microscopic images, and store them in jpg format.

[0015] The specific content of step (3) is as follows: the sample data set is divided into a training set, a validation set and a test set according to the ratio of 8:1:1;

[0016] In step (5), when calculating the heat map, rectangular conversion is required. The weighted mapping is used to calculate the mapped rectangle and input it into the network. The calculation formula is as follows:

[0017] R a = xa + a(1 - x)cosθ (5)

[0018] R b = xb + b(1 - x)sinθ (6)

[0019] In the formula, R a and R b are the length and width of the rectangle respectively, x is the weighting coefficient, a is the major axis of the ellipse, b is the minor axis of the ellipse, and θ is the rotation angle of the ellipse.

[0020] In step (5), the detection loss function L of the uredospore counting network model is calculated as follows:

[0021] L = λ heatmap L heatmap + λ ab L ab + λ offset L offset + λ ang L ang (3)

[0022]

[0023]

[0024]

[0025]

[0026] In the formula, L is the total loss, L offset is the bias loss, L ab is the major and minor axis loss, L heatmap is the heat map loss, L ang is the angle loss, N is the number of categories, is the confidence that the pixel point (x, y) belongs to the category c, α and β are hyperparameters, and their values are 2 and 4 respectively; is the predicted major and minor axes, s k is the true major and minor axes; R is the downsampling factor, is the predicted center point, p is the true center point, is the predicted center point bias, λ heatmap is the heat map loss coefficient, λab is the loss coefficient of the major and minor axes, λ offset is the offset loss coefficient, λ ang The angular loss coefficient, A is the predicted angle, is the true angle.

[0027] As can be seen from the above technical solutions, the beneficial effects of the present invention are as follows: First, the counting accuracy of the uredospore counting network model in the present invention is high, reaching 99.03%; Second, the segmentation rate of the uredospore counting network model in the present invention is relatively high, reaching 86.45%; Third, the detection rate of the uredospore counting network model in the present invention is relatively high, reaching 14 images per second.; Fourth, the memory occupied by the uredospore counting network model in the present invention is small, only 46.8MB. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is the flowchart of the method of the present invention;

[0029] Figure 2 is the structural diagram of the improved Unet++ network model in the present invention;

[0030] Figure 3 is the structural diagram of the uredospore counting network model;

[0031] Figure 4 is the detection image with detection frames and counting. DETAILED DESCRIPTION OF THE INVENTION

[0032] As Figure 1 shown, a method for monitoring uredospores of wheat stripe rust based on a spin Unet++ network, the method includes the following steps in sequence:

[0033] (1) Obtain uredospore microscopic images: Use a spore trap to capture uredospores of wheat stripe rust in the air in the field and collect microscopic images under a microscope to obtain uredospore microscopic images;

[0034] (2) Construct a sample data set: Label and save the obtained uredospore microscopic images through the LabelImg tool to obtain a sample data set;

[0035] (3) Divide the sample data set: Divide the sample data set into a training set, a test set and a validation set;

[0036] (4) As Figure 2 shown, improve the Unet++ network model: Set the sizes and number of channels at the input end and output end of the Unet++ network model to be the same, and perform spin connection, that is, connect the output end of the Unet++ network model to the input end, and add a spin coefficient K for iterative training to obtain the improved Unet++ network model;

[0037] (5) As shown in Figure 3 the figure, construct a uredospore counting network model: input the images in the sample dataset, as well as the position, size, and rotation angle of the target information of the image, into the improved Unet++ network model for spin training to obtain the uredospore counting network model, and output the corresponding heat map, elliptical center point, major and minor axes of the ellipse, and elliptical rotation angle information, as well as the images in the sample dataset with detection frames;

[0038] (6) Train the uredospore counting network model: input the training set into the uredospore counting network model for training to obtain the trained uredospore counting network model;

[0039] (7) Obtain the microscopic images of uredospores to be detected, input the microscopic images of uredospores to be detected into the trained uredospore counting network model, and output the images with detection frames and counts, as shown in Figure 4 the figure.

[0040] Specifically, step (1) means: use a TPBZ3 type spore trap to simulate capturing the uredospores of wheat stripe rust in the field air. In the wild wheat field, take out the glass slide carrier of the TPBZ3 type spore trap, and place a glass slide evenly coated with vaseline on the carrier. After capturing for one minute, take out the glass slide to obtain the glass slides of uredospores of stripe rust with capture times of 60, 120, 180, and 240 minutes respectively. Repeat multiple times to obtain the glass slides with different spore densities required;

[0041] Observe and photograph the glass slides with a BX52 type inverted microscope at a magnification of 10×20. Under the microscope, collect microscopic images of each glass slide, randomly select 5 fields of view for each slide to take pictures to obtain microscopic images of uredospores, and store them in jpg format.

[0042] Specifically, step (3) means: divide the sample dataset into a training set, a validation set, and a test set according to a ratio of 8:1:1;

[0043] In step (5), when calculating the heat map, rectangular conversion is required. The weighted mapping is used to calculate the mapped rectangle and input it into the network. The calculation formula is as follows:

[0044] R a = xa + a(1 - x)cosθ (1)

[0045] R b = xb + b(1 - x)sinθ (2)

[0046] In the formula, R a and R bare the length and width of the rectangle respectively, x is the weighting coefficient, a is the major axis of the ellipse, b is the minor axis of the ellipse, and θ is the rotation angle of the ellipse.

[0047] In step (5), the detection loss function L of the uredospore counting network model is calculated as follows:

[0048] L = λ heatmap L heatmap + λ ab L ab + λ offset L offset + λ ang L ang (3)

[0049]

[0050]

[0051]

[0052]

[0053] In the formula, L is the total loss, L offset is the bias loss, L ab is the major and minor axis loss, L heatmap is the heatmap loss, L ang is the angle loss, N is the number of classes, is the confidence that the pixel point (x, y) belongs to class c, α and β are hyperparameters, and their values are 2 and 4 respectively; is the predicted major and minor axes, S k is the true major and minor axes; R is the downsampling factor, is the predicted center point, p is the true center point, is the predicted center point bias, λ heatmap is the heatmap loss coefficient, λ ab is the major and minor axis loss coefficient, λ offset is the bias loss coefficient, λ ang is the angle loss coefficient, A is the predicted angle, is the true angle.

[0054] The same dataset was used to train the original Centernet model and the uredospore counting network model in the same batch. All models were trained in the same software and hardware environment. The operating system was Windows 11, the processor was 5950X, and the graphics card was 1 GPU 3090 with a video memory size of 24GB. The Pytorch 1.7 deep learning framework was used for training, and the number of dataset images was 21,420. The dataset was divided into a training set, a validation set, and a test set in the ratio of 8:1:1. Among them, the test set was divided into three categories according to the number of spores in the image: less than 10, 10 - 30, and more than 30, and tested separately. During training, the sample batch size was 4, the training was carried out for 20 epochs, and the learning rate was 1.25×10 -4 .

[0055] In the present invention, K = 3 was selected as the experimental spin coefficient. Under the same conditions, when using different mapping rectangle weighting coefficients (x = 0.1, 0.3, 0.5, 0.7, 0.9) in the feature extraction network with the improved Unet++ (spin connection with K = 3 layers) network, training was carried out in the same environment. The experimental comparison data is shown in Table 1.

[0056] Table 1 Accuracy and recall rate of different weighting coefficients

[0057] Table 1 Accuracy and recall rate of different weighting coefficients%

[0058]

[0059] It can be seen from Table 1 that by changing the size of the weighting coefficient, the accuracy of the uredospore counting network model of the present invention reaches the highest at the weighting coefficient x = 0.5, which is 99.03%, an increase of 0.49 percentage points compared with the accuracy of the original CenterNet model, and the segmentation rate increases by 10.35 percentage points. When the weighting coefficient x = 0.9, the segmentation rate reaches the highest, which is 87.56%, but the accuracy decreases. Therefore, in order to ensure that the network has a high counting accuracy and obtain a high segmentation rate, the present invention selects the weighting coefficient x = 0.5 for subsequent experiments.

[0060] Using the original CenterNet model and the uredospore counting network model of the present invention, experiments were carried out on the to-be-detected images with different spore densities, and the data shown in Table 2 was obtained:

[0061] Table 2 Comparison of detection results of three algorithms for spores with different densities

[0062] Table 2 Comparison of detection results of three algorithms for spores with different densities

[0063]

[0064] As can be seen from Table 2, for the urediniospore images with different numbers of spores, due to the increase in spore density, the detection difficulty will also increase. And when the number of spores increases to more than 30, a large number of adhesion phenomena will occur in the image, making it easy to produce false detections and missed detections in the detection results. However, the improved model urediniospore counting network model proposed in the present invention has reached the highest accuracy and coincidence rate under three densities. The accuracy rate has reached 99.03%, and the coincidence rate has reached 86.45%. In addition, the size of the urediniospore counting network model has been reduced by 66.09%, only 46.8 MB, making the urediniospore counting network model easier to be deployed to servers or remote devices. In summary, the urediniospore counting network model proposed in the present invention can provide an effective method support for the automatic counting method of wheat stripe rust urediniospores.

Claims

1. A method for monitoring the urediniospores of wheat stripe rust based on a spin Unet++ network, characterized in that: The method includes the following steps in sequence: (1) Obtain urediniospore microscopic images: Use a spore trap to capture urediniospores of Puccinia striiformis f. sp. tritici in the field air, and collect microscopic images under a microscope to obtain urediniospore microscopic images; (2) Construct a sample data set: Label and save the obtained urediniospore microscopic images through the LabelImg tool to obtain a sample data set; (3) Divide the sample data set: Divide the sample data set into a training set, a test set, and a validation set; (4) Improve the Unet++ network model: Set the sizes and numbers of channels at the input end and output end of the Unet++ network model to be the same, and perform spin connection, that is, connect the output end of the Unet++ network model to the input end, and add a spin coefficient K for iterative training to obtain an improved Unet++ network model; (5) Construct a urediniospore counting network model: Input the images in the sample data set, as well as the positions, sizes, and rotation angles of the target information of the images, into the improved Unet++ network model for spin training to obtain a urediniospore counting network model, and output the corresponding heat map, ellipse center point, ellipse major and minor axes, and ellipse rotation angle information, as well as the images in the sample data set with detection frames; (6) Train the urediniospore counting network model: Input the training set into the urediniospore counting network model for training to obtain a trained urediniospore counting network model; (7) Obtain the urediniospore microscopic image to be detected, input the urediniospore microscopic image to be detected into the trained urediniospore counting network model, and output an image with a detection frame and count.

2. The wheat stripe rust urediniospore monitoring method based on the spin Unet++ network according to claim 1, characterized in that: The specific content of step (1) is as follows: Use a TPBZ3 type spore trap to simulate the capture of urediniospores of Puccinia striiformis f. sp. tritici in the field air. In the wild wheat field, take out the glass slide carrier of the TPBZ3 type spore trap, and place a glass slide evenly coated with vaseline on the carrier. After capturing for one minute, take out the glass slide to obtain glass slides of urediniospores of Puccinia striiformis f. sp. tritici with capture times of 60, 120, 180, and 240 min respectively. Repeat multiple times to obtain glass slides with different spore densities as required; Use a BX52 type inverted microscope to observe and photograph the glass slides, with a magnification of 10×20. Under the microscope, collect microscopic images of each glass slide, randomly select 5 fields of view for each slide to take pictures to obtain urediniospore microscopic images, and store them in jpg format.

3. The wheat stripe rust urediniospore monitoring method based on the spin Unet++ network according to claim 1, characterized in that: The specific content of step (3) is as follows: Divide the sample data set into a training set, a validation set, and a test set according to a ratio of 8:1:

1.

4. The wheat stripe rust urediniospore monitoring method based on the spin Unet++ network according to claim 1, characterized in that: In step (5), when calculating the heat map, rectangular conversion is required, and the weighted mapping is used to calculate the mapped rectangle and input it into the network. The calculation formula is as follows: R a = xa + a(1 - x)cosθ (1) R b = xb + b(1 - x)sinθ (2) wherein, R a and R b are the length and width of the rectangle respectively, x is the weighting coefficient, a is the major axis of the ellipse, b is the minor axis of the ellipse, and θ is the rotation angle of the ellipse.

5. The wheat stripe rust urediniospore monitoring method based on the spin Unet++ network according to claim 1, characterized in that: In step (5), the calculation formula of the detection loss function L of the urediniospore counting network model is as follows: L = λ heatmap L heatmap + λ ab L ab + λ offset L offset + λ ang L ang (3) where \(L\) is the total loss, \(L_{ offset}\) offset is the bias loss, \(L_{ ab}\) ab is the major and minor axis loss, \(L_{ heatmap}\) heatmap is the heatmap loss, \(L_{ ang}\) ang is the angle loss, \(N\) is the number of classes, is the confidence that the pixel point \((x, y)\) belongs to class \(c\), \(\alpha\) and \(\beta\) are hyperparameters with values of 2 and 4 respectively; is the predicted major and minor axis, \(s_{ k}\) k is the true major and minor axis; \(R\) is the downsampling factor, is the predicted center point, \(p\) is the true center point, is the predicted center point offset, \(\lambda_{ heatmap}\) heatmap is the heatmap loss coefficient, \(\lambda_{ ab}\) ab is the major and minor axis loss coefficient, \(\lambda_{ offset}\) offset is the bias loss coefficient, \(\lambda_{ ang}\) ang is the angle loss coefficient, \(A\) is the predicted angle, is the true angle.

Images