Wheat stripe rust spore image detection method and system based on Yolov5s

Abstract

The present invention discloses a method and system for detecting wheat stripe rust spores using microscopic images based on Yolov5s. The method comprises the following steps: acquiring spore microscopic images; preprocessing the spore microscopic images; constructing a spore detection network, which uses a clustering algorithm to re-determine preset anchor frames, incorporates a C3TR unit using a transformer model in the Backbone module, and employs a lightweight upsampling layer in the Neck module; training the spore detection network; and detecting and counting spores using the trained spore detection network. The present invention can accurately detect spores in spore microscopic images, while minimizing the number of model parameters.

Description

Technical Field

[0001] The present invention relates to the technical field of image detection, and in particular to a method and system for detecting wheat stripe rust spore microscopic images based on Yolov5s. Background Art

[0002] Wheat stripe rust is a major biological disaster affecting wheat production safety in my country. The spores of the pathogen, Puccinia striata, can be spread long distances by high-altitude air currents, leading to explosive outbreaks and widespread epidemics. Therefore, monitoring spore counts in the early stages of the disease and implementing rapid and accurate response measures can minimize economic losses and pesticide use, ultimately ensuring a good wheat harvest.

[0003] However, since spores are small and densely sticky, and are affected by dust and other impurities when collecting spores, there are many impurities in the spore microscopic images, resulting in missed and false detection problems in spore detection. When spores partially overlap, it will affect the accuracy of the algorithm. Summary of the Invention

[0004] To address the problems of the above-mentioned existing technologies, the present invention provides a method and system for detecting wheat stripe rust spores using microscopic images based on Yolov5s. This method can accurately detect and count spores in microscopic images with a low miss detection rate, meeting the lightweight application requirements of edge device deployment. The technical solution is as follows:

[0005] In a first aspect, a method for detecting wheat stripe rust spores by microscopic imaging based on Yolov5s is provided, comprising the following steps:

[0006] A pre-built Yolov5s-based spore detection model is trained based on a sample set. The training process includes: clustering the labeled bounding boxes of samples in the sample set based on size to obtain labeled bounding boxes of multiple category sizes, and determining the size of a preset anchor box of the Yolov5s model based on the sizes of the labeled bounding boxes of the multiple category sizes;

[0007] Spore detection is performed on the microscopic image to be detected based on the trained spore detection model to obtain a spore detection result. The spore detection model processes the input image in the following manner:

[0008] (1) Image feature extraction in the Backbone module;

[0009] (2) The image features extracted in (1) are fused in the Neck module;

[0010] (3) In the Prediction module, target detection is performed based on the image features fused in (2) to obtain the location of stripe rust spores.

[0011] In some preferred embodiments, in (1), the step of extracting image features includes:

[0012] (11) performing feature extraction in sequence using four sequentially connected feature extraction layers, and obtaining a first feature map output by the second feature extraction layer and a second feature map output by the third feature extraction layer, wherein the first feature extraction layer, the second feature extraction layer, and the third feature extraction layer include sequentially connected CBS units and CSP1 units, and the fourth feature extraction layer includes sequentially connected CBS units and SPP units;

[0013] (12) The feature map output by the fourth feature extraction layer is processed by a C3TR unit to obtain a third feature map. The C3TR unit uses the first path to perform convolution processing and the second path to perform convolution processing and self-attention mechanism. The output of the first path and the output of the second path are spliced ​​and fused, and the spliced ​​and fused results are convolution processed.

[0014] In some preferred embodiments, in (12), the self-attention mechanism of the second path is implemented based on the transformer model unit, and the self-attention mechanism action unit includes a plurality of sequentially connected blocks, each block includes a multi-head attention module and a feedforward neural network connected in sequence, and the multi-head attention module and the feedforward neural network respectively adopt a residual connection structure, and the image after convolution processing in the second path is recorded as the first image, and the image embedding and position embedding of the first image are input into the multi-head attention module to obtain the attention feature, and the attention feature is input into the feedforward neural network to obtain the output of the self-attention mechanism action.

[0015] In some preferred embodiments, said (2) includes:

[0016] The feature fusion is performed in sequence using four sequentially connected feature fusion layers, wherein the first feature fusion layer and the second feature fusion layer include sequentially connected CBS units, upsampling layers, concat fusion layers, and CSP2 units, and the third feature fusion layer and the fourth feature fusion layer include sequentially connected CBS units, concat fusion layers, and CSP2 units;

[0017] In the feature fusion layer, the first feature map is input into the concat fusion layer of the second feature fusion layer, the second feature map is input into the concat fusion layer of the first feature fusion layer, and the third feature map is input into the input of the Neck module, i.e., the CBS unit of the first feature fusion layer;

[0018] The output of the CBS unit of the first feature fusion layer is simultaneously input into the concat fusion layer of the fourth feature fusion layer, and the output of the CBS unit of the second feature fusion layer is simultaneously input into the concat fusion layer of the third feature fusion layer;

[0019] The outputs of the CSP2 units of the second feature fusion layer, the third feature fusion layer, and the fourth feature fusion layer are respectively recorded as the first fused feature map, the second fused feature map, and the third fused feature map, which are input to the Prediction module.

[0020] In some preferred embodiments, the upsampling layer comprises:

[0021] (21) Use a 1*1 convolution kernel to reduce the dimension of the feature image. The dimension reduction formula is as follows:

[0022] C m =σ 2 (K up ) 2

[0023] Among them C m is the number of feature layer channels after dimensionality reduction, σ is the upsampling multiple, K up is the predicted upsampling kernel size;

[0024] (22) Expand the channel dimension in the spatial dimension by reorganizing the width, height, and number of channels of the feature map into σW, σH, and K up 2 Get the predicted upsampling kernel;

[0025] (23) The prediction results are normalized by softmax so that the sum of the weights of the convolution kernel is 1;

[0026] (24) Multiply the features on each layer of feature map by the predicted upsampling kernel to obtain the upsampling result.

[0027] In some preferred embodiments, during the training of the spore detection model, the labeled bounding boxes of the samples in the sample set are clustered based on their sizes using the following steps:

[0028] Take the marked border as the target to be clustered and obtain the distance between any two targets to be clustered;

[0029] For any target to be clustered, obtain the number of targets whose distances between any target to be clustered and other targets are less than a first preset value, which is recorded as the first number;

[0030] For the current target to be clustered, obtain the target to be clustered B whose first number is greater than the first number of the current target to be clustered A, obtain the cumulative sum of the distances between the current target to be clustered and all the targets to be clustered B and record it as the second value;

[0031] The order in which the clustered objects become cluster centers is determined based on the product of the first number and the second number, wherein the larger the product of the first number and the second number, the higher the order in which the objects become cluster centers.

[0032] Based on multiple determined cluster centers, for the remaining targets C to be clustered, sort them from large to small based on the size of the first number of each target C to be clustered, and determine the cluster to which each target C to be clustered belongs in turn. For each target C to be clustered, divide it into the cluster where the cluster center that is closest to the target C to be clustered and whose first number is greater than the target C to be clustered is located.

[0033] In some preferred embodiments, determining the order in which the objects to be clustered become cluster centers based on the product of the first number and the second value further includes:

[0034] Sort the product values ​​based on the size of the product of the first number and the second number;

[0035] The product value sorting sequence is analyzed for the mutation point of the change trend. The target to be clustered corresponding to the product value before the mutation point is the cluster center, and the target to be clustered corresponding to the product value after the mutation point is the non-cluster center.

[0036] In some preferred embodiments, the target to be clustered corresponding to the product value before the mutation point is a cluster center, and further comprising:

[0037] The target to be clustered corresponding to the product value before the mutation point is the candidate cluster center;

[0038] For the current candidate cluster center C, obtain the first target D to be clustered whose number is greater than the first number of the current candidate cluster center C, obtain the minimum distance between the target to be clustered D and the current candidate cluster center C and record it as the first distance. If the first distance is less than the second preset value, the candidate cluster center C is determined as a non-cluster center.

[0039] In some preferred embodiments, the training process of the spore detection model further includes: performing image enhancement on samples in the sample set, wherein the image enhancement includes expanding through the styleGAN network and then rotating 180°, flipping, and rotating 180° before flipping and expanding.

[0040] In a second aspect, a wheat stripe rust spore microscopic image detection system based on Yolov5s is provided, comprising:

[0041] A spore detection model training unit is used to train a pre-built Yolov5s-based spore detection model based on a sample set. The training process includes: clustering the labeled bounding boxes of samples in the sample set based on size to obtain labeled bounding boxes of multiple category sizes, and determining the size of a preset anchor box of the Yolov5s model based on the sizes of the labeled bounding boxes of the multiple category sizes;

[0042] The spore detection unit is used to perform spore detection on the microscopic image to be detected based on the trained spore detection model to obtain a spore detection result. The spore detection model processes the input image including: (1) extracting image features in the Backbone module; (2) fusing the image features extracted in (1) in the Neck module; and (3) performing target detection on the fused image features in (2) in the Prediction module to obtain the position of stripe rust spores.

[0043] The present invention provides a Yolov5s-based method and system for detecting wheat stripe rust spores in microscopic images, exhibiting the following beneficial effects: First, compared with existing technologies, it can accurately detect and count spores in microscopic spore images, and the model is smaller. Second, the present invention optimizes the feature extraction network Backbone module and the feature fusion network Neck module, while also optimizing the prediction anchor frame. This method achieves a detection accuracy of 96.68% for wheat stripe rust spores, with a missed detection rate of 1.55%, meeting the application requirements of terminal device deployment. Third, compared with existing technologies, the present invention can detect dense sticky spores in complex environments, demonstrating broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is a flow chart of the microscopic image detection method for wheat stripe rust spores based on Yolov5s;

[0045] Figure 2 This is the original microscopic image of the spore;

[0046] Figure 3 Schematic diagram of the structure of the spore detection model of the embodiment of the present application;

[0047] Figure 4 Schematic diagram of the structure of the C3TR unit in the spore detection model of the present application embodiment;

[0048] Figure 5 1 is a schematic diagram of spore image detection results achieved by the detection method of an embodiment of the present application;

[0049] Figure 6 2 is another schematic diagram of the spore image detection results achieved by the detection method according to an embodiment of the present application. DETAILED DESCRIPTION

[0050] The specific embodiments of the present invention are further described below.

[0051] See also Figure 1 The present invention provides a method for detecting wheat stripe rust spores by microscopic imaging based on Yolov5s, comprising the following steps:

[0052] Step 1: Obtain the original microscopic image of the spores, such as Figure 2 As shown;

[0053] Step 2: Preprocess the original microscopic images of spores, then perform data augmentation on the images to construct a spore dataset for annotation. Finally, the spore dataset is divided into training, validation, and test sets in a 6:2:2 ratio.

[0054] Step 3: construct a spore detection model. The detection model is constructed based on Yolov5s and improved based on a preset improvement method; the spore detection model is referred to as the Yolo-Spore model below.

[0055] Step 4: Use the training set and validation set to train the spore detection model;

[0056] Step 5: After the model training is completed, a weight file is generated. The image to be detected and the trained weights are input to obtain the prediction results. The wheat stripe rust spores in the microscopic image are located and counted based on the spore prediction results.

[0057] The preprocessing in step 2 involves removing redundant images with high similarity and overly blurred images due to camera shake. Data augmentation in step 2 involves augmenting the image with the styleGAN network, followed by a 180° rotation, flipping, and then a 180° rotation followed by flipping and augmentation.

[0058] The entire process of StyleGAN can be divided into the following steps:

[0059] a. Generate a latent vector z by random sampling from a standard normal distribution as the initial input;

[0060] b. Map the potential vector z to the feature space F through the mapping network (G_mapping);

[0061] c. Use the generative network (G_generator) to convert the feature space F into the image I;

[0062] d. Use the discriminator network (D_adversarial) to evaluate the realism of the generated image I;

[0063] e. Based on the output of the discriminator, update the parameters of the mapping network and the generation network through the principle of adversarial training;

[0064] f. Repeat the above steps until the generated image reaches a satisfactory quality.

[0065] In step 2, labeling involves using the LabelImg tool to annotate the spore image with a rectangular box, generating the initial Yolo format annotation data. The Yolo format is (cls, x1, y1, x2, y2), where cls represents the class of the target. x1 and y1 represent the x and y coordinates of the center of the labeled box. x2 and y2 represent the width and height of the labeled box.

[0066] In step 3 above, the spore detection model processes the input image as follows:

[0067] Step 31: extract image features in the Backbone module;

[0068] Step 32, fusing the image features extracted in step 31 in the Neck module;

[0069] In step 33, the Prediction module performs target detection on the image features fused in step 32 to obtain the location of stripe rust spores.

[0070] See also Figure 3 and Figure 4 In the embodiment of the present application, the spore detection model is improved based on the Yolov5s model. Specifically, the processing process of the Backbone module in step 31 above includes:

[0071] Step 311, performing feature extraction in sequence using four sequentially connected feature extraction layers, and obtaining a first feature map output by the second feature extraction layer and a second feature map output by the third feature extraction layer, wherein the first feature extraction layer, the second feature extraction layer, and the third feature extraction layer include sequentially connected CBS units and CSP1 units, and the fourth feature extraction layer includes sequentially connected CBS units and SPP units;

[0072] In step 312, the feature map output by the fourth feature extraction layer is processed by a C3TR unit to obtain a third feature map. The C3TR unit uses the first path to perform convolution processing and the second path to perform convolution processing and self-attention mechanism. The first path output and the second path output are spliced ​​and fused, and the spliced ​​and fused results are convolution processed.

[0073] The four sequentially connected feature extraction layers are denoted as the first feature extraction layer, the second feature extraction layer, the third feature extraction layer, and the fourth feature extraction layer. The CBS unit consists of a Conv2d convolutional layer, a BN normalization layer, and a SiLU activation function. The C3TR unit performs convolution processing and an attention mechanism on the input image and uses a residual connection structure to obtain a third feature map. In the embodiment of the present application, efficient feature information is obtained through the self-attention mechanism of the C3TR unit. It should be noted that, consistent with conventional Yolov5s, the spore detection model implemented in this application also has a Focus unit before the first feature extraction layer to perform a slice convolution operation on the input image.

[0074] In the above step 312, the self-attention mechanism of the second path is implemented based on the transformer model unit. The self-attention mechanism action unit includes multiple sequentially connected blocks, and each block includes a multi-head attention module and a feedforward neural network connected in sequence. The multi-head attention module and the feedforward neural network respectively adopt a residual connection structure. The image after convolution processing in the second path is recorded as the first image. The image embedding and position embedding of the first image are input into the multi-head attention module to obtain attention features, and the attention features are input into the feedforward neural network to obtain the output of the self-attention mechanism action.

[0075] In an embodiment of the present application, a multi-head self-control mechanism with residual structure connection is adopted in the C3TR unit, and the transformer model unit is used to calculate the correlation between all features of the entire feature map to obtain feature global information and sufficient context information.

[0076] The processing process of the Neck module in the above step 32 adopts the following method: four sequentially connected feature fusion layers are used to perform feature fusion in sequence, wherein the first feature fusion layer and the second feature fusion layer include a CBS unit, an upsampling layer, a concat fusion layer, and a CSP2 unit connected in sequence, and the third feature fusion layer and the fourth feature fusion layer include a CBS unit, a concat fusion layer, and a CSP2 unit connected in sequence; in the feature fusion layer, the first feature map is input into the concat fusion layer of the second feature fusion layer, the second feature map is input into the concat fusion layer of the first feature fusion layer, and the third feature map is input into the input of the Neck module, i.e., the CBS unit of the first feature fusion layer; the output of the CBS unit of the first feature fusion layer is simultaneously input into the concat fusion layer of the fourth feature fusion layer, and the output of the CBS unit of the second feature fusion layer is simultaneously input into the concat fusion layer of the third feature fusion layer; the outputs of the CSP2 units of the second feature fusion layer, the third feature fusion layer, and the fourth feature fusion layer are respectively recorded as the first fusion feature map, the second fusion feature map, and the third fusion feature map, and the first fusion feature map, the second fusion feature map, and the third fusion feature map are input into the Prediction module.

[0077] The four sequentially connected feature fusion layers are respectively recorded as the first feature fusion layer, the second feature fusion layer, the third feature fusion layer and the fourth feature fusion layer; in the embodiment of the present application, the Neck module includes two upsampling processes. In the first feature fusion layer, the result of the first upsampling is fused with the second feature map output by the Backbone module; in the second feature fusion layer, the result of the second upsampling is fused with the first feature map output by the Backbone module; after entering the third feature fusion layer, the CBS unit output of the third feature fusion layer is fused with the CBS unit output of the second feature fusion layer; after entering the fourth feature fusion layer, the CBS unit output of the fourth feature fusion layer is fused with the CBS unit output of the first feature fusion layer; the fusion results in the second feature fusion layer, the third feature fusion layer and the fourth feature fusion layer are respectively subjected to enhanced network feature fusion by a CSP2 unit to output the first fused feature map, the second fused feature map and the third fused feature map, and the Prediction module uses the first fused feature map, the second fused feature map and the third fused feature map for spore detection.

[0078] Furthermore, the upsampling process of the Neck module upsampling layer in the above step 32 includes the following steps:

[0079] In step 321, a 1*1 convolution kernel is used to reduce the dimension of the feature image. The dimension reduction formula is as follows:

[0080] C m =σ 2 (Kup ) 2

[0081] Among them C m is the number of feature layer channels after dimensionality reduction, σ is the upsampling multiple, K up is the predicted upsampling kernel size;

[0082] Step 322, expand the channel dimension in the spatial dimension by reorganizing the width, height, and number of channels of the feature map into σW, σH, K up 2 Get the predicted upsampling kernel;

[0083] Step 323: normalize the prediction result through softmax so that the sum of the weights of the convolution kernel is 1;

[0084] In step 324, the features on each layer of feature map are multiplied by the predicted upsampling kernel to obtain an upsampling result.

[0085] In an embodiment of the present application, a lightweight upsampling layer is used, which can generate corresponding upsampling kernels for features, thereby increasing the corresponding weights of the target features of interest.

[0086] Regarding step 4 above, the training process of the spore detection model, in the embodiment of the present application, a pre-built Yolov5s-based spore detection model is trained based on a sample set. The training process includes: clustering the labeled bounding boxes of the samples in the sample set based on size to obtain labeled bounding boxes of multiple category sizes, and determining the size of the preset anchor boxes of the Yolov5s model based on the sizes of the labeled bounding boxes of the multiple category sizes. The embodiment of the present application optimizes the anchor box size by re-clustering the data set to obtain the anchor box, thereby improving the accuracy of target detection of the spore detection model. The clustering method includes:

[0087] Step 41: take the marked border as the target to be clustered and obtain the distance between any two targets to be clustered;

[0088] Step 42: for any target to be clustered, obtain the number of distances between any target to be clustered and other targets that are less than a first preset value, which is recorded as the first number;

[0089] Step 43: for the current target to be clustered, obtain the target to be clustered B whose first number is greater than the first number of the current target to be clustered A, obtain the cumulative sum of the distances between the current target to be clustered and all the targets to be clustered B and record it as the second value;

[0090] Step 44: determining the order in which the objects to be clustered become cluster centers based on the product of the first number and the second value, wherein the larger the product of the first number and the second value, the higher the order in which the objects to be clustered become cluster centers;

[0091] Step 45: Based on multiple determined cluster centers, for the remaining targets C to be clustered, sort them from large to small based on the size of the first number of each target C to be clustered, and determine the cluster to which each target C to be clustered belongs in turn. For each target C to be clustered, divide it into the cluster where the cluster center that is closest to the target C to be clustered and whose first number is greater than that of the target C to be clustered is located.

[0092] In the embodiment of the present application, the first number represents the number of targets that are closer to the current target to be clustered. The larger the first number, the greater the possibility that the target to be clustered will become the cluster center. The second number represents the distance between the target to be clustered and other targets to be clustered that are more likely to become cluster centers. If the distance is larger, the greater the possibility that the target to be clustered will become another cluster center. In the present application, the initial cluster center is adaptively determined based on the distribution of the targets to be clustered, thereby avoiding the impact of inaccurate selection of the initial cluster center on the clustering results. In addition, by adaptively determining the initial cluster center, the clustering efficiency is improved.

[0093] In one embodiment, the above step 44 determines the order in which the objects to be clustered become cluster centers based on the product of the first number and the second number, and further includes:

[0094] Step 441, sorting the product values ​​based on the size of the product of the first number and the second number;

[0095] In step 442, the product value sorting sequence is analyzed for mutation points of the change trend. The target to be clustered corresponding to the product value before the mutation point is the cluster center, and the target to be clustered corresponding to the product value after the mutation point is the non-cluster center.

[0096] In the embodiment of the present application, based on the analysis of the trend mutation point of the product value sorting sequence, it is clear that the target to be clustered cannot be used as the cluster center. It can be understood that the larger the product value, the more preferentially the corresponding target to be clustered is used as the cluster center. The product value of the first number and the second number of the cluster center are similar, and the product value of the first number and the second number of the non-cluster center are similar. The trend mutation point of the product value sorting sequence analysis can be used as the dividing point between the two.

[0097] In one embodiment, the above step 442, based on the target to be clustered corresponding to the product value before the mutation point as the cluster center, further includes:

[0098] Step 4421: The target to be clustered corresponding to the product value before the mutation point is selected as a candidate cluster center;

[0099] Step 4422: For the current candidate cluster center C, obtain the first target D to be clustered whose number is greater than the first number of the current candidate cluster center C, obtain the minimum distance between the target to be clustered D and the current candidate cluster center C and record it as the first distance. If the first distance is less than the second preset value, the candidate cluster center C is determined to be a non-cluster center.

[0100] In the embodiment of the present application, it is further determined whether there is another candidate cluster center D in a relatively close range that is more likely to become another cluster center. If so, the candidate cluster center C is determined to be a non-cluster center.

[0101] It should be noted that the loss function used in the training process of the spore detection model is CIOU Loss, and the calculation formula of CIOU Loss is: in D gt =(x gt ,y gt , w gt , h gt ), D=(x,y,w,h),x gt ,y gt is the center coordinate of the real frame, w gt , h gt is the width and height of the real box, x, y are the center coordinates of the predicted box. w, h are the width and height of the predicted box. b, b gt is the center point of the predicted box and the center point of the real box, p is the Euclidean distance, and c is the diagonal distance of the minimum closure area that contains both the real box and the predicted box. m is the similarity ratio between the aspect ratio of the real box and the predicted box,

[0102] The training process of the spore detection model in step 4 above includes the following steps:

[0103] Step 401: Set training parameters: Use pre-trained weights from the COCO dataset for transfer learning. Use stochastic gradient descent (SGD) to optimize model weights during model training. Model hyperparameter settings include a batch size of 8, an initial learning rate of 0.01, a momentum of 0.937, and 100 epochs.

[0104] In step 402, the modified Yolov5s model is trained using the labeled training set and validation set, and after each training period, the performance of the model is evaluated using the test set to obtain the effect of each stage of training.

[0105] In step 5 above, the output of the spore detection model is in the format of (x1, y1, x2, y2, conf, cls), which respectively represent the center coordinates x and y of the output prediction box on the image, the width and height of the target box, the confidence level (confidence), and the class to which the target belongs. It should be noted that the multiple target boxes output by the spore detection model are redundantly eliminated using the SIOU-NMS algorithm. The SIOU-NMS calculation method is as follows:

[0106]

[0107] Where IOU is the intersection-over-union ratio of two boxes, Δ is the distance loss, and Ω is the shape loss.

[0108]

[0109] γ=2-Λ

[0110]

[0111] in are the center point coordinates of the true target box and the predicted target box.

[0112]

[0113] w gt ,w,h gt , h is the width and height of the real box and the predicted box.

[0114] The present application also provides a wheat stripe rust spore microscopic image detection system based on Yolov5s, comprising:

[0115] A spore detection model training unit is used to train a pre-built Yolov5s-based spore detection model based on a sample set. The training process includes: clustering the labeled bounding boxes of samples in the sample set based on size to obtain labeled bounding boxes of multiple category sizes, and determining the size of a preset anchor box of the Yolov5s model based on the sizes of the labeled bounding boxes of the multiple category sizes;

[0116] The spore detection unit is used to perform spore detection on the microscopic image to be detected based on the trained spore detection model to obtain a spore detection result. The spore detection model processes the input image including: (1) extracting image features in the Backbone module; (2) fusing the image features extracted in (1) in the Neck module; and (3) performing target detection on the fused image features in (2) in the Prediction module to obtain the position of stripe rust spores.

[0117] The wheat stripe rust fungus spore microscopic image detection system based on Yolov5s provided in this embodiment belongs to the same concept as the wheat stripe rust fungus spore microscopic image detection method embodiment based on Yolov5s provided in the above embodiment. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0118] To analyze the importance of different modules to the model, we conducted ablation experiments on a self-built spore dataset. Using the Yolov5s model as a benchmark, we compared the performance of the models after adding different modules. The experimental comparison results are shown in Table 1.

[0119] Using this application's clustering algorithm to re-cluster the anchor box sizes of stripe rust spores and replace the original anchor boxes generated by k-means in the network, the network achieved improvements across all metrics, with precision increasing by 1.22 percentage points and false negatives decreasing by 0.63 percentage points. This demonstrates that the predicted box sizes generated by this application's clustering algorithm are more consistent with the size of stripe rust spores, effectively enhancing the detection of some anchor box regression tasks and improving the network's ability to detect and locate spores.

[0120] This application adds a C3TR unit to the end of the feature extraction network in the original Yolov5s model. The improved network model's precision increased by 0.68 percentage points and reduced the false negative rate by 0.78 percentage points. This demonstrates that the C3TR structure can effectively enhance the network's ability to extract stripe rust spore features. By replacing the upsampling module in the feature fusion portion of the network with a lightweight upsampling module, network feature fusion was enhanced, precision increased by 0.92 percentage points, and the false negative rate decreased by 0.69 percentage points.

[0121] Table 1 Comparison of the effects of different modules on model performance

[0122]

[0123]

[0124] Under the same conditions, the Yolo-spore model was used to conduct comparative experiments with the Faster-RCNN, SSD, Yolov4, and Yolov5s models. The experimental comparison data are shown in Table 2.

[0125] Table 2 Comparison of detection results of different models on spore dataset

[0126]

[0127] Results show that the proposed algorithm improves both detection accuracy and false positive rate compared to other mainstream detection models, reaching an accuracy of 98.68%. Compared to Yolov5s, Yolov4, and SSD, detection accuracy increased by 1.42 percentage points, 5.53 percentage points, and 5.79 percentage points, respectively. The false positive rate increased by 1.04 percentage points, 5.39 percentage points, and 7.44 percentage points, respectively. Compared to the traditional two-stage Faster-RCNN algorithm, both detection accuracy and false positive rate were significantly improved. In summary, the proposed Yolo-spore algorithm offers advantages in both detection accuracy and false positive rate, while requiring only a slight increase in size compared to the original Yolov5s model.

[0128] In order to intuitively analyze the detection and counting effect of the model, three spore images with different spore numbers were randomly selected from the test set, and different methods were used to detect and count wheat stripe rust spores. The results are shown in Table 5.

[0129] Table 5 Count comparison of different methods

[0130] method Fig1 / (number) Fig2 / (number) Fig3 / (number) Manual counting 69 116 153 Yolov5s 73 113 144 Our 69 116 152

[0131] As can be seen from Table 5, for images with a small number of spores, the original Yolov5s network and the improved network in this paper have similar results. When the number of spores reaches a certain value, the original Yolov5s network has a large counting deviation, while the network in this paper can still count accurately.

[0132] like Figure 5 , Figure 6 As shown, the present invention can detect all spores when the number of spores is small. When the number of spores increases to more than 100, the increased density of spores, spore adhesion, and overlap increase the difficulty of model recognition. However, the Yolo-Spore model improves the anchor frame to better match the spore size, and improvements have been made to the feature extraction and feature fusion modules, allowing the network to significantly focus on the spore region despite background interference. Even when the spore density increases, spores can still be detected well. Therefore, the Yolo-Spore model detection method used in the present invention is superior to traditional methods, with a lower probability of false detection and missed detection.

[0133] In summary, to improve the accuracy and efficiency of wheat stripe rust spore detection, the present invention uses the K-Means++ clustering algorithm to re-cluster the anchor frame sizes of stripe rust spores, replacing the original anchor frames generated by k-means in the network. This enhances the anchor frame regression task in the detection portion and improves the network's ability to detect and locate spores. C3TR is added to the end of the feature extraction network in the original Yolov5s model to improve the network's feature extraction capabilities. Replacing the upsampling module in the feature fusion portion of the network with a lightweight upsampling module further enhances the network's feature fusion capabilities. Results show that the present invention can effectively extract spore features and detect and count spores with high accuracy and low false negative rates, with an accuracy of 98.68% and a false negative rate of 1.55%. Furthermore, compared to other methods, the present invention still achieves better detection results, demonstrating its superior practicality and robustness for densely packed small objects. Furthermore, compared to other models, the present method has a model size close to that of the original Yolov5s, but significantly smaller than that of other models.

[0134] The present invention is not limited to the above-mentioned specific implementation methods. Various changes made by ordinary technicians in this field based on the above-mentioned concept without creative work are all within the scope of protection of the present invention.

Claims

1. A method for detecting wheat stripe rust spores by microscopic imaging based on Yolov5s, characterized in that: The steps include: A pre-built Yolov5s-based spore detection model is trained based on a sample set. The training process includes: clustering the labeled bounding boxes of samples in the sample set based on size to obtain labeled bounding boxes of multiple category sizes, and determining the size of a preset anchor box of the Yolov5s model based on the sizes of the labeled bounding boxes of the multiple category sizes; Spore detection is performed on the microscopic image to be detected based on the trained spore detection model to obtain a spore detection result. The spore detection model processes the input image in the following manner: (1) Image feature extraction in the Backbone module; (2) The image features extracted in (1) are fused in the Neck module; (3) In the Prediction module, target detection is performed based on the image features fused in (2) to obtain the location of stripe rust spores; During the training process of the spore detection model, the labeled bounding boxes of the samples in the sample set are clustered based on their sizes, and the following steps are adopted: taking the labeled bounding boxes as the target to be clustered, obtaining the distance between any two targets to be clustered; for any target to be clustered, obtaining the number of targets whose distances between any target to be clustered and other targets are less than a first preset value, recorded as the first number; for the current target to be clustered, obtaining the target to be clustered B whose first number is greater than the first number of the current target to be clustered A, and obtaining the cumulative distances between the current target to be clustered and all targets to be clustered B The sum is recorded as the second value; the order in which the targets to be clustered become cluster centers is determined based on the size of the product of the first number and the second value, wherein the larger the product of the first number and the second value, the higher the order in which the targets become cluster centers; based on multiple determined cluster centers, for the remaining targets C to be clustered, sort them from large to small based on the size of the first number of each target C to be clustered, and determine the cluster to which each target C to be clustered belongs in turn; for each target C to be clustered, divide it into the cluster where the cluster center that is closest to the target C to be clustered and whose first number is greater than that of the target C to be clustered is located.

2. A wheat stripe rust spore microscopic image detection method based on Yolov5s according to claim 1, characterized in that, In (1), the step of extracting image features includes: (11) performing feature extraction in sequence using four sequentially connected feature extraction layers, and obtaining a first feature map output by the second feature extraction layer and a second feature map output by the third feature extraction layer, wherein the first feature extraction layer, the second feature extraction layer, and the third feature extraction layer include sequentially connected CBS units and CSP1 units, and the fourth feature extraction layer includes sequentially connected CBS units and SPP units; (12) The feature map output by the fourth feature extraction layer is processed by a C3TR unit to obtain a third feature map. The C3TR unit uses the first path to perform convolution processing and the second path to perform convolution processing and self-attention mechanism. The output of the first path and the output of the second path are spliced ​​and fused, and the spliced ​​and fused results are convolution processed.

3. A wheat stripe rust spore microscopic image detection method based on Yolov5s according to claim 2, characterized in that, In (12), the self-attention mechanism of the second path is implemented based on the transformer model unit. The self-attention mechanism unit includes a plurality of sequentially connected blocks. Each block includes a multi-head attention module and a feedforward neural network connected in sequence. The multi-head attention module and the feedforward neural network respectively adopt a residual connection structure. The image after the convolution processing in the second path is recorded as the first image. The image embedding and position embedding of the first image are input into the multi-head attention module to obtain the attention feature. The attention feature is input into the feedforward neural network to obtain the output of the self-attention mechanism.

4. A wheat stripe rust spore microscopic image detection method based on Yolov5s according to claim 1, characterized in that, Said (2) includes: The feature fusion is performed in sequence using four sequentially connected feature fusion layers, wherein the first feature fusion layer and the second feature fusion layer include sequentially connected CBS units, upsampling layers, concat fusion layers, and CSP2 units, and the third feature fusion layer and the fourth feature fusion layer include sequentially connected CBS units, concat fusion layers, and CSP2 units; In the feature fusion layer, the first feature map is input into the concat fusion layer of the second feature fusion layer, the second feature map is input into the concat fusion layer of the first feature fusion layer, and the third feature map is input into the input of the Neck module, i.e., the CBS unit of the first feature fusion layer; The output of the CBS unit of the first feature fusion layer is simultaneously input into the concat fusion layer of the fourth feature fusion layer, and the output of the CBS unit of the second feature fusion layer is simultaneously input into the concat fusion layer of the third feature fusion layer; The outputs of the CSP2 units of the second feature fusion layer, the third feature fusion layer, and the fourth feature fusion layer are respectively recorded as the first fused feature map, the second fused feature map, and the third fused feature map, which are input to the Prediction module.

5. A wheat stripe rust spore microscopic image detection method based on Yolov5s according to claim 4, characterized in that, The upsampling layer includes: (21) Use a 1*1 convolution kernel to reduce the dimension of the feature image. The dimension reduction formula is as follows: C m =s 2 (K up ) 2 Among them C m is the number of feature layer channels after dimensionality reduction, σ is the upsampling multiple, K up is the predicted upsampling kernel size; (22) Expand the channel dimension in the spatial dimension by reorganizing the width, height, and number of channels of the feature map into σW, σH, and K up 2 Get the predicted upsampling kernel; (23) The prediction results are normalized by softmax so that the sum of the weights of the convolution kernel is 1; (24) Multiply the features on each layer of feature map by the predicted upsampling kernel to obtain the upsampling result.

6. A wheat stripe rust spore microscopic image detection method based on Yolov5s according to claim 1, characterized in that, The determining of the order in which the objects to be clustered become cluster centers based on the product of the first number and the second value further includes: Sort the product values ​​based on the size of the product of the first number and the second number; The product value sorting sequence is analyzed for the mutation point of the change trend. The target to be clustered corresponding to the product value before the mutation point is the cluster center, and the target to be clustered corresponding to the product value after the mutation point is the non-cluster center.

7. A wheat stripe rust spore microscopic image detection method based on Yolov5s according to claim 6, characterized in that, The target to be clustered corresponding to the product value before the mutation point is the cluster center, and further includes: The target to be clustered corresponding to the product value before the mutation point is the candidate cluster center; For the current candidate cluster center C, obtain the first target D to be clustered whose number is greater than the first number of the current candidate cluster center C, obtain the minimum distance between the target to be clustered D and the current candidate cluster center C and record it as the first distance. If the first distance is less than the second preset value, the candidate cluster center C is determined as a non-cluster center.

8. A wheat stripe rust spore microscopic image detection method based on Yolov5s according to claim 1, characterized in that, The training process of the spore detection model further includes: performing image enhancement on samples in the sample set, wherein the image enhancement includes expanding through the styleGAN network and then rotating 180°, flipping, and rotating 180° before flipping and expanding.

9. A wheat stripe rust spore microscopic image detection system based on Yolov5s, characterized in that: include: A spore detection model training unit is used to train a pre-built Yolov5s-based spore detection model based on a sample set. The training process includes: clustering the labeled bounding boxes of samples in the sample set based on size to obtain labeled bounding boxes of multiple category sizes, and determining the size of a preset anchor box of the Yolov5s model based on the sizes of the labeled bounding boxes of the multiple category sizes; The spore detection unit is used to perform spore detection on the microscopic image to be detected based on the trained spore detection model to obtain a spore detection result. The spore detection model processes the input image including: (1) extracting image features in the Backbone module; (2) fusing the image features extracted in (1) in the Neck module; (3) performing target detection on the fused image features in (2) in the Prediction module to obtain the position of stripe rust spores; In the spore detection model training unit, during the training process of the spore detection model, the labeled bounding boxes of the samples in the sample set are clustered based on their sizes, and the following steps are adopted: taking the labeled bounding boxes as the target to be clustered, obtaining the distance between any two targets to be clustered; for any target to be clustered, obtaining the number of distances between any target to be clustered and other targets less than a first preset value, recorded as the first number; for the current target to be clustered, obtaining the target to be clustered B whose first number is greater than the first number of the current target to be clustered A, and obtaining the distances between the current target to be clustered and all targets to be clustered B. The cumulative sum of the distances between them is recorded as the second value; the order in which the targets to be clustered become cluster centers is determined based on the size of the product of the first number and the second value, wherein the larger the product of the first number and the second value, the higher the order in which the targets become cluster centers; based on multiple determined cluster centers, for the remaining targets C to be clustered, sort them from large to small based on the size of the first number of each target C to be clustered, and determine the cluster to which each target C to be clustered belongs in turn; for each target C to be clustered, divide it into the cluster where the cluster center that is closest to the target C to be clustered and whose first number is larger than that of the target C to be clustered is located.

Images