A method for counting corn tassel branches based on digital images and semi-supervised learning

Abstract

A corn tassel branch counting method based on digital images and semi-supervised learning belongs to the field of plant phenotype measurement. The method is: the construction of a corn tassel branch identification model YOLOv5-C3CA: a new network model YOLOv5-C3CA is constructed based on YOLOv5 for corn tassel branch identification; a corn tassel branch identification model based on YOLOv5-C3CA model and introduction of semi-supervised learning; corn tassel identification and branch number extraction: the YOLOv5-C3CA detection model based on semi-supervised learning is applied to identify the corn tassel; after identifying the tassel, the YOLOv5-C3CA training model based on semi-supervised learning is continuously applied to extract the number of corn tassel branches. The corn tassel identification and tassel branch identification of the application are relatively accurate, which shows that the corn tassel branch counting scheme based on semi-supervised learning is feasible, which not only ensures the detection performance of the model, but also saves the data labeling workload.

Description

Technical Field

[0001] This invention belongs to the field of plant phenotyping, and in particular relates to a method for counting maize tassel branches based on digital images and semi-supervised learning. Background Technology

[0002] To address global food security challenges, plant phenotyping analysis is needed to provide foundational data for crop breeding and precision agriculture. Traditionally, plant phenotyping data is primarily acquired manually, which is not only time-consuming and labor-intensive but also susceptible to subjective factors.

[0003] Digital cameras are a widely used sensor in plant phenotyping research. They are characterized by good imaging quality, low image acquisition cost, and low susceptibility to external environmental influences. Digital imaging can efficiently acquire plant phenotypic traits, providing fundamental data for the study of the structural characteristics of plant organs, plants, and plant populations.

[0004] With the rapid development of computer vision in recent years, deep learning has become one of the main methods for processing plant phenotypes. However, deep learning requires a large number of labeled samples to train the model, which is costly and time-consuming. To ensure the timeliness of crop research, semi-supervised learning methods are more suitable for real-world scenarios. They can train the model using only a small number of labeled samples and a large number of unlabeled samples, greatly saving the amount of labeling work while ensuring research accuracy.

[0005] The number of branches on a maize tassel is a crucial factor determining tassel size and pollen quantity, and is one of the important indicators in maize breeding. Excessive tassel branches not only consume nutrients but also produce excessive pollen, hindering plant growth. Real-time and accurate detection of tassel branch counts provides fundamental data for field activities such as detasseling arrangements and high-density planting of varieties, promoting the development of precision agriculture. Traditionally, maize tassel branch counting is done manually, which is time-consuming and costly, making large-scale measurement difficult. Therefore, establishing a method for automatically counting maize tassel branches based on semi-supervised learning is essential. Summary of the Invention

[0006] The purpose of this invention is to solve the problems of existing maize tassel branch counting methods being time-consuming, costly, and unable to achieve large-scale measurement, and to provide a maize tassel branch counting method based on digital images and semi-supervised learning, which aims to obtain the number of maize tassel branches more efficiently and accurately.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] A method for counting branches of maize tassels based on digital images and semi-supervised learning, the method comprising the following steps:

[0009] Step 1: Construction of the YOLOv5-C3CA model for identifying maize tassel branches: A new network model, YOLOv5-C3CA, is built based on YOLOv5 for identifying maize tassel branches;

[0010] Step 2: Based on the YOLOv5-C3CA model from Step 1, introduce a semi-supervised learning model for identifying maize tassel branches;

[0011] Step 3: Identification and Branch Number Extraction of Maize Tassels: The YOLOv5-C3CA detection model based on semi-supervised learning obtained in Step 2 is used to identify maize tassels; after identifying the tassels, the YOLOv5-C3CA training model based on semi-supervised learning is used to extract the number of branches of the maize tassels.

[0012] The advantages of this invention over the prior art are as follows: the identification of maize tassels and tassel branches in this invention are more accurate, indicating that the proposed maize tassel branch counting scheme based on semi-supervised learning is feasible. It not only ensures the detection performance of the model, but also saves the workload of data labeling. Attached Figure Description

[0013] Figure 1 The flowchart shows the process of identifying maize tassels and tassel branches based on YOLOv5-C3CA.

[0014] Figure 2 This is a diagram of the C3CA structure.

[0015] Figure 3 Here is a diagram of the Bifpn structure;

[0016] Figure 4 The flowchart shows the YOLOv5-C3CA object detection process based on semi-supervised learning.

[0017] Figure 5 Image showing the effect of corn tassel identification;

[0018] Figure 6 This is a diagram illustrating the effect of identifying branches in the tassel of a corn plant. Detailed Implementation

[0019] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments, but it is not limited thereto. Any modifications or equivalent substitutions to the technical solution of the present invention that do not depart from the spirit and scope of the technical solution of the present invention should be covered within the protection scope of the present invention.

[0020] Specific Implementation Method 1: This implementation method describes a method for counting branches of maize tassels based on digital images and semi-supervised learning. The method steps are as follows:

[0021] Step 1: Construction of the YOLOv5-C3CA model for identifying maize tassel branches: A new network model, YOLOv5-C3CA, is built based on YOLOv5 for identifying maize tassel branches; Figure 1 The flowchart for identifying male tassel branches in maize is as follows: First, the improved model YOLOv5-C3CA is applied to detect maize male tassels. Sub-images are then cropped based on the predicted bounding boxes. Finally, the YOLOv5-C3CA model is applied to detect male tassel branches in the sub-images.

[0022] Step 2: Based on the YOLOv5-C3CA model in Step 1, a semi-supervised learning model for identifying maize tassel branches is introduced. Since deep learning models heavily rely on a large amount of labeled data, and there are problems such as high data labeling costs and small sample sizes of image data, this invention introduces semi-supervised learning to create a new YOLOv5-C3CA model for identifying maize tassel branches, which effectively reduces data labeling work while ensuring model accuracy.

[0023] Step 3: Identification and Branch Number Extraction of Maize Tassels: The YOLOv5-C3CA detection model based on semi-supervised learning obtained in Step 2 is used to identify maize tassels; after identifying the tassels, the YOLOv5-C3CA training model based on semi-supervised learning is used to extract the number of branches of the maize tassels.

[0024] Specific Implementation Method Two: For the maize tassel branch counting method based on digital images and semi-supervised learning described in Specific Implementation Method One, please refer to... Figure 1 In step one, the YOLOv5-C3CA network comprises four parts: Input, Backbone, Neck, and Prediction. The Input is used for adaptive image scaling; the view image's width and height in pixels must be divisible by 32 (in this invention, the image is standardized to 640×640 pixels). The Backbone consists of multiple convolutional modules (Conv), multiple feature extraction modules (C3), a feature extraction module with coordinate attention (C3CA), and a multi-scale fusion module (SPPF), arranged in the following order: Conv, Conv, C3, Conv, C3, Conv, C3, Conv, C3CA, and SPPF. The second and third C3 modules and SPPF are input to the Neck. After inputting the image, features of the corn tassel and tassel branches are extracted based on the bounding box region. The use of the feature extraction module with coordinate attention (C3CA) enhances the model's ability to capture spatial location information of the target. Its structure is as follows: Figure 2As shown, this module consists of Conv and an embedded coordinate attention convolution module (ConvwithCoordinateAttention, CAConv) (C3CA, for example). Figure 2 (As shown); The Neck consists of multiple Conv modules, multiple upsampling modules (Upsample), multiple connection modules (Concat-Bifpn), and multiple C3 modules, arranged in the following order: Conv, Upsample, Concat-Bifpn, C3, Conv, Upsample, Concat-Bifpn, C3, Conv, Concat-Bifpn, C3, Conv, Concat-Bifpn, C3, Conv, Concat-Bifpn, C3. The first and fourth Conv modules are connected using Concat-Bifpn, as are the second and third Conv modules. This invention replaces the original Concat connection method with Bifpn, and its structure is as follows. Figure 3 As shown, this structure can repeatedly apply top-down and bottom-up multi-scale features for feature fusion, which can more easily and quickly fuse target features of different sizes obtained by the backbone network, thereby improving the model's feature fusion capability; after obtaining the three outputs of Neck, Prediction predicts the bounding box coordinates, category, and confidence of the target.

[0025] Specific Implementation Method 3: The corn tassel branch counting method based on digital images and semi-supervised learning described in Specific Implementation Method 2. The Conv module consists of two-dimensional convolution (Conv2d), batch normalization, and activation function (SiLU), increasing network depth and improving target feature extraction capability; the C3 module contains 3 Conv2d modules and multiple Bottleneck modules to perform residual processing on the input feature information, extracting finer-grained features; the SPPF module fuses local and global features to improve the representation capability of the input feature map and further improve the running speed; Upsample upsamples the input feature information to expand the feature map; Concat-Bifpn performs feature fusion on the input feature map to expand the dimension of the feature map; the three output features of Prediction are 80×80×18, 40×40×18, and 20×20×18, respectively.

[0026] Specific Implementation Method Four: A method for counting maize tassel branches based on digital images and semi-supervised learning, as described in Specific Implementation Method One, such as... Figure 4 As shown, the algorithm flow in step two is as follows:

[0027] (1) The initial teacher model (YOLOv5-C3CA) was trained using labeled data to predict unlabeled data;

[0028] (2) Predict the unlabeled data using the trained teacher model, and filter out the bounding boxes with a class confidence of less than 0.3 in the unlabeled data. Create pseudo-labels based on the bounding boxes predicted by the model. The file format is .xml.

[0029] (3) The student model is trained by fusing labeled data and pseudo-labeled data (YOLOv5-C3CA). In order to improve the generalization and robustness of the model, input noise and model noise are added during the training process. The input noise is data augmentation and the model noise is stochastic gradient descent optimizer (SGD).

[0030] (4) Use the student model in step (3) as the new teacher model, and return to step (2) to train the model repeatedly for 3 cycles.

[0031] Example 1:

[0032] The invention effect of using digital images acquired by UAVs. The digital images were collected by UAVs in Lishu County, Jilin Province. The acquisition device was a DJI Wukong 2 UAV equipped with a ZENMUSE X5S camera. The flight altitude of the UAV was set to 15m, and the resolution of the acquired images was 5472×3648. The effectiveness of the method of the invention was verified based on the 265 digital image data acquired. The process of counting branches of corn tassels is mainly divided into two steps: (1) First, the LabelImg software is used to annotate the tassels in the UAV images to obtain an XML format file. The obtained XML format file and the original image are directly used to generate a corn tassel dataset, which is input into the YOLOv5-C3CA model based on semi-supervised learning for training and testing. Finally, the corn tassel recognition is realized, and the corn tassel sub-images are cropped according to the bounding box predicted by the model. (2) Then, the LabelImg software is used to annotate the tassel branches in the tassel sub-images, which are processed into XML format files required for identifying tassel branches, and a corn tassel branch dataset is generated. The maize tassel branch dataset was input into the YOLOv5-C3CA model based on semi-supervised learning for training and validation, ultimately achieving the extraction of maize tassel branch information.

[0033] In the tassel detection phase, 100 images were labeled. 20 images were used to test the model's performance in detecting tassels, while the remaining 165 images were unlabeled and pseudo-labels were generated using a teacher model. For tassel branch detection, 1689 images were used, with 100 labeled. 20 of these were used to validate the model's performance in detecting tassel branches, while the remaining 1589 images were unlabeled. To validate the model's detection performance, the evaluation metrics were: Precision (P), Recall (R), and Mean Average Precision (mAP), as shown in the following formulas:

[0034]

[0035]

[0036]

[0037]

[0038] In the formula, TP represents true positives, which is the number of correctly detected targets; FP represents false positives, which is the number of incorrectly detected targets; FN represents false negatives, which is the number of missed targets; N represents the number of categories of corn tassels or corn tassel branches; AP is the integral of precision over recall; mAP is the average AP of all categories. The higher the mAP, the better the detection performance of the model.

[0039] Ultimately, the YOLOv5-C3CA model based on semi-supervised learning achieved P, R, and mAP values ​​of 94.6%, 80.8%, and 89.4%, respectively, in detecting maize tassels. The detection results for maize tassel branches are shown in Table 1. The mAP values ​​of the student models trained in three rounds were all above 80%, with Round-2 showing the best overall detection performance and a recall rate exceeding 80%. Figure 5 This is a demonstration of the effectiveness of corn tassel identification. Figure 6 The image shows the results of corn tassel branch recognition. As can be seen from the image, the corn tassel and tassel branch recognition are quite accurate, indicating that the proposed corn tassel branch counting scheme based on semi-supervised learning is feasible. It not only ensures the detection performance of the model, but also saves the workload of data annotation.

[0040] Table 1 Results of maize tassel branch detection

[0041] Loop count Accuracy Recall rate mAP Round-1 83.5％ 73.7％ 81.4％ Round-2 94.6％ 80.8％ 89.4％ Round-3 96.4％ 79.4％ 89.2％

Claims

1. A method for counting branches of maize tassels based on digital images and semi-supervised learning, characterized in that: The method steps are as follows: Step 1: Construction of the YOLOv5-C3CA model for corn tassel branch recognition: A new network model, YOLOv5-C3CA, is constructed based on YOLOv5 for corn tassel branch recognition. The YOLOv5-C3CA network consists of four parts: Input, Backbone, Neck, and Prediction. Input is used for adaptive image scaling; the view image's width and height in pixels must be divisible by 32. The Backbone consists of multiple convolutional modules (Conv), multiple feature extraction modules (C3), a feature extraction module (C3CA) with embedded coordinate attention, and a multi-scale fusion module (SPPF), arranged in the following order: Conv, Conv, C3, Conv, C3, Conv, C3, Conv, C3CA, and SPPF. The second and third C3 modules and SPPF are input to the Neck. After inputting the image, the features of the maize tassel and tassel branches are extracted based on the bounding box region. The Neck consists of multiple Conv modules, multiple upsampling modules (Upsample), multiple connection modules (Concat-Bifpn), and multiple C3 modules, in the following order: Conv, Upsample, Concat-Bifpn, C3, Conv, Upsample, Concat-Bifpn, C3, Conv, Concat-Bifpn, C3, Conv, Concat-Bifpn, C3. The first and fourth Conv modules are connected using Concat-Bifpn, and the second and third Conv modules are also connected using Concat-Bifpn. After obtaining the three outputs of the Neck, Prediction predicts the bounding box coordinates, category, and confidence of the target. Step Two: Based on the YOLOv5-C3CA model from Step One, a semi-supervised learning model for identifying maize tassel branches is introduced; the algorithm flow in Step Two is as follows: (1) The initial teacher model YOLOv5-C3CA was trained using labeled data to predict unlabeled data; (2) Predict the unlabeled data using the trained teacher model, and filter out the bounding boxes with a class confidence of less than 0.3 in the unlabeled data. Create pseudo-labels based on the bounding boxes predicted by the model. The file format is .xml. (3) The student model YOLOv5-C3CA was trained by fusing labeled data and pseudo-labeled data. In order to improve the generalization and robustness of the model, input noise and model noise were added during the training process. The input noise was data augmentation and the model noise was stochastic gradient descent optimizer. (4) Use the student model in step (3) as the new teacher model, and return to step (2) to repeatedly train the model, with the number of cycles being 3. Step 3: Identification of corn tassels and extraction of branch count: The semi-supervised learning-based corn tassel branch identification model obtained in Step 2 is used to identify corn tassels; after identifying the tassels, the semi-supervised learning-based corn tassel branch identification model is used again to extract the number of corn tassel branches.

2. The method for counting branches of maize tassels based on digital images and semi-supervised learning according to claim 1, characterized in that: The Conv module consists of 2D convolution Conv2d, Batch Normalization, and the SiLU activation function; the C3 module contains three Conv2d modules and multiple Bottleneck modules to perform residual processing on the input feature information; the SPPF module fuses local and global features to improve the representation capability of the input feature map; Upsample upsamples the input feature information to expand the feature map; Concat-Bifpn performs feature fusion on the input feature map to expand the dimension of the feature map; the three output features of Prediction are 80×80×18, 40×40×18, and 20×20×18, respectively.

Images