Flower pod counting method, system, equipment and medium

Abstract

The invention discloses a flower pod counting method, system, equipment and medium, and relates to the technical field of image processing, and the method comprises the steps: obtaining video data and image data of shot soybean flower pods, and marking soybean flowers and soybean pods in the video data and the image data; training a YOLOV8 model by using the video and image data marked with the soybean flowers and the soybean pods to obtain a soybean flower and pod detection model; and inputting to-be-detected soybean flower and pod video data into the tracking model, obtaining a soybean flower and pod detection result through the soybean flower and pod detection model, and tracking and counting the detected soybean flower and pods through a tracking algorithm OC-SORT. According to the invention, the number of soybean flowers and pods can be rapidly obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a flower pod counting method, system, equipment and medium. Background Art

[0002] As one of the most important food and oil crops in the world, soybean has important economic and nutritional value. In order to increase soybean yield, it is very important to obtain accurate soybean phenotypes. Due to the complexity of soybean, there are very few studies on soybean phenotypes. The phenotype of soybean flowers and pods is the basic phenotype that affects soybean yield, so it is urgent to study the quantitative phenotype of flowers and pods.

[0003] In the prior art, after taking an image of soybeans, target detection technology is used to obtain the number of soybean flowers and pods.

[0004] However, the above technology needs to spend a lot of time to capture images at a fixed angle and cannot quickly obtain the number of soybean flowers and pods. Summary of the invention

[0005] The embodiments of the present invention provide a method, system, device and medium for counting soybean flowers and pods, which can solve the problem in the prior art that the number of soybean flowers and pods cannot be quickly obtained.

[0006] An embodiment of the present invention provides a flower pod counting method, comprising the following steps: acquiring captured soybean flower pod video data and image data, and marking soybean flowers and soybean pods in the video data and the image data; training a YOLOV8 model using the video data and image data with the marked soybean flowers and soybean pods to obtain a soybean flower pod detection model; combining the soybean flower pod detection model with a tracking algorithm OC-SORT to construct a tracking model for tracking and detecting soybean flower pods; inputting the soybean flower pod video data to be tested into the tracking model, obtaining a soybean flower pod detection result through the soybean flower pod detection model, and tracking and counting the detected soybean flower pods through the tracking algorithm OC-SORT.

[0007] Furthermore, the captured soybean flower and pod video data and image data specifically include the following steps: shooting from the first node at the bottom of the soybean plant upwards in sequence; during the shooting process, all soybean flowers and soybean pods growing at each node are photographed.

[0008] Furthermore, the step of marking the soybean flowers and soybean pods comprises: using an image annotation tool LabelImg to mark the soybean flowers and soybean pods in the image; and using a video annotation tool DarkLabel to mark the soybean flowers and soybean pods in the video.

[0009] Furthermore, the video data and image data with labeled soybean flowers and soybean pods are used to train the YOLOV8 model, and the specific steps include: performing image enhancement on the image data with labeled soybean flowers and soybean pods by horizontally flipping and vertically flipping; and training the YOLOV8 model using the processed video and image data.

[0010] The embodiment of the present invention provides a flower pod counting system, comprising:

[0011] The data acquisition module is used to acquire the video data and image data of the captured soybean flower pods, and to mark the soybean flowers and soybean pods in the video data and image data; the model construction module is used to train the YOLOV8 model using the video data and image data with the marked soybean flowers and soybean pods to obtain the soybean flower pod detection model; the soybean flower pod detection model is combined with the tracking algorithm OC-SORT to construct a tracking model for tracking and detecting soybean flower pods; the detection and counting module is used to input the video data of the soybean flower pods to be tested into the tracking model, obtain the detection results of the soybean flower pods through the soybean flower pod detection model, and track and count the detected soybean flower pods through the tracking algorithm OC-SORT.

[0012] An embodiment of the present invention provides a computer device, comprising: a memory and a processor; the memory stores a computer program, and the processor implements the above-mentioned flower pod counting method when executing the computer program.

[0013] An embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned method for counting flower pods is implemented.

[0014] The embodiments of the present invention provide a flower pod counting method, system, device and medium. Compared with the prior art, the beneficial effects thereof are as follows:

[0015] The video data and image data of labeled soybean flowers and soybean pods are used to train the YOLOV8 model to obtain the soybean flower and pod detection model. The soybean flower pod video data to be tested is input into the tracking model, the soybean flower pod detection results are obtained through the soybean flower pod detection model, and the detected soybean flower pods are tracked and counted through the tracking algorithm OC-SORT.

[0016] Among them, video data of soybean pods can be quickly obtained through video shooting, and inputting the video data into a tracking model for tracking and detecting soybean pods can quickly detect the results of the soybean pods and obtain the number of soybean pods based on the results, avoiding the need to spend time shooting images from a fixed angle and achieving the purpose of fast counting through video data. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A schematic diagram of a process flow provided by an embodiment of the present invention;

[0018] Figure 2 The soybean flower and pod image provided by the embodiment of the present invention;

[0019] Figure 3 The changing process of soybean node position video flower and pod provided by the embodiment of the present invention;

[0020] Figure 4 A soybean image enhancement processing diagram provided by an embodiment of the present invention;

[0021] Figure 5 This is the accuracy result chart of the model 3 test set;

[0022] Figure 6 is the detection result of the flower pod image;

[0023] Figure 7 This is the tracking and counting results of the flower pod video;

[0024] Figure 8 This is a correlation test chart between the predicted soybean flower and pod counting results and the actual results. DETAILED DESCRIPTION

[0025] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention, so the present invention is not limited by the specific embodiments disclosed below.

[0026] See also Figure 1 The embodiment of the present invention provides a flower pod counting method, comprising the following steps:

[0027] Step 1: Obtain the captured video data and image data of soybean flowers and pods, and mark the soybean flowers and pods in the video data and image data.

[0028] Specifically, to collect videos of soybean flowers and pods, it is necessary to shoot from the first node at the bottom upwards in sequence. All soybean flowers and pods must be captured at each node. There is no limit to the shooting angle, and try not to record flower pods at other nodes of the plant or soybeans from other plants.

[0029] Step 2: Use the video data and image data with labeled soybean flowers and soybean pods to train the YOLOV8 model to obtain the soybean flower and pod detection model; combine the soybean flower and pod detection model with the tracking algorithm OC-SORT to construct a tracking model for tracking and detecting soybean flower pods.

[0030] Specifically, each data set is divided into 3 parts, and the test set is divided according to 0.1 of the total data set, and the remaining data is divided into training set and verification set according to the ratio of 9:1. Label Img is used to manually annotate the image data set, and the data set format is based on the VOC data set. Before using DarkLabel to annotate the video data set, the flower pod videos shot at other locations are cropped using the Microsoft Clipchamp video processing software that comes with Windows 11. The cropped video is cut off with the black edges using Format Factory, and then the annotation format is the MOT data set format. The data set is divided into 3 parts, and YOLOv8 and YOLOv7 are input for training. The optimizer is adjusted to obtain the optimal model YOLOv8 after training, and the optimizer is SGD.

[0031] YOLOv8 combined with StrongSORT, YOLOv8 combined with OC-SORT, and YOLOv8 combined with ByteTrack were used to track and count soybean flowers and soybean pods respectively. Among them, the YOLOv8 combined with OC-SORT model provided higher accuracy, tracking precision, and ID recognition accuracy in the detection and tracking of soybean flowers and pods.

[0032] Step 3: Input the soybean flower pod video data to be tested into the tracking model, obtain the soybean flower pod detection result through the soybean flower pod detection model, and track and count the detected soybean flower pods through the tracking algorithm OC-SORT.

[0033] The embodiment of the present invention provides a flower pod counting system, comprising:

[0034] The data acquisition module is used to acquire the video data and image data of the soybean flower pods, and mark the soybean flowers and soybean pods in the video data and image data. The model construction module is used to train the YOLOV8 model using the video data and image data with the soybean flowers and soybean pods marked to obtain the soybean flower pod detection model; the soybean flower pod detection model is combined with the tracking algorithm OC-SORT to construct a tracking model for tracking and detecting soybean flower pods. The detection and counting module is used to input the video data of the soybean flower pods to be tested into the tracking model, obtain the detection results of the soybean flower pods through the soybean flower pod detection model, and track and count the detected soybean flower pods through the tracking algorithm OC-SORT.

[0035] An embodiment of the present invention provides a computer device, comprising: a memory and a processor; the memory stores a computer program, and the processor implements the steps of a flower pod counting method when executing the computer program.

[0036] An embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of a flower pod counting method are implemented.

[0037] A specific embodiment is as follows:

[0038] 1. Material acquisition: The image data of soybean flowers were taken from experimental fields in multiple urban areas, and the varieties planted were Dongnong 252 (DN252), Heinong 51 (HN51), Chunfengzao and Zhe Nong 6; the image data of soybean pods were taken from the experimental fields of a certain university, and the experimental varieties were Dongnong 252 (DN252) and Heinong 51 (HN51). The video data of soybean flowers and pods were taken from the phenotyping park of a certain university from June 28, 2023 to October 9, 2023. There were 9 varieties planted (CX253, CX88, CX227, CX264, CX225, CX232, HN531, CX194, JL165), each with four plants, a total of 36 plants, because one plant of variety CX194 became ill and died, and finally 35 plants were left.

[0039] 2. Image and video acquisition: Due to the complex structure of soybeans, with many branches, leaves and main stems are blocked. The image shooting angle is a bird's-eye view, with the angle between the phone and the main stem being about 25 degrees and the distance from the node being about 10 to 15 cm. All images were taken with an iPhone 8 in natural conditions during the day. Figure 2 shown.

[0040] The video was collected under natural conditions during the day, using an iPhone14 to shoot videos of all soybean plant nodes, starting from the first node at the bottom and moving upwards, one node at a time. All soybean flowers and pods should be captured at each node. The shooting angle is not limited, and try not to record other node pods of the plant or soybean pods of other plants, such as Figure 3 shown.

[0041] 3. Data production: We established an image dataset and a video dataset of soybean flower pods, adjusted the images to 400×400 pixels, and used LabelImg to manually annotate the image dataset. The dataset format was based on the VOC dataset. Before annotating the video dataset with DarkLabel, we used the Microsoft Clipchamp video processing software that comes with Windows 11 to crop the flower pod videos shot at other locations. We used the format factory to cut off the black edges of the cropped videos, and then annotated them in the MOT dataset format. This experiment has three datasets, namely, flower dataset 1 with only flower annotations, pod dataset 2 with only pod annotations, and flower pod dataset 3 with both flower and pod annotations. The test set was divided into 0.1 of the total, and the remaining data was divided into the training set and the validation set in a ratio of 9:1.

[0042] 4. Preprocess the image: Use image enhancement methods to increase the diversity of input images by cropping, rotating, scaling, etc., so that the model has better robustness and generalization ability when facing images in different scenes. This paper uses two methods to enhance the image: horizontal flip and vertical flip. Figure 4 shown.

[0043] 5. Training of target detection model: This paper first uses 4 models to train the flower pod dataset (dataset 3). Model 1 is YOLOv7, its parameters are optimizer is SGD, initial learning rate is 0.01, momentum value is 0.937, Epoch is 200, Batchsize is 8, model 2 is YOLOv7, its parameters are optimizer is Adam, initial learning rate is 0.01, momentum value is 0.937, Epoch is 200, Batchsize is 8, model 3 is YOLOv8, its parameters are optimizer is SGD, initial learning rate is 0.01, momentum value is 0.937, Epoch is 200, Batchsize is 8, model 4 is YOLOv8, its parameters are optimizer is Adam, initial learning rate is 0.01, momentum value is 0.937, Epoch is 200, Batchsize is 8. The test results are shown in Table 1. The accuracy of the YOLOv8 model is generally higher than that of the YOLOv7 model, and its training time is also shorter than that of the YOLOv7 model. Figure 5 and Figure 6 Then, the soybean flower dataset (dataset 1) and soybean pod dataset (dataset 2) were trained separately. It was found that the detection results of model 3 were more accurate than those of other models whether they were trained separately or together. Therefore, this paper finally selected model 3 for flower and pod training.

[0044] Table 1 Test set accuracy and training time of the four models

[0045] Model F1 Precision Recall mAP Soybean flower mAP Soybean pod mAP Training time Model 1 0.75 0.782 0.7269 0.812 0.807 0.816 25.522 Model 2 0.82 0.821 0.813 0.868 0.864 0.872 25.418 Model 3 0.98 0.98 0.971 0.991 0.987 0.994 21.554 Model 4 0.89 0.903 0.884 0.945 0.941 0.95 21.505

[0046] 6. Evaluation of target tracking model: The trained model 3 is used as the detection part of flower and pod tracking and counting, and 70 soybean flower test videos and 85 soybean pod test videos are selected. YOLOv8 combined with StrongSORT, Yolov8 combined with OC-SORT, and YOLOv8 combined with ByteTrack are used to track and count soybean flower and soybean pod videos, respectively. The test results are shown in Table 2 and Table 3. Among them, the YOLOv8 combined with OC-SORT model provides higher accuracy, tracking precision, and ID recognition accuracy in the detection and tracking of soybean flowers and pods.

[0047] Table 2 Accuracy of YOLOv8 combined with three tracking algorithms on flower test videos (%)

[0048]

[0049] Table 3 Accuracy of YOLOv8 combined with three tracking algorithms on flower test videos (%)

[0050]

[0051] 7. Recognition and counting of soybean pods: YOLOv8 combined with OC-SORT model is used to infer and count soybean pods test videos. The results are as follows: Figure 7 As shown in the figure, which consists of some frames, it shows the tracking effect and counting results of flower and pod videos. 70 flower videos and 85 pod videos were selected as the test video data for counting. YOLOv8 combined with OC-SORT algorithm was used to count the number of soybean flower and pods and compared with the true value. Figure 8 It can be seen that the correlation between soybean flower count and the true value reaches 0.83, the MSE value is 0.43, the MAE value is 0.37, and the RMSE value is 0.65. The correlation coefficient between soybean pods and the true value is 0.81, the MSE value is 0.33, the MAE value is 0.33, and the RMSE value is 0.57. This shows that the algorithm combining YOLOv8 and OC-SORT also has a good counting ability for soybean pods in different scenarios.

[0052] 8. Comparison between target detection and counting methods and target tracking and counting methods: The same test video is decomposed into 30 frames per second, and the program is run to automatically identify and count the flower pods in each frame. Whether it is flowers or pods, the target tracking and counting method performs better than the target detection and counting method and has a higher correlation.

[0053] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A method for counting flower pods, characterized in that: The following steps are involved: Acquire the captured video data and image data of soybean flowers and pods, and mark the soybean flowers and soybean pods in the video data and image data; The video data and image data with labeled soybean flowers and soybean pods are used to train the YOLOV8 model to obtain the soybean flower and pod detection model; The soybean pod detection model is combined with the tracking algorithm OC-SORT to construct a tracking model for tracking and detecting soybean pods. The video data of soybean pods to be tested is input into the tracking model, the detection results of soybean pods are obtained through the soybean pod detection model, and the detected soybean pods are tracked and counted through the tracking algorithm OC-SORT.

2. A method for counting flower pods as claimed in claim 1, characterized in that: The specific steps of obtaining the captured soybean flower and pod video data and image data include: Take photos from the first node at the bottom of the soybean plant upwards; During the shooting process, all soybean flowers and soybean pods growing at each node were photographed.

3. A method for counting flower pods as claimed in claim 1, characterized in that: The steps of marking soybean flowers and soybean pods include: Use the image annotation tool Label Img to annotate soybean flowers and soybean pods in the image; Use the video annotation tool DarkLabel to annotate soybean flowers and soybean pods in the video.

4. A method for counting flower pods as claimed in claim 1, characterized in that: The YOLOV8 model is trained using video data and image data with soybean flowers and soybean pods marked, and the specific steps include: The image data of labeled soybean flowers and soybean pods are enhanced by horizontal and vertical flipping; Use the processed video and image data to train the YOLOV8 model.

5. A flower pod counting system, characterized in that: include: A data acquisition module, used to acquire the video data and image data of the soybean flowers and pods, and mark the soybean flowers and pods in the video data and image data; A model building module is used to train the YOLOV8 model using video data and image data with labeled soybean flowers and soybean pods to obtain a soybean flower and pod detection model; the soybean flower and pod detection model is combined with the tracking algorithm OC-SORT to build a tracking model for tracking and detecting soybean flower and pods; The detection and counting module is used to input the video data of the soybean pods to be tested into the tracking model, obtain the detection results of the soybean pods through the soybean pod detection model, and track and count the detected soybean pods through the tracking algorithm OC-SORT.

6. A computer device comprising: Memory and processor; The memory stores a computer program, wherein the processor implements a flower pod counting method according to any one of claims 1 to 4 when executing the computer program.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, a flower pod counting method according to any one of claims 1 to 4 is implemented.

Images