Pollen Activity Recognition Model Training Method, System, Recognition Method, and System

Abstract

The present invention relates to a method and system for training a pollen activity recognition model. First, a training data set is obtained, and the training data set includes multiple pollen staining pictures for training. Then, a recognition model to be trained is established, and the recognition model to be trained is an object detection model. Finally, the recognition model to be trained is trained using the training data set to obtain a recognition model. The present invention also provides a method and system for recognizing pollen activity. An unstained pollen picture to be recognized is obtained, and then, using the unstained pollen picture to be recognized as an input, the recognition model obtained by the above training method is used to recognize the unstained pollen picture to be recognized, so as to obtain pollen activity data. Furthermore, the trained recognition model can be used to automatically recognize the unstained pollen picture to be recognized, improving the recognition speed and efficiency and avoiding the time-consuming and laborious problems caused by manual counting.

Description

Technical Field

[0001] The present invention relates to the technical field of pollen activity recognition, and particularly to a method and system for training a pollen activity recognition model, as well as a recognition method and system. Background Art

[0002] Pollen is the male gametophyte of seed plants and can transmit the genetic information of male parents during the sexual reproduction process of higher plants. Its viability varies greatly depending on plant species and the environment. Pollen activity is a prerequisite for pollen to have the ability to grow, germinate, and develop. It is often involved in aspects such as agricultural production, crop hybridization breeding, crop fruiting mechanism, and pollen physiology. Moreover, pollen development and pollen tube germination are important systems for studying plant cell polar growth and signal transduction. Therefore, establishing a rapid pollen activity identification method is the basis for the selection of male sterile plants and the improvement of hybridization techniques in cross-breeding. Furthermore, among environmental factors, temperature and humidity have a great impact on pollen activity. Against the background of global warming, through a rapid pollen activity identification method, it is helpful to deeply explore pollen development genes involved in environmental stress responses at the plant population level, and to discuss whether these genes are involved in changes in pollen development physiological states, laying a foundation for further comprehensively analyzing the functions of target genes by molecular biology means and preparing conditions for rapid variety improvement by genetic engineering means. Thus, it can be seen that the activity state of pollen is closely related to genetic breeding work.

[0003] Pollen viability staining solution (TTC method) is a commonly used method for determining pollen activity at present. The basic principle of the TTC staining method is that pollen with activity has a stronger respiratory function. NADH or FADH2 produced by the respiratory function of pollen can reduce colorless TTC (2,3,5-triphenyltetrazolium chloride) to red TTF (triphenylformazan). Therefore, pollen with activity will show different degrees of red after treatment with the TTC method, while pollen without activity cannot carry out respiratory function and cannot make TTC show color. By observing the color change of the stained pollen, the activity of the pollen can be known.

[0004] In the actual scientific research process, in order to understand the traits of a population, usually a relatively large number of pollens are stained at one time, and after staining, manual counting is carried out to obtain the activity of this batch of pollens. However, the method of manual counting is time-consuming and laborious. Therefore, there is an urgent need for a method and system that can automatically identify pollen activity. Summary of the Invention

[0005] The object of the present invention is to provide a method and system for training a pollen activity recognition model, as well as a recognition method and system, so as to realize the automatic recognition of pollen activity and improve the recognition efficiency.

[0006] To achieve the above object, the present invention provides the following solutions:

[0007] In a first aspect, the present invention is used to provide a method for training a pollen activity recognition model, and the training method includes:

[0008] Obtain a training data set; the training data set includes multiple pollen staining pictures for training;

[0009] Establish a recognition model to be trained; the recognition model to be trained is an object detection model;

[0010] Use the training data set to train the recognition model to be trained to obtain a recognition model.

[0011] The present invention is also used to provide a pollen activity recognition model training system, and the training system includes:

[0012] A first acquisition module for obtaining a training data set; the training data set includes multiple pollen staining pictures for training;

[0013] A construction module for establishing a recognition model to be trained; the recognition model to be trained is an object detection model;

[0014] A training module for using the training data set to train the recognition model to be trained to obtain a recognition model.

[0015] In a second aspect, the present invention is used to provide a pollen activity recognition method, and the recognition method includes:

[0016] Obtain a pollen staining picture to be recognized; the pollen staining picture to be recognized includes multiple stained pollens;

[0017] Use the pollen staining picture to be recognized as an input, and use the recognition model to recognize the pollen staining picture to be recognized to obtain pollen activity data; the pollen activity data includes the number of pollens with activity, the number of pollens without activity, and the total number of pollens.

[0018] The present invention is also used to provide a pollen activity recognition system, and the recognition system includes:

[0019] A second acquisition module for obtaining a pollen staining picture to be recognized; the pollen staining picture to be recognized includes multiple stained pollens;

[0020] A recognition module for using the pollen staining picture to be recognized as an input and using the recognition model to recognize the pollen staining picture to be recognized to obtain pollen activity data; the pollen activity data includes the number of pollens with activity, the number of pollens without activity, and the total number of pollens.

[0021] According to the specific embodiments provided by the present invention, the following technical effects are disclosed by the present invention:

[0022] The present invention is used to provide a method and system for training a pollen activity recognition model. First, a training data set is obtained, and the training data set includes multiple pollen staining pictures for training. Then, a recognition model to be trained is established, and the recognition model to be trained is a target detection model. Finally, the recognition model to be trained is trained using the training data set to obtain a recognition model. The present invention also provides a method and system for recognizing pollen activity. An pollen staining picture to be recognized is obtained, and then, using the pollen staining picture to be recognized as an input, the recognition model obtained by the above training method is used to recognize the pollen staining picture to be recognized, and pollen activity data is obtained. Furthermore, the trained recognition model can be used to automatically recognize the pollen staining picture to be recognized, improving the recognition speed and efficiency, and avoiding the time-consuming and laborious problem caused by manual counting. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required to be used in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0024] Figure 1 It is a flowchart of the training method provided in Embodiment 1 of the present invention;

[0025] Figure 2 It is a flowchart of the method for obtaining a training data set provided in Embodiment 1 of the present invention;

[0026] Figure 3 It is a schematic diagram of the pollen staining picture provided in Embodiment 1 of the present invention;

[0027] Figure 4 It is a schematic diagram of the pollen staining picture after annotation provided in Embodiment 1 of the present invention;

[0028] Figure 5 It is a system block diagram of the training system provided in Embodiment 2 of the present invention;

[0029] Figure 6 It is a flowchart of the recognition method provided in Embodiment 3 of the present invention;

[0030] Figure 7 It is a schematic diagram of the pollen staining picture recognized by the recognition model provided in Embodiment 3 of the present invention;

[0031] Figure 8 It is a system block diagram of the recognition system provided in Embodiment 4 of the present invention;

[0032] Figure 9 It is a schematic diagram of the human-computer interaction interface provided in Embodiment 4 of the present invention. Detailed implementation manners

[0033] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0034] The purpose of the present invention is to provide a pollen activity recognition model training method, system, recognition method, and system to achieve automatic recognition of pollen activity and improve recognition efficiency.

[0035] To make the above objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific implementation manners.

[0036] Embodiment 1:

[0037] As Figure 1 shown, this embodiment is used to provide a pollen activity recognition model training method, and the training method includes:

[0038] S1: Obtain a training data set; the training data set includes multiple training pollen staining pictures;

[0039] As Figure 2 shown, S1 may include:

[0040] S11: Stain multiple pollens using the TTC staining method to obtain stained pollens;

[0041] Use the TTC staining method to stain plant pollens for pollen pretreatment to obtain stained pollens.

[0042] S12: Take pictures of the stained pollens to obtain multiple training pollen staining pictures; each training pollen staining picture includes multiple stained pollens;

[0043] Specifically, take pictures of the stained pollens after being stained by the TTC staining method under a microscope to obtain a captured image. At this time, a captured image can be directly used as a training pollen staining picture, and then multiple training pollen staining pictures can be obtained through multiple shootings. As Figure 3 shown, it gives a schematic diagram of a pollen staining picture. It is also possible to split each captured image to obtain multiple training pollen staining pictures. Furthermore, by cropping and splitting the captured image, the size of the training pollen staining pictures can be reduced, and the training speed of the model can be improved.

[0044] Generally, the captured images obtained by photographing under a microscope are in high-definition tif format. In this embodiment, the code can be used to convert them into jpg format picture files that occupy less memory to reduce the memory occupation.

[0045] S13: Annotate the training pollen-stained pictures to obtain a pollen annotation file corresponding to each training pollen-stained picture;

[0046] The purpose of this embodiment is to automatically distinguish whether plant pollen is active through a model. Through preliminary observation, for the indicators in dimensions such as roundness, area, and perimeter, there are no differences between active pollen and inactive pollen. Therefore, the above indicators cannot be used as indicators to distinguish whether pollen is active. However, through observing the pollen-stained pictures, it is known that the color differences between active pollen and inactive pollen in the RGB channel are very obvious. Active pollen presents colors such as dark red and reddish-brown, while inactive pollen presents colors such as light yellow and gray. Therefore, this embodiment can distinguish the active state of pollen in the pollen-stained pictures by color.

[0047] Annotating the training pollen-stained pictures can include: using annotation frames to respectively frame each stained pollen in the training pollen-stained pictures and annotating labels to obtain a pollen annotation file corresponding to the training pollen-stained picture. The annotation frame is the minimum bounding box of the stained pollen, and the labels include being active and not being active.

[0048] More specifically, the Lambelimg image annotation software can be used to annotate each training pollen-stained picture so that the pixel edges of each visible stained pollen are within its corresponding annotation frame. In this embodiment, the minimum bounding box of each stained pollen is used as the correct annotation frame corresponding to the stained pollen, and the labels are respectively marked as "live" (being active) and "die" (not being active) to distinguish active and inactive pollen. At this time, the annotated pollen-stained pictures obtained are as Figure 4 shown, and then a pollen annotation file is obtained.

[0049] It should be noted that the pollen annotation file obtained by annotating with the Lambelimg image annotation software is in yaml format. At this time, the yaml format pollen annotation file needs to be converted into a format that can be recognized by the recognition model to be trained.

[0050] S14: All the training pollen-stained pictures and the pollen annotation file corresponding to each training pollen-stained picture form a training data set.

[0051] S2: Establish a recognition model to be trained; the recognition model to be trained is an object detection model;

[0052] The recognition model to be trained used in this embodiment can be any object detection model with object detection function. Specifically, the recognition model to be trained can be a YOLO series deep learning model, a Faster-Rcnn model based on the pytorch or tensorflow framework, or an SSD deep learning model based on the caffe framework.

[0053] S3: Use the training dataset to train the recognition model to be trained to obtain a recognition model.

[0054] In this embodiment, a hybrid training method combining pre-training and re-training is used to train the recognition model to be trained. Specifically, the coco dataset is used to pre-train the recognition model to be trained to obtain a pre-trained model, and the training dataset is used to re-train the pre-trained model to obtain a recognition model.

[0055] As an optional implementation manner, in this embodiment, the training dataset can be divided. The labeled pollen stained training pictures are randomly divided into a training set, a test set, and a validation set according to a ratio of 8:1:1, and the pollen stained training pictures and their corresponding labels (i.e., pollen annotation files) are stored in the corresponding folders according to the above division method, waiting for the recognition model to be trained to call. The training set is used to train the recognition model to be trained, the validation set is used to validate the recognition model obtained after training, and the test set is used to test the recognition model obtained after training to test its recognition effect.

[0056] Next, taking the recognition model to be trained as the YOLOv5 model as an example, the above training method will be further described:

[0057] The YOLOv5 deep learning network includes four parts, namely the input end, the backbone end, the head end, and the output end. The pollen stained training pictures are input into the network through the input end and enter the backbone end. The backbone end is composed of a total of nine layers of structures including Focus, CSP, and SSP, mainly for feature extraction of the pollen stained training pictures; the 10th layer network to the 18th layer network constitute the head end, mainly for upsampling and downsampling the feature map, and at the same time performing feature fusion; the output end is composed of the 19th and 20th layers, and the final standard box is determined through the non-maximum suppression algorithm.

[0058] When training the YOLOv5 model, the method used is as follows: Use S1 to obtain the training dataset, and batch convert the coordinate information in the pollen annotation file in yaml format corresponding to each training pollen-stained image in the training dataset into a txt format file that can be recognized by the YOLOv5 model. Randomly divide the labeled training pollen-stained images into a training set, a test set, and a validation set according to the ratio of 8:1:1. The YOLOv5 model is first pre-trained on the coco dataset to obtain the YOLOv5m model, and then the YOLOv5m model is used to train on the training set to obtain the recognition model.

[0059] After the training is completed, the obtained recognition model is tested on the test set. The model recognition accuracy reaches 90.4%, the mean average precision (map) is 0.988, and the recall is 1. This proves that the recognition model trained by the training method of this embodiment can accurately identify the pollen viability.

[0060] Obtaining pollen viability through manual counting is time-consuming and laborious, and due to the irregular distribution of pollen, it is difficult to guarantee the accuracy during counting. The training method provided in this embodiment can train a recognition model, and using this recognition model can replace manual counting of the stained pollen. While ensuring a certain accuracy, it can greatly save the recognition time and improve the recognition efficiency.

[0061] Embodiment 2:

[0062] This embodiment is used to provide a pollen viability recognition model training system, as Figure 5 shown, the training system includes:

[0063] The first acquisition module M1 is used to acquire the training dataset; the training dataset includes multiple training pollen-stained images;

[0064] The construction module M2 is used to establish the recognition model to be trained; the recognition model to be trained is an object detection model;

[0065] The training module M3 is used to train the recognition model to be trained using the training dataset to obtain the recognition model.

[0066] Embodiment 3:

[0067] This embodiment is used to provide a pollen viability recognition method, as Figure 6 shown, the recognition method includes:

[0068] T1: Obtain the pollen-stained image to be recognized; the pollen-stained image to be recognized includes multiple stained pollens;

[0069] Specifically, first, use the TTC staining method to stain multiple pollens for pollen pretreatment to obtain stained pollens, and then take pictures of all the stained pollens under a microscope to obtain stained pollen pictures to be recognized. The schematic diagram of the stained pollen pictures to be recognized is referred to Figure 3 。

[0070] T2: Use the stained pollen pictures to be recognized as input, and use the recognition model to recognize the stained pollen pictures to be recognized to obtain pollen activity data; the pollen activity data includes the number of pollens with activity, the number of pollens without activity, and the total number of pollens.

[0071] As Figure 7 shown, it gives the schematic diagram of the stained pollen pictures obtained after the recognition model recognizes the stained pollen pictures to be recognized. At the same time, the recognition model will automatically give pollen activity data based on Figure 7 the described stained pollen pictures.

[0072] This embodiment proposes a pollen activity recognition method based on the pytorch framework, which uses the recognition model trained in Embodiment 1 to automatically recognize pollen activity with high recognition accuracy, providing good technical support for judging pollen vitality and genetic breeding. This recognition method ensures an extremely fast recognition speed while taking into account the detection accuracy, and the detection speed can reach 270 FPS.

[0073] Embodiment 4:

[0074] This embodiment is used to provide a pollen activity recognition system. As Figure 8 shown, the recognition system includes:

[0075] The second acquisition module M4 is used to acquire stained pollen pictures to be recognized; the stained pollen pictures to be recognized include multiple stained pollens;

[0076] The recognition module M5 is used to use the stained pollen pictures to be recognized as input, and use the recognition model to recognize the stained pollen pictures to be recognized to obtain pollen activity data; the pollen activity data includes the number of pollens with activity, the number of pollens without activity, and the total number of pollens.

[0077] As Figure 9 shown, the recognition system of this embodiment further includes a human-computer interaction interface. Specifically, use the pyqt5 plug-in to make a human-computer interaction interface (GUI interface), and this human-computer interaction interface can realize functions such as selecting a detection folder, an output information text box, starting detection, and exiting the program.

[0078] More specifically, connect the trained recognition model to the interface of the GUI, so that multiple buttons on the human-computer interaction interface have corresponding functions. The user clicks on the button for selecting the folder to be detected to select the detection folder, which contains the pollen staining pictures to be recognized. After the selection, the user clicks on the start detection button to start the recognition. At this time, the human-computer interaction interface provides the pollen staining pictures to be recognized to the second acquisition module according to the user's selection, and the second acquisition module and the recognition module cooperate to obtain the pollen activity data. The human-computer interaction interface also interacts with the recognition module to obtain and display the pollen activity data, specifically on the detection result area of the human-computer interaction interface. The user can exit the human-computer interaction interface by clicking on the exit program button.

[0079] The detection system provided in this embodiment creates a GUI for the deep learning model. The user starts the detection by clicking on this human-computer interaction interface, sends the selected detection folder into the recognition model, obtains the output result of the pollen activity data and displays it in the text box at the detection result area, which is convenient for users without a code foundation to use this system for pollen activity detection.

[0080] In each embodiment of this specification, the key points are the differences from other embodiments. The same or similar parts among the embodiments can be referred to each other. For the system disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the description in the method section.

[0081] In this article, specific examples are used to elaborate on the principles and implementation methods of the present invention. The descriptions of the above embodiments are only used to help understand the method of the present invention and its core idea; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A pollen activity recognition method, characterized in that, The recognition method includes: Obtain a pollen stained picture to be recognized; the pollen stained picture to be recognized includes multiple stained pollens; use the recognition model to recognize the pollen stained picture to be recognized with the pollen stained picture to be recognized as the input, and obtain pollen activity data; the pollen activity data includes the number of active pollens, the number of inactive pollens, and the total number of pollens. The training method of the recognition model includes: obtaining a training data set; The training data set includes multiple training pollen stained pictures; Establish a recognition model to be trained; the recognition model to be trained is an object detection model; Use the training data set to train the recognition model to be trained to obtain a recognition model; The YOLOv5 deep learning network includes four parts, namely the input end, the backbone end, the head end and the output end. The training pollen stained picture enters the backbone end through the input end of the network. The backbone end is composed of nine layers of structures including Focus, CSP, and SSP, and extracts features from the training pollen stained picture; The 10th layer to the 18th layer of the network constitute the head end, which mainly performs upsampling and downsampling on the feature map and fuses features at the same time; The output end is composed of the 19th and 20th layers, and determines the final standard box through the non-maximum suppression algorithm. When training the YOLOv5 model, the method used is: convert the coordinate information in the pollen annotation file in yaml format corresponding to each training pollen stained picture in the training data set into a txt format file that can be recognized by the YOLOv5 model through code, and randomly divide the labeled training pollen stained pictures into a training set, a test set and a validation set according to the ratio of 8:1:

1. The YOLOv5 model is first pre-trained on the coco data set to obtain the YOLOv5m model, and then the YOLOv5m model is used to train on the training set to obtain the recognition model. The specific process of obtaining the training data set includes: Stain multiple pollens using the TTC staining method to obtain stained pollens; Take pictures of the stained pollens to obtain multiple training pollen stained pictures; Each of the training pollen stained pictures includes multiple of the stained pollens; Annotate the training pollen stained pictures to obtain a pollen annotation file corresponding to each training pollen stained picture; All the training pollen stained pictures and the pollen annotation file corresponding to each training pollen stained picture form a training data set; The specific process of annotating the training pollen stained pictures includes: Use annotation boxes to frame each of the stained pollens in the training pollen stained pictures respectively and label them to obtain the pollen annotation file corresponding to the training pollen stained picture; the annotation box is the minimum bounding box of the stained pollen; the labels include active and inactive.

2. The recognition method according to claim 1, wherein The specific process of taking pictures of the stained pollens to obtain multiple training pollen stained pictures includes: Take pictures of the stained pollens under a microscope to obtain a captured image; Split each of the captured images to obtain multiple pollen staining pictures for training.

3. The recognition method according to claim 1, characterized in that, The recognition model to be trained is a YOLO series deep learning model, a Faster-Rcnn model, or an SSD deep learning model.

4. The recognition method according to claim 1, characterized in that Training the recognition model to be trained using the training dataset to obtain the recognition model specifically includes: Pre-training the recognition model to be trained using the coco dataset to obtain a pre-trained model; Re-training the pre-trained model using the training dataset to obtain the recognition model.

5. A pollen activity recognition system, characterized in that, The recognition system includes: A second acquisition module for acquiring pollen staining pictures to be recognized; the pollen staining pictures to be recognized include multiple stained pollens; A recognition module for using the pollen staining pictures to be recognized as input and using the recognition model to recognize the pollen staining pictures to be recognized to obtain pollen activity data; the pollen activity data includes the number of active pollens, the number of inactive pollens, and the total number of pollens; The training system of the recognition model includes: A first acquisition module for acquiring a training dataset; The training dataset includes multiple pollen staining pictures for training; A construction module for establishing a recognition model to be trained; the recognition model to be trained is an object detection model; A training module for training the recognition model to be trained using the training dataset to obtain the recognition model; Training the recognition model to be trained using the training dataset to obtain the recognition model; The YOLOv5 deep learning network consists of four parts, namely the input end, the backbone end, the head end, and the output end. The pollen staining pictures for training enter the network through the input end and enter the backbone end. The backbone end consists of a total of nine layers of structures, namely Focus, CSP, and SSP, to extract features from the pollen staining pictures for training; The 10th to 18th layers of the network form the head end, which mainly performs upsampling and downsampling on the feature map and performs feature fusion at the same time; The output end consists of the 19th and 20th layers, and the final standard box is determined through the non-maximum suppression algorithm; When training the YOLOv5 model, the method used is: Convert the coordinate information in the pollen annotation file in yaml format corresponding to each pollen staining picture for training in the training dataset into a txt format file that can be recognized by the YOLOv5 model through code, and randomly divide the pollen staining pictures for training with labels into a training set, a test set, and a validation set according to a ratio of 8:1:

1. The YOLOv5 model is first pre-trained on the coco dataset to obtain the YOLOv5m model, and then the YOLOv5m model is used to train on the training set to obtain the recognition model; The specific process of obtaining the training dataset includes: Stain multiple pollens using the TTC staining method to obtain stained pollens; Take pictures of the stained pollens to obtain multiple pollen staining pictures for training; Each pollen staining picture for training includes multiple stained pollens; Label the pollen-stained pictures for training to obtain a pollen annotation file corresponding to each pollen-stained picture for training; All the pollen-stained pictures for training and the pollen annotation file corresponding to each pollen-stained picture for training constitute a training data set; The labeling of the pollen-stained pictures for training specifically includes: Use a bounding box to frame each stained pollen in the pollen-stained picture for training respectively, and label the tags to obtain a pollen annotation file corresponding to the pollen-stained picture for training; the bounding box is the minimum bounding rectangle of the stained pollen; the tags include having activity and not having activity.

6. The recognition system according to claim 5, wherein The recognition system further includes a human-computer interaction interface; the human-computer interaction interface is used to provide the pollen-stained picture to be recognized to the second acquisition module according to the user's selection, and interact with the recognition module to acquire and display the pollen activity data.

Images