A method and system for automatically counting and quantifying leaf trichomes of non-heading Chinese cabbage

Abstract

The application discloses a kind of cabbage leaf surface trichome automatic counting and quantification method and system, the method includes the following steps: obtaining the three-dimensional topographic image of cabbage leaf surface as the image to be identified;The image to be identified is input into neural network detection counting model to identify and count the trichome of cabbage leaf surface, obtain the trichome quantity of cabbage leaf surface;The neural network detection counting model is obtained by training LeafMaskR-CNN model;Based on the trichome quantity, calculate the trichome density of cabbage leaf surface;The trichome density is used to quantify the leaf surface trichome character of cabbage.The application is based on the three-dimensional topographic image of cabbage leaf surface, using trained LeafMask R-CNN model, realizes the automatic acquisition counting and quantitative evaluation of cabbage leaf surface trichome, improves the efficiency and accuracy of quantifying trichome density.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of plant phenomics, in particular to a method and system for automatic counting and quantification of leaf trichomes of Brassica campestris. BACKGROUND

[0002] Brassica campestris ssp. chinensis Makino belongs to a subspecies of Brassica campestris ssp. chinensis Makino of Brassica campestris L. of Brassicaceae, is originally from southern China, and is one of the main leafy vegetables in China. Leaf is an essential organ for the growth and development of Brassica campestris, and widely participates in physiological activities such as photosynthesis, transpiration and respiration. Different types of Brassica campestris have diverse leaf morphologies, which differ in size, color, shape, wrinkle degree and smoothness. Leaf phenotypic traits are directly related to the yield and commodity of Brassica campestris, and rapid and accurate acquisition, analysis and evaluation of leaf phenotypic traits have profound significance for digital identification of Brassica campestris germplasm resources.

[0003] Trichomes are hair-like structures that extend from the epidermis of various organs. They exist on many crops, such as rice, tobacco, tomato, potato, Chinese cabbage and cotton. Leaf trichome trait of Brassica campestris refers to the presence or absence and amount of leaf trichomes at the harvest stage. Trichomes can increase leaf thickness to reduce heat and water loss from the epidermis, and can also defend against insect and mechanical damage, effectively improving the herbivory resistance, insect resistance and stress resistance of Brassica campestris. In addition, the development degree and bifurcation number of trichomes are important basis for the differentiation of Brassica campestris, and can be used as an indicator marker for some traits in marker-assisted breeding, which has significant research value.

[0004] The traditional method of quantifying trichome density currently relies largely on visual inspection and manual counting, which is time-consuming and labor-intensive and prone to human error. SUMMARY

[0005] The present application aims to provide a method and system for automatic counting and quantification of leaf trichomes of Brassica campestris to improve the efficiency and accuracy of quantifying trichome density.

[0006] To achieve the above-mentioned purpose, the present application provides the following solutions:

[0007] A method for automatic counting and quantification of leaf trichomes of Brassica campestris, the method comprising the following steps:

[0008] Obtaining a three-dimensional topographic image of the leaf surface of Brassica campestris as a to-be-recognized image;

[0009] inputting the image to be identified into a neural network detection counting model to identify and count the trichomes on the leaf surface of the Chinese cabbage, to obtain the number of trichomes on the leaf surface of the Chinese cabbage; the neural network detection counting model is obtained by training a LeafMask R-CNN model;

[0010] calculating the trichome density of the leaf surface of the Chinese cabbage based on the number of trichomes; the trichome density is used to quantify the trichome trait of the leaf surface of the Chinese cabbage.

[0011] Optionally, the step of training the LeafMask R-CNN model comprises:

[0012] obtaining a three-dimensional topographic image of the leaf surface of the Chinese cabbage as a sample image;

[0013] annotating the trichomes on the leaf surface of the Chinese cabbage in the sample image based on Labelme software to construct a trichome dataset;

[0014] dividing the trichome dataset into a training set and a test set;

[0015] feeding the sample images marked in the training set into the LeafMask R-CNN model for training;

[0016] testing the trained LeafMask R-CNN model using the sample images marked in the test set to obtain a test result;

[0017] when the test result does not meet the preset condition, returning to the step of obtaining a three-dimensional topographic image of the leaf surface of the Chinese cabbage as a sample image, and continuing to obtain sample images to expand the trichome dataset until the test result meets the preset condition.

[0018] Optionally, the step of obtaining a three-dimensional topographic image of the leaf surface of the Chinese cabbage as a sample image comprises:

[0019] picking a leaf with the largest area at the harvesting stage of the Chinese cabbage;

[0020] unfolding the front surface of the picked leaf and placing it on a stage of a shape measurement laser microscope system;

[0021] scanning the leaf using the shape measurement laser microscope system to obtain a three-dimensional topographic image of the leaf surface of the Chinese cabbage.

[0022] Optionally, the preset condition is that the mask recognition accuracy in the test result is greater than an accuracy threshold.

[0023] Optionally, the LeafMask R-CNN model comprises a ResNet50 residual network, an FPN feature extraction network, an RPN region of interest extraction network, a symmetrization operation network, and an end network.

[0024] The ResNet50 residual network and the FPN feature extraction network are used for feature map extraction on the three-dimensional topographic image.

[0025] The RPN region of interest extraction network is used for region of interest detection on the extracted feature map to generate a candidate region detection frame.

[0026] The symmetrization operation network is used to map the candidate region detection frame to the feature map by using the RoIAlign region of interest symmetrization operation to obtain a mapped feature map.

[0027] The end network is used for recognition, segmentation mask, and counting on the mapped feature map.

[0028] Optionally, the FPN feature extraction network adopts a bidirectional fusion FPN feature pyramid feature extraction method.

[0029] The bidirectional fusion FPN feature pyramid feature extraction method is to first up-sample the deep feature map by 2 times from deep to shallow, and then down-sample the shallow feature map by 3x3 convolution with a step of 2 from shallow to deep.

[0030] Optionally, the area of the candidate region detection frame is automatically adjusted according to the result of Kernel K-Means clustering on the sample image in the RPN region of interest extraction network.

[0031] Optionally, the specific steps of Kernel K-Means clustering on the sample image are as follows:

[0032] The length and width of each sample image are constructed into data points.

[0033] The data points in the two-dimensional space are mapped to a high-dimensional feature space through nonlinear mapping.

[0034] The data points are clustered in the high-dimensional feature space using a polynomial kernel function.

[0035] The number of data points of each class obtained by clustering is used to obtain the area of the candidate region detection frame.

[0036] An automatic counting and quantification system for leaf trichomes of non-heading Chinese cabbage, the system is applied to the above method, the system comprises:

[0037] A three-dimensional topographic image acquisition module is used to acquire a three-dimensional topographic image of the leaf surface of non-heading Chinese cabbage as a to-be-recognized image.

[0038] a recognition counting module, configured to input the to-be-recognized image into a neural network detection counting model to recognize and count the trichomes on the leaf surface of the pak choi, and obtain the number of the trichomes on the leaf surface of the pak choi; the neural network detection counting model is obtained by training a LeafMask R-CNN model;

[0039] a quantification module, configured to calculate the trichome density of the leaf surface of the pak choi based on the number of the trichomes; the trichome density is used to quantify the trichome trait of the pak choi.

[0040] An electronic device includes a memory disposed at each participant, a processor disposed at each participant, and a computer program stored on the memory and executable on the processor, and the processor implements the above method when executing the computer program.

[0041] According to the specific embodiments of the present application, the following technical effects are provided:

[0042] The present application discloses a kind of pak choi leaf surface trichome automatic counting and quantification method and system, the method includes the following steps: obtain the three-dimensional topographic image of the leaf surface of pak choi, as to-be-recognized image;The to-be-recognized image is input into neural network detection counting model to recognize and count the trichomes on the leaf surface of the pak choi, and obtain the number of the trichomes on the leaf surface of the pak choi;The neural network detection counting model is obtained by training a LeafMask R-CNN model;The trichome density of the leaf surface of the pak choi is calculated based on the number of the trichomes;The trichome density is used to quantify the trichome trait of the pak choi.The present application is based on the three-dimensional topographic image of the leaf surface of the pak choi, using trained Leaf Mask R-CNN model, realizes the automatic acquisition counting of the trichomes on the leaf surface of the pak choi and quantitative evaluation, provides a feasible scheme for high-quality, high-accuracy and high-resolution phenotype information acquisition, greatly promotes the full mining of plant phenomics data, and powerfully promotes the leap-forward progress of plant breeding research. BRIEF DESCRIPTION OF DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed in the embodiments. Obviously, the drawings described below are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0044] Figure 1 A flow chart of a pak choi leaf surface trichome automatic counting and quantification method provided by the present application embodiment;

[0045] Figure 2 A structural diagram of the LeafMask R-CNN model provided for the embodiments of the present application. DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative labor fall within the protection scope of the present application.

[0047] The purpose of the present application is to provide a method and system for automatically counting and quantifying the leaf surface trichomes of Brassica campestris L. var. pekinensis to improve the efficiency of quantifying trichome density.

[0048] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0049] In order to avoid subjectivity, new technologies such as 4D confocal microscopy, micro X-ray tomography and polarized light microscopy provide new means for highly accurate and automated or semi-automated counting of trichomes. Chinese patent CN202210055645 proposes a method for observing the leaf epidermal hairs of rice, which involves making leaf sections and observing and photographing using a scanning electron microscope, manually counting the number of trichomes in a certain area, and dividing the trichome density by the area. Chinese patent CN202210003899 provides a fluorescence microscopic sectioning technology for observing and counting the glandular hairs and non-glandular hairs of Artemisia leaf. By using fluorescence staining, the distribution of glandular hairs and non-glandular hairs can be clearly observed, and the observation and counting of glandular hairs and non-glandular hairs can be completed respectively.

[0050] The above-mentioned methods are simple, efficient, reproducible and accurate, but still require the preparation of leaf sections and manual counting, which is time-consuming and subjective. The present application uses a shape measurement laser microscopy system to directly obtain a three-dimensional topographic image of the leaf surface of Brassica campestris L. var. pekinensis, and constructs a neural network detection and counting model LeafMask R-CNN based on a deep learning image target detection algorithm, realizes automatic acquisition and counting of the leaf surface trichomes of Brassica campestris L. var. pekinensis, and provides a feasible solution for high-quality, high-accuracy and high-resolution phenotype information acquisition, which greatly promotes the full exploitation of plant phenomics data and effectively promotes the leap-forward progress of plant breeding research.

[0051] Embodiment 1

[0052] Embodiment 1 of the present application provides a method for automatically counting and quantifying the leaf surface trichomes of Brassica campestris L. var. pekinensis, which comprises the following steps:

[0053] Obtain a three-dimensional topographic image of the leaf surface of the Chinese cabbage as a to-be-identified image.

[0054] Input the to-be-identified image into a neural network detection counting model to identify and count the trichomes on the leaf surface of the Chinese cabbage, and obtain the number of trichomes on the leaf surface of the Chinese cabbage; the neural network detection counting model is obtained by training a LeafMask R-CNN model.

[0055] Calculate the trichome density of the leaf surface of the Chinese cabbage based on the number of trichomes; the trichome density is used to quantify the trichome trait of the leaf surface of the Chinese cabbage.

[0056] As shown in Figure 1 The specific steps of the automatic counting and quantification method for the trichomes on the leaf surface of the Chinese cabbage provided by the embodiments of the present application are as follows:

[0057] Step S1, during the harvesting period of the Chinese cabbage, the largest leaf is picked, and the leaf is placed on the stage of a shape measurement laser microscopic system with the front side unfolded, and a three-dimensional topographic image of the leaf surface of the Chinese cabbage is collected by scanning using the shape measurement laser microscopic system.

[0058] Step S2, the leaf trichome dataset is randomly divided into a training set and a test set, and the leaf trichome mask of each training set sample (i.e. sample image) is manually labeled using Labelme software as a label for supervised training, and a label file in json format is generated.

[0059] Step S3, the labeled sample image is sent to the constructed LeafMask R-CNN model for training. The test set is used to test the trained LeafMask R-CNN model, and if the mask recognition accuracy of the test result meets the accuracy threshold, step S4 is entered, otherwise step S2 is returned, and the sample image is expanded and then re-labeled, trained and tested. Until the training is completed, the neural network detection counting model is obtained. The established neural network detection counting model can extract the trichomes on the leaf surface of the Chinese cabbage from the three-dimensional topographic image, and realize automatic detection, identification and counting of the leaf surface trichomes.

[0060] The LeafMask R-CNN model of the present embodiment is improved by referring to the target detection and segmentation network Mask R-CNN. Like Mask R-CNN, the overall architecture of LeafMask R-CNN is as follows: Figure 2As shown in the figure, it mainly includes ResNet50 residual network, FPN feature extraction network, RPN region of interest extraction network, symmetry operation network and end network. The overall process is as follows: (1) Input the sample image in the training set, use the feature pyramid of ResNet50 residual network and FPN feature extraction network as feature extractor to realize the feature map (Feature Maps) extraction of three-dimensional morphology image. (2) The feature map is input into the RPN region of interest extraction network through 3×3 convolution operation, and the number of channels of each feature map is kept consistent through 1×1 convolution operation. The classification error loss (Softmax) and bounding box regression loss (Bbox reg) are calculated respectively to generate the candidate region detection frame (Proposal). (3) The RoIAlign region of interest spatial symmetry operation is used to map the candidate region detection frame to the feature map. (4) The fully connected layer is used and the classification error loss (Softmax) and bounding box regression loss (Bbox reg) are calculated to obtain the classification category (Class) and rectangular bounding box (Box). The segmentation mask (Mask) is obtained by using the full convolutional neural network, thereby realizing the recognition and counting of leaf bristles.

[0061] The LeafMask R-CNN model has the following two improvements over the object detection and segmentation network Mask R-CNN:

[0062] a) In the ResNet50 residual network and FPN feature extraction network, a bidirectional fusion FPN feature pyramid is used, which upsamples the deep feature map by 2 times from deep to shallow, and then downsamples the shallow feature map by 3×3 convolution with a step size of 2 from shallow to deep, so as to promote the network to better enhance the feature expression of the shallow feature map. The specific process is as follows: (1) Figure 2 As shown in the figure, after ResNet50 is used to obtain the feature maps C1 to C5 of each layer, C5 is upsampled by 2 times and added to C4, and then a 3×3 convolution operation is performed to obtain the feature map c4. (2) In order from deep to shallow, the feature maps c4 to c2 are upsampled by 2 times and added to the feature maps C3 to C1 respectively, and then a 3×3 convolution operation is performed, so that C1 generates the corresponding feature map P1 output. (3) P1 is subjected to a 3×3 convolution operation with a stride of 2 and added to c2, and then a 3×3 convolution operation is performed to obtain the feature map P2 output. (4) In order from shallow to deep, the feature maps P2 to P4 are subjected to a 3×3 convolution operation with a stride of 2 and added to the feature maps c3, c4, and C5 respectively, and then a 3×3 convolution operation is performed to generate the corresponding feature maps P3 to P5 output.

[0063] b) The area of ​​the candidate region detection box in the RPN region of interest extraction network is automatically adjusted based on Kernel K-Means clustering of the sample image. Assuming that the sample image contains n objects, each object's length and width attributes constitute a data point. A nonlinear mapping is used to map the data points in the input space to a high-dimensional feature space. A polynomial kernel function is used to replace the inner product of the nonlinear mapping, and clustering is performed in the feature space. The area of ​​the candidate region detection box is obtained by determining the number of clustered data points at each cluster center. This achieves adaptive region of interest extraction, allowing adjustments based on actual scene requirements.

[0064] Step S4: deploying the neural network detection and counting model trained in step S3 to detect the three-dimensional morphological image of the leaf surface collected in real time, identifying and counting the bristles on the leaf surface of the non-heading cabbage.

[0065] Step S5: Automatically obtain the projection area of ​​the three-dimensional topography image using shape measurement laser microscope system software. Calculate the density of the bristles on the leaf surface of the non-heading cabbage by dividing the number of bristles in the projection area of ​​the three-dimensional topography image by the projection area.

[0066] Example 2

[0067] Example 2 of the present invention provides a system for automatically counting and quantifying leaf hairs of non-heading Chinese cabbage, the system being applied to the method of Example 1, and comprising:

[0068] The three-dimensional shape image acquisition module is used to acquire the three-dimensional shape image of the leaf surface of the non-heading cabbage as the image to be identified.

[0069] The recognition and counting module is used to input the image to be recognized into a neural network detection and counting model to recognize and count the bristles on the leaf surface of non-heading cabbage, thereby obtaining the number of bristles on the leaf surface of non-heading cabbage; the neural network detection and counting model is obtained by training the LeafMask R-CNN model.

[0070] A quantification module is used to calculate the density of bristles on the leaf surface of non-heading cabbage based on the number of bristles; the bristle density is used to quantify the bristle traits of the leaf surface of non-heading cabbage.

[0071] The specific implementation method of each module in Example 2 of the present invention is the same as the specific steps in Example 1, and will not be repeated here.

[0072] Example 3

[0073] Embodiment 3 of the present invention provides an electronic device, comprising a memory disposed at each participant, a processor disposed at each participant, and a computer program stored on the memory and executable on the processor, wherein the training method of embodiment 1 is implemented when the processor executes the computer program.

[0074] Compared with the related art, the embodiment of the present application automatically acquires the trait of leaf trichome of non-heading Chinese cabbage by constructing a neural network detection counting model, solves the problems of time-consuming and laborious and subjective errors caused by manual visual counting, first provides a feasible quantitative scheme for digital identification and evaluation of the trait of leaf trichome of non-heading Chinese cabbage, and effectively promotes the breeding improvement and screening of non-heading Chinese cabbage. Meanwhile, the present application does not need to make sections and is simple and easy to operate and has good repeatability.

[0075] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts between the various embodiments can be mutually referred to.

[0076] The principles and implementation manners of the present application are described by applying specific examples in this paper, and the above description of the embodiments is only used to help understand the method of the present application and its core idea; meanwhile, for the general technical personnel in the field, the specific implementation manners and application ranges will be changed according to the idea of the present application. In conclusion, the content of the specification should not be understood as the limitation of the present application.

Claims

1. A method for automatically counting and quantifying the bristles on the leaves of non-heading cabbage, characterized in that: The method comprises the following steps: Acquire a three-dimensional topographic image of a non-heading cabbage leaf surface as an image to be recognized; The image to be identified is input into a neural network detection and counting model to identify and count the bristles on the leaf surface of non-heading cabbage, thereby obtaining the number of bristles on the leaf surface of non-heading cabbage; the neural network detection and counting model is obtained by training the LeafMask R-CNN model; the LeafMask The R-CNN model includes: a ResNet50 residual network, an FPN feature extraction network, an RPN region of interest extraction network, a symmetry operation network and an end network; the ResNet50 residual network and the FPN feature extraction network are used to extract feature maps from three-dimensional morphology images; the RPN region of interest extraction network is used to detect regions of interest on the extracted feature maps and generate candidate region detection frames; the symmetry operation network is used to map the candidate region detection frames to the feature maps using the RoIAlign region of interest spatial symmetry operation to obtain the mapped feature maps; the end network is used to identify, segment, mask and count the mapped feature maps; the FPN feature extraction network adopts a bidirectional fusion FPN feature pyramid feature extraction method; the bidirectional fusion FPN feature pyramid feature extraction method is to first upsample the deep feature maps by 2 times from deep to shallow, and then downsample the shallow feature maps by 3×3 convolution with a step size of 2 from shallow to deep; the RPN region of interest extraction network automatically adjusts the area of ​​the candidate region detection frame based on the result of KernelK-Means clustering of the sample image; The density of the bristles on the leaf surface of the non-heading cabbage is calculated based on the number of the bristles; and the density of the bristles is used to quantify the bristle traits of the leaf surface of the non-heading cabbage.

2. The method for automatically counting and quantifying the leaf hairs of non-heading Chinese cabbage according to claim 1, characterized in that: The steps for training the LeafMask R-CNN model are: Acquire a three-dimensional topographic image of a non-heading cabbage leaf surface as a sample image; Using Labelme software, the leaf bristles of the non-heading cabbage in the sample image are labeled to construct a leaf bristle dataset; Divide the leaf bristle dataset into training set and test set; Send the labeled sample images in the training set to the LeafMask R-CNN model for training; Use the labeled sample images in the test set to test the trained LeafMask R-CNN model and obtain the test results; If the test result does not meet the preset conditions, the process returns to step "obtaining a three-dimensional topographic image of the non-heading cabbage leaf surface as a sample image", continues to obtain sample images, and expands the leaf bristle dataset until the test result meets the preset conditions.

3. The method for automatically counting and quantifying the leaf hairs of non-heading Chinese cabbage according to claim 2, characterized in that: The method of obtaining a three-dimensional topographic image of the leaf surface of non-heading cabbage as a sample image specifically includes: During the harvest period of non-heading cabbage, pick the largest leaves; unfolding the front side of the removed leaf and placing it on the stage of a shape measurement laser microscope system; The shape measurement laser microscope system is used to scan the leaf to obtain a three-dimensional topographic image of the non-heading cabbage leaf surface.

4. The method for automatically counting and quantifying the leaf hairs of non-heading Chinese cabbage according to claim 2, characterized in that: The preset condition is that the mask recognition accuracy in the test result is greater than the accuracy threshold.

5. The method for automatically counting and quantifying the leaf hairs of non-heading cabbage according to claim 1, characterized in that: The specific steps for performing Kernel K-Means clustering on sample images are: The length and width of each sample image are used as data points; Map the data points in the two-dimensional space to the high-dimensional feature space through nonlinear mapping; Use polynomial kernel function to cluster data points in high-dimensional feature space; The area of ​​the candidate region detection box is obtained according to the number of data points of each class obtained by clustering.

6. An automatic counting and quantification system for the bristles on the leaves of non-heading cabbage, characterized in that: The system is applied to the method according to any one of claims 1 to 5, and the system includes: a three-dimensional shape image acquisition module, used to acquire a three-dimensional shape image of the leaf surface of non-heading cabbage as an image to be identified; an identification and counting module, configured to input the image to be identified into a neural network detection and counting model to identify and count the bristles on the leaf surface of the non-heading cabbage, thereby obtaining the number of bristles on the leaf surface of the non-heading cabbage; the neural network detection and counting model is obtained by training a LeafMask R-CNN model; A quantification module is used to calculate the density of bristles on the leaf surface of non-heading cabbage based on the number of bristles; the bristle density is used to quantify the bristle traits of the leaf surface of non-heading cabbage.

7. An electronic device, characterized in that: The method comprises a memory arranged at each participant, a processor arranged at each participant, and a computer program stored on the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 5 when executing the computer program.

Images