Tea flower identification, flower quantity estimation and flowering phase estimation method

Abstract

The invention relates to the technical field of tea tree phenotype identification, and discloses a tea tree flower identification, flower quantity estimation and flowering phase estimation method, which comprises the following steps of: obtaining the number of tea tree buds, blooming flowers and decayed flowers according to acquired tea tree flower images and acquisition time through an offline trained tea tree flower identification model; obtaining the number of tea tree buds and blooming flowers with higher accuracy through an offline trained flower quantity linear calibration model; and outputting the collection time, the number of decayed flowers output by the tea flower recognition model and the number of tea buds and blooming flowers output by the flower quantity linear calibration model through an ANN flowering phase detection model trained offline to obtain the flowering phase category of the tea flowers. The method provided by the invention is high in generalization and robustness, can accurately identify different illumination, tea tree varieties, flowering densities and tea tree flowers, flower quantities and flowering phases in different flowering phase stages, and provides reliable technical support and data analysis basis for agricultural production management and tea tree flowering phase character investigation.

Description

Technical Field

[0001] The invention belongs to the field of tea tree phenotype identification, and in particular relates to a method for tea tree flower identification, flower quantity estimation and flowering period estimation. Background Art

[0002] Tea flower, as the reproductive organ of tea tree, is of great significance to tea hybrid breeding, observation of tea growth conditions, and tea genetics and classification research. The number of tea flowers is one of the characteristics of tea varieties. The number of tea flowers is closely related to the genes of tea varieties, growth conditions, cultivation and management methods, light, and temperature. In tea cultivation, removing flower buds can regulate carbon and nitrogen metabolism, increase tea bud yield, increase amino acid content, and have a positive effect on tea quality. There are significant differences in the number of flowers, the start and end time of flowering, and the duration of flowering among different varieties. Among them, the flowering period will affect the selection of parents in tea hybrid breeding, and the flowering period needs to be relatively consistent for hybridization. In breeding work, it is necessary to investigate tea flower traits such as flower amount and flowering period.

[0003] However, traditional observation of tea tree flower traits is mainly based on manual measurement and naked eye observation, which consumes a lot of manpower and has a strong subjective problem. There are obvious differences in the methods and standards for observing the amount of flowers and flowering period between different studies. The flowering period of tea trees is long, and the flowering period of different varieties spans a large range. A single manual survey is prone to errors and cannot accurately describe each variety and individual. Picking tea tree flowers and counting them, and then estimating the overall number of flowers and flower yield, cannot accurately measure the overall amount of flowers, and picking will also affect the overall amount of flowers.

[0004] Machine learning includes support vector machine SVM, partial least squares regression PLSR, K-means and other machine learning algorithms that can be used to process large amounts of data and establish models for predicting labels such as yield and phenological period. In recent years, deep learning algorithms such as YOLO and Faster RCNN have been widely used in image recognition and classification processing in various industries, and have performed well in the fields of pest and disease identification, phenological period detection, yield estimation, rice ear and other crop target detection in the agricultural field. Therefore, machine learning and deep learning can be applied to tea flower target detection, flower quantity and flowering period estimation, and tea flower quantity and flowering period detection models can be established. There are cases of target detection in other crops using algorithms such as YOLO, such as a large-scale oil tea forest agricultural phenological monitoring method (application publication number CN 118334581A), an oil tea fruit flower recognition and positioning system (publication (announcement) number: CN109886062A), and an Android-based rose flower recognition method (publication (announcement) number: CN114463639A).

[0005] However, applying algorithms such as Yolo to tea flower recognition faces a variety of technical challenges, including background interference, diverse tea tree species resources, long tea flowering period, and dynamic changes in tea flower quantity throughout the flowering period.

[0006] Serious background noise: Tea trees grow in outdoor tea gardens with complex backgrounds. Different lighting, backgrounds, and weather conditions will cause the captured tea tree flower images to have a lot of background noise and changes. For example, the circular light spots on the leaves in the background are mistakenly identified as flower buds. It is difficult to observe the number of flowers under strong light and direct light conditions, so the model needs to be highly robust and generalizable.

[0007] Diversity of tea tree varieties: There are many types of tea trees, and the tea tree flower phenotypes of different varieties vary in size, color, etc. The tea tree varieties have various tree shapes and postures, including small trees and shrubs, and the tree postures include open, semi-open, and upright. The growth position and growth state of tea tree flowers of different tree shapes and postures are different.

[0008] Category imbalance and category confusion: In tea tree flower images, the number of tea tree buds is greater than that of open and decaying tea tree flowers, which indicates a category imbalance problem. This will cause the model to over-identify flower buds and ignore open flowers, affecting recognition accuracy and causing errors in flower quantity discrimination. There are a large number of intermediate types between tea tree buds, open flowers, and decaying flowers. The morphology between different types is difficult to distinguish, which will cause a decrease in recognition accuracy.

[0009] Tea flowers are seriously blocked by tea leaves and flowers between them: Tea flowers generally grow on the sides of tea trees and tea rows are densely planted. Tea flowers are often blocked by branches and leaves, which appear partially blocked and segmented in the image, affecting the recognition of tea flowers. Tea flowers usually grow in several places, and the buds and flowers block each other, which also makes it difficult to detect and accurately count tea flowers.

[0010] The number of flowers on tea trees at different positions varies greatly: the number of flowers on tea trees of the same variety planted in different positions on a tea row varies greatly. Usually, the tea trees at the two ends of the tea row with sufficient light have more flowers, while the tea trees in the middle of the tea row with lack of light have fewer flowers. Uneven image collection or too few images will cause the model to have large errors in its assessment of the number of flowers on the tea trees.

[0011] In addition, the flowering dynamics, starting time of flowering, and duration of flowering of tea trees vary greatly among different varieties and in different environments. The amount of flowers changes dynamically throughout the flowering period. Observation over only a period of time cannot accurately reflect the amount of flowers of all varieties throughout the flowering period. Flowering period identification is required to use peak flowering period data to observe and compare the amount of flowers. Summary of the invention

[0012] The technical problem to be solved by the present invention is to provide a method for identifying tea flowers, estimating the amount of flowers and estimating the flowering period, which is used to identify and distinguish tea flowers of different tea varieties in complex field environments, as well as the number of buds, blooming flowers and withered flowers of tea trees.

[0013] In order to solve the above technical problems, the present invention provides a method for tea tree flower identification, flower quantity estimation and flowering period estimation, comprising: collecting tea tree flower images, then inputting the collected images and collection time into a computer, obtaining the number of tea tree flower buds, open flowers and decayed flowers through an offline trained tea tree flower recognition model, and then obtaining the number of tea tree flower buds and open flowers with higher accuracy through an offline trained flower quantity linear calibration model; the collection time, the number of decayed flowers output by the tea tree flower recognition model, and the number of tea tree flower buds and open flowers output by the flower quantity linear calibration model are output through an offline trained ANN flowering period detection model to obtain the flowering period category of the tea tree flower.

[0014] As an improvement of the method of tea tree flower identification, flower quantity estimation and flowering period estimation of the present invention:

[0015] The tea flower recognition model is based on the YOLOv5 model, and an SE module is added after the third C3 module of the backbone network.

[0016] As a further improvement of the method for tea tree flower identification, flower quantity estimation and flowering period estimation of the present invention:

[0017] The formula of the linear calibration model of flower quantity is as follows:

[0018]

[0019] Among them, w 1 and w 2 is the weight parameter, x 1 is the number of tea tree buds, x 2 is the number of open flowers, To predict the final amount of flowers.

[0020] As a further improvement of the method for tea tree flower identification, flower quantity estimation and flowering period estimation of the present invention:

[0021] The ANN flowering period detection model is a multi-layer perceptron, including an input layer, 6 hidden layers and an output layer.

[0022] As a further improvement of the method for tea tree flower identification, flower quantity estimation and flowering period estimation of the present invention:

[0023] The offline training process of the tea flower recognition model is as follows:

[0024] (1) Periodically collect time series images of tea flowers, covering different tea varieties, lighting, backgrounds, and flowering stages;

[0025] (2) Each image was cropped to a uniform size and randomly divided into a tea flower training set, a tea flower validation set, and a tea flower test set. Then, each sample was subjected to data augmentation and manually labeled with the labels of “tea bud”, “open flower”, and “decayed flower”.

[0026] (3) During the training process, the loss function is used to calculate various losses, calculate the gradient, and then the model parameters are optimized through back propagation of the SGD optimizer.

[0027] As a further improvement of the method for tea tree flower identification, flower quantity estimation and flowering period estimation of the present invention:

[0028] The offline training process of the linear calibration model of flower quantity is as follows:

[0029] (1) constructing data subsets of the tea flower time series images according to different tea varieties, light and environment, and then using the offline trained tea flower recognition model to detect, outputting the flower quantity data of each data subset, including the number of tea buds and open flowers, using manually observed flower quantity data as labels, and then randomly dividing into a flower quantity calibration training set and a flower quantity calibration verification set;

[0030] (2) Recollect tea flower images, classify them by tea tree variety, light, and environment, and manually measure the number of tea flowers in each image as the flower quantity calibration test set;

[0031] (3) The flower quantity calibration training set is input into the flower quantity linear calibration model, and the loss is obtained through the MSE loss function. The SGD optimizer back propagates to continuously update the parameters. After each training cycle (epoch), the performance of the model is evaluated using the flower quantity calibration validation set. After the training, the flower quantity linear calibration model classified by tea tree variety, light and environment is obtained;

[0032] The flower quantity calibration test set was input into the tea tree flower recognition model to obtain the uncalibrated flower quantity results. Then, the trained flower quantity linear calibration model was used to output the calibrated flower quantity. The calibrated flower quantity was compared with the manually measured flower quantity for R 2 Correlation analysis was performed to further verify the effect of the linear calibration model of flower quantity.

[0033] As a further improvement of the method for tea tree flower identification, flower quantity estimation and flowering period estimation of the present invention:

[0034] The offline training process of the ANN flowering period detection model is as follows:

[0035] (1) detecting the tea tree flower images in the tea tree flower time series images using the offline trained tea tree flower recognition model to obtain the number of tea tree buds, open flowers and decayed flowers corresponding to each image, and then processing each sample, including adding time data, filtering out low-quality data, and averaging every three samples of the same time and variety to generate a new sample;

[0036] (2) Each new sample is manually labeled with a flowering period category label and divided into a flowering period training set and a flowering period verification set. In addition, tea tree flower images from different years are collected as a flowering period test set.

[0037] (3) The flowering period training set samples are input into the model for forward propagation. The predicted probability of each flowering period category is output through the softmax function and used together with the flowering period category label to calculate the cross entropy loss. The parameters are updated through back propagation of the Adam optimizer. After each training cycle, the performance of the model is evaluated using the flowering period validation set to obtain the trained ANN flowering period detection model, which is then tested using the flowering period test set to verify the accuracy and generalization of the trained ANN flowering period detection model.

[0038] As a further improvement of the method for tea tree flower identification, flower quantity estimation and flowering period estimation of the present invention:

[0039] The method for collecting tea tree flower images is:

[0040] (1) The shooting angle is perpendicular to the ground;

[0041] (2) The shooting position is outside the tea row, and the image should include the flowers on the side of the tea tree;

[0042] (3) Collect images at evenly spaced intervals at different locations along each tea row.

[0043] The beneficial effects of the present invention are mainly reflected in:

[0044] 1. The method of the present invention is highly generalizable and robust, and can be applied to the identification of tea flowers of different tea varieties in complex field environments, and can be used to identify tea flowers of different light, tea varieties, flowering densities and different flowering stages in a complex tea flower growth environment.

[0045] 2. The linear model for flower quantity detection of the present invention calibrates the number of flower buds, open flowers and withered flowers, and realizes accurate counting of the flower quantity of flowers of different varieties of tea trees, making it closer to the actual flower quantity of tea trees in different environments, tea varieties and different stages of flowering period, thereby improving the adaptability of the model in different environments and the accuracy of flower quantity assessment, and providing support for tea tree trait investigation and tea tree flower thinning.

[0046] 3. The present invention realizes tea tree flowering period detection through time series images and tea tree flowering period classification model based on artificial neural network ANN, providing reliable technical support and data analysis basis for agricultural production management and tea tree flowering period trait investigation, while assisting flower quantity detection to make flower quantity data more reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The specific implementation modes of the present invention are further described in detail below with reference to the accompanying drawings.

[0048] Figure 1 It is a structural schematic diagram of the tea flower recognition model of the present invention;

[0049] Figure 2 It is a structural schematic diagram of the ANN flowering period detection model of the present invention;

[0050] Figure 3 It is a flow chart of the offline training process of the tea tree flower quantity detection model and the ANN flowering period detection model of the present invention;

[0051] Figure 4 This is a comparison chart of the accuracy of the tea flower recognition model of the present invention and different existing YOLO networks on the same tea flower test set;

[0052] Figure 5 A comparison chart of the computational effort of the tea flower recognition model of the present invention and different existing YOLO networks on the same tea flower test set;

[0053] Figure 6 It is a confusion matrix diagram of the ANN flowering period detection model of the present invention on the flowering period verification set data;

[0054] Figure 7 It is a confusion matrix diagram of the ANN flowering period detection model of the present invention on the flowering period test set data. DETAILED DESCRIPTION

[0055] The present invention is further described below in conjunction with specific embodiments, but the protection scope of the present invention is not limited thereto:

[0056] Embodiment 1, a method for identifying tea tree flowers, estimating flower quantity and estimating flowering period.

[0057] First, the tea flower images were taken vertically on the ground by mobile phone to construct the training data set of the tea flower quantity detection model. The sample images in the data set were scaled to 640×640 and input into the tea flower recognition model. Different feature maps were generated through the backbone feature extraction network. The feature maps were further downsampled and feature fused in the neck part. Combining shallow and deep features, the C3 modules of 18, 21, and 24 output feature maps of 80×80, 40×40, and 20×20 sizes for detecting small targets, medium targets, and large targets respectively. The model was divided into grids to generate anchor frames of different sizes and densities. The anchor frames with high scores (including target scores and category scores) were retained and repeated anchor frames were suppressed using NMS. The retained anchor frames were displayed on the image, and the category confidence of each anchor frame and its predicted category was displayed to obtain the number of detected tea buds (bud), open flowers (B flower), and decayed flowers (Wflower). After that, the linear model trained in each variety and environment was calibrated to make it closer to the true value of the flower quantity, and the flower quantity was output. Then, the flowering period is determined by the ANN flowering period detection model based on the number and time information of the tea tree buds, open flowers, and decaying flowers, thereby realizing automatic flowering period determination. The offline training process of the tea tree flower quantity detection model and the ANN flowering period detection model of the present invention is as follows: Figure 3 The combination of flower quantity and flowering period detection models can realize accurate dynamic monitoring of tea tree flower quantity and flowering period, specifically:

[0058] 1. Construction and offline training of tea tree flower quantity detection model

[0059] The tea tree flower quantity detection model of the present invention includes a tea tree flower recognition model (TflosYOLO model) and a flower quantity linear calibration model. The tea tree flower images of different tea tree varieties, illumination, background and flowering stage are collected, and the recognition and counting of tea tree flowers in three different stages are realized by the tea tree flower recognition model (TflosYOLO model), and the tea tree flower data sets of different tea tree varieties, illumination, background and flowering stage are detected respectively, and the tea tree flower quantity data of different varieties of tea trees are obtained. Afterwards, the uncropped tea tree flower time series images of each variety are detected by the offline trained TflosYOLO model, and the tea tree flower buds, the number of open flowers and the manually measured tea tree flower quantity data are jointly constructed to be applicable to each variety, environment, and flowering stage. The output flower quantity is calibrated, and the flower quantity data is output to realize accurate evaluation of the flower quantity of different varieties and cultivation environments.

[0060] 1.1. Tea tree flower image data collection and preparation:

[0061] The mobile phone collects time series images of tea flowers covering different tea tree varieties, lighting, backgrounds and flowering stages, and builds a dedicated data set for model training and verification. The tea flowers in the collected images include 26 tea tree varieties, and the lighting conditions include front light, back light, strong light, and weak light. The background includes the tea tree environment background under different planting locations and densities. The flowering period includes five different stages: the beginning of flowering, the early flowering period, the middle flowering period, the late flowering period, and the end of flowering.

[0062] Shooting methods include:

[0063] (1) Time series images of different tea tree varieties were collected at intervals of approximately one week.

[0064] (2) The shooting angle of the mobile phone is perpendicular to the ground, and the shooting focal length is 0.6 times.

[0065] (3) The shooting distance is about 26 cm from the outer side of the tea row. The shooting height should be as consistent as possible, and the image should include the flowers on the side of the tea tree. It can be adjusted as needed. For example, some tea trees are small trees with high branches and flowers, so the shooting height should be increased.

[0066] (4) To ensure that the images accurately reflect the amount of tea tree flowers, images were collected evenly at different locations in the tea row, with images taken every 1–2 m.

[0067] 1.2. Tea tree flower quantity detection model

[0068] The input tea flower image is first passed through the offline trained tea flower recognition model (TflosYOLO model) to detect the number of tea buds (bud), open flowers (B flower), and withered flowers (W flower). Then, the flower quantity linear calibration model trained in each variety and environment is used to make it closer to the actual value of the flower quantity, and the flower quantity is output.

[0069] (1) The tea flower recognition model (TflosYOLO model) of the present invention integrates the SE (Squeeze-and-Excitation Networks) module based on the YOLOv5 model to filter the noise information in the image, focus on more critical channel information, and optimize the feature extraction process. The depth and width of the model are changed to improve the recognition accuracy of the model, and the model's adaptability to complex backgrounds and lighting changes is enhanced on the basis of reducing the number of parameters and the amount of calculation. The TflosYOLO model achieves high recognition accuracy of tea flowers in different environments with a small amount of calculation, and has high generalization and robustness.

[0070] The TflosYOLO model structure is based on the YOLOv5 model version 7.0, including the backbone CSPDarknet-53 network, the feature fusion neck part and the final detection and output layers, such as Figure 1As shown. The original image is uniformly scaled to 640×640 and input into the backbone network for feature extraction. The convolution operation in the conv and C3 modules is used to downsample and generate feature maps. Downsampling is performed by 2, 4, 8, 16, and 32 times at layers 0, 1, 3, 5, and 7, respectively. This model adds an SE module to the seventh layer of the backbone network. The C3 module on the sixth layer outputs the feature map as input to the SE module, adjusts the weight of each channel in the feature map, and enhances the model's representation of important features, which helps to improve the performance and generalization ability of the model and reduce the impact of background noise. The last layer of the backbone network, the SPPF module, performs pyramid pooling and multi-scale fusion on the input feature map. It compresses the feature map through pooling, extracts information, and fuses information at different scales to enhance model information.

[0071] After that, the feature map is forward propagated to the neck, and the 12th and 16th layers are upsampled to fuse the neck part with the feature map of the previous layer. The 13th layer concat module fuses the 6th and 12th layers; the 17th layer concat module fuses the 4th and 16th layers; the 20th layer concat module fuses the 15th and 19th layers; the 23rd layer concat module fuses the 11th and 22nd layers. Reusing the previous feature map improves the utilization efficiency and expression ability of the feature map and avoids the loss of some features.

[0072] Finally, the C3 modules of 18, 21, and 24 output 8x, 16x, and 32x downsampled feature maps as input to the YOLO layer for target detection and output. Feature maps of sizes 80×80, 40×40, and 20×20 are used to detect small targets, medium-sized targets, and large targets, respectively.

[0073] (2) Flower quantity linear calibration model

[0074] Tea tree flower images of different tea tree varieties, lighting, backgrounds and flowering stages are collected, and the TflosYOLO model is used to realize the recognition and counting of tea tree flowers in three different stages. Tea tree flowers of different tea tree varieties, lighting, backgrounds and flowering stages are detected respectively to obtain tea tree flower quantity data of different tea tree varieties. However, in actual use, there are errors between the flower quantity data output by the TflosYOLO model and the actual flower quantity due to factors such as environment and planting management. For example, there are significant differences in the flowering conditions of pruned tea trees and unpruned tea trees. The unpruned tea trees have more flowers, dense distribution, easy to block each other and be blocked by unpruned branches. Therefore, it is necessary to calibrate the linear model of unpruned management to make the flower quantity detection closer to the actual real flower quantity. The present invention jointly constructs a linear calibration model for flower quantity suitable for each variety, environment, and flowering stage based on the number of tea tree flower buds and open flowers output by the TflosYOLO model and the manually measured tea tree flower quantity data, calibrates the flower quantity output by the TflosYOLO model, outputs the flower quantity data, and realizes accurate evaluation of the flower quantity of different varieties and cultivation environments.

[0075] The linear calibration model of flower quantity uses the pytorch framework, the root mean square error MSE loss function, and the SGD optimizer. The total number of model training rounds is 50. The formula of the linear model for flower quantity estimation is as follows:

[0076]

[0077] Among them, w 1 and w 2 is the weight parameter, x 1 is the number of tea tree buds, x 2 The number of open flowers. To predict the final amount of flowers.

[0078] 1.3 Offline training of tea tree flower quantity detection model

[0079] 1.3.1 Offline training of tea flower recognition model

[0080] (1) Construction of training set

[0081] (a) Image preprocessing and data augmentation

[0082] In step 1.1, the original image size captured by the mobile phone is 3280×2464. Each image is cropped and divided into 4 images to better fit the subsequent model input. The cropped image size is 1640×1232.

[0083] (b) The cropped images are randomly assigned to the tea flower training set, tea flower validation set, and tea flower test set of the tea flower quantity detection model in a 6:2:2 ratio, which are used to train the model and evaluate the model's accuracy, speed, and other performance.

[0084] (c) Data enhancement: The images in the tea flower training set, tea flower validation set, and tea flower test set were subjected to data enhancement such as mosaic enhancement, flipping, translation, and color enhancement and used as the input of the model to solve the problem of model underfitting caused by insufficient training data, especially insufficient training data for decaying flowers.

[0085] (d) The YOLOv5 model is a supervised learning model, which requires manual labeling of the target location and category to be detected for model training and verification. The tea flowers are labeled using LabelImg software, and are divided into tea buds, open flowers, and decaying flowers, and saved as YOLO label format label files.

[0086] (2) Training parameters:

[0087] The tea flower recognition model (TflosYOLO model) of the present invention is trained with a total round number of 300, a batch size of 8, a learning rate of 0.01, and an SGD optimizer. By changing the depth_multiple and width_multiple parameters, the final model size is between YOLOv5s and YOLOv5m, and the performance is higher than different versions such as YOLOv5s, YOLOv5m, YOLOv5l, and YOLOv5x.

[0088] (3) Loss Function

[0089] YOLO is a one-step method for target detection. It does not have a separate network for generating accurate candidate target frames. Instead, it directly generates candidate target frames, i.e., anchor frames (bounding boxes), on the input image, and uses a classifier to classify and regress these frames. The images in the tea flower training set are divided into S×S grids according to the network feature map. Each grid generates a certain number of anchor frames, and outputs 5+n values ​​corresponding to each bounding box, i.e., four coordinate information, one target score, and scores for n categories (assuming that the model implements n types of classification). After that, various losses are calculated during the training process, the gradient is calculated, and then the model parameters are changed through the optimizer back propagation. The loss function includes classification loss + target loss + positioning loss, and the loss function is the sum of these three multiplied by their respective weights.

[0090] Loss = w box l box +w obj l obj +w cls l cls (1)

[0091] Among them, l box The box regression loss is used to measure the position difference between the model prediction box and the true box; objThe target confidence loss measures the accuracy of the model's judgment on whether a target exists; cls The classification loss indicates how accurately the model classifies the target, w box , w obj , w cls are all weight parameters.

[0092] During the test, the tea tree flower test set image is input to generate a feature map, which is also divided into S×S grids. The anchor frames of each grid are scored, the anchor frames with low scores are deleted, and the NMS algorithm is used to delete duplicate anchor frames. The confidence score is displayed for the anchor frames retained at the end. Confidence score calculation:

[0093] confidence score=Pr(object)*IoU(pred,truth)*Pr(class) (2)

[0094] Where: Pr(object) represents the probability of the target existence, IoU represents the intersection-over-union ratio of the predicted box and the real box, and Pr(class) represents the probability that the predicted box belongs to (D each) category. The IoU calculation formula is as follows:

[0095]

[0096] Among them B P (predicted bounding box) represents the predicted box, B gt (ground truth box) represents the manually annotated real box.

[0097] The accuracy of the tea flower recognition model on the tea flower validation set is 0.792 (79.2%), and the accuracy on the tea flower test set is 0.802 (80.2%).

[0098] After the tea tree flower recognition model is trained offline, the flower quantity data of each image and the target detection results of buds, open flowers, and decaying flowers (the target detection box is displayed on the image) are output for offline training of the flower quantity linear calibration model.

[0099] 1.3.2 Offline training of the linear calibration model for flower quantity

[0100] (1) Training data construction

[0101] The flower quantity data output by the tea tree flower recognition model training is used as the training data input for the flower quantity linear calibration model.

[0102] In step 1.1, the original image size of the tea tree flower time series image collected by the mobile phone is 3280×2464, including different tea tree varieties, lighting, background and flowering stage. The original images are selected and classified into data subsets containing different tea tree varieties, lighting, and environment, and then the offline trained tea tree flower recognition model is used for detection, and the flower quantity data of each data subset (including the number of tea tree buds and open flowers) is output. The flower quantity data of each picture is a sample, and the manually observed flower quantity data is used as a label to produce data subsets of the flower quantity linear calibration model classified by different tea tree varieties, lighting, and environment.

[0103] For the data subsets of different tea tree varieties, lighting, and environments, they are randomly divided into a flower quantity calibration training set and a flower quantity calibration validation set for the flower quantity linear calibration model in a ratio of 3:1. In addition, new tea tree flower images of different tea tree varieties, lighting, and environments are collected, and the tea tree flowers in each image are manually measured according to the tea tree variety, lighting, and environment classification. After the image is collected, the tea tree flowers in the corresponding range are collected and counted to construct the flower quantity calibration test set of the flower quantity linear calibration model. The images of tea tree flowers are re-collected to construct the flower quantity calibration test set, with the aim of conducting an unbiased evaluation of the model and testing its generalization ability on unseen data.

[0104] (2) Training process

[0105] The flower quantity calibration training set is input into the model, and the loss is obtained through the MSE loss function. The SGD optimizer back-propagates and continuously updates the parameters. After each training cycle (epoch), the performance of the model is evaluated using the flower quantity calibration validation set. The total number of rounds for all sample training is 50. After the training, the optimal flower quantity linear calibration model for each tea tree flower variety under different environments and light classifications is obtained.

[0106] (3) Testing process

[0107] The flower quantity calibration test set was input into the tea tree flower recognition model to obtain the uncalibrated flower quantity results. Then, the trained flower quantity linear calibration model was used to output the calibrated flower quantity. The calibrated flower quantity was compared with the manually measured flower quantity for R 2 Correlation analysis was performed to further verify the effect of the linear calibration model of flower quantity. 2 The calculation formula is as follows:

[0108]

[0109] Where n is the number of samples, y i To manually measure the amount of flowers, For the model to predict the amount of flowers, This is the average value of the manually measured flower quantity.

[0110] R of the linear calibration model of flower quantity of each tea tree flower variety under different environments and light classifications on the corresponding flower quantity calibration test set 2 It is above 0.9.

[0111] 2. Construction of ANN flowering period detection model

[0112] 2.1、Construction of flowering period dataset:

[0113] The flowering period of tea trees is divided into five categories: the beginning of flowering, the early flowering period, the middle flowering period, the late flowering period, and the final flowering period. The original uncropped tea tree flower images in the tea tree flower time series images collected in step 1.1 are used to construct the training set and validation set used by the ANN flowering period detection model. Because the flowering period of tea trees is affected by factors such as climate and there are differences between different years, tea tree flower images of different years are collected to construct a flowering period test set to further verify the generalization of the flowering period detection model. The tea tree flower quantity detection model trained offline in step 1 estimates the flower quantity corresponding to each image (the number of tea tree buds, open flowers, and decaying flowers), and adds time data. The flower quantity and time data of each time series image constitute a flowering period sample, which together construct the original flowering period data set.

[0114] After that, the original flowering period dataset was preprocessed. First, low-quality data was filtered out. For example, images of varieties with too few flowers could not be used for flowering period judgment and were deleted from the dataset. After that, a new sample was generated by taking the average of every three samples from the same time and variety to avoid the influence of extreme samples and reflect the overall flowering period of the variety. Then, each generated new sample was manually labeled with the flowering period category label, which included the beginning of flowering, early flowering period, middle flowering period, late flowering period and final flowering period. The flowering period data was divided into a flowering period training set and a flowering period verification set according to an 8:2 ratio, and images from different years were used as flowering period test sets for testing.

[0115] 2.2、Design and training of ANN flowering period detection model:

[0116] The ANN flowering period detection model is a 7-layer multi-layer perceptron MLP neural network structure, including an input layer, 6 hidden layers and an output layer. Figure 2 The input layer consists of 4 neurons, which are used to input the number and time information of tea tree buds, open flowers, and decaying flowers in each picture. The label is the manually marked flowering period category label. After 6 hidden layers and softmax classification, the flowering period category with the highest probability is finally output (beginning of flowering, early flowering period, middle flowering period, late flowering period, and final flowering period). The structure diagram of the flowering period discrimination model is shown in Figure 2.

[0117] The ANN flowering period detection model uses the pytorch framework, ReLU activation function, softmax, and Adam optimizer. The key training parameters are as follows:

[0118] Number of training samples: 3667

[0119] Training batch size: 16

[0120] Learning rate lr: 0.001

[0121] Epochs: 80

[0122] Software version: pytorch-cuda=11.8, Cuda 11.3, Python 3.8.

[0123] Training process:

[0124] The flowering period training set samples are fed into the model for forward propagation. The predicted probability of each flowering period category is output through the softmax function, and used together with the flowering period category label to calculate the cross entropy loss, i.e. CrossEntropyLoss. The parameters are updated through the Adam optimizer back propagation. After each training cycle (epoch), the flowering period validation set is used to evaluate the performance of the model. All samples were cycled for 80 times to obtain the trained ANN flowering period detection model. The Softmax function formula is as follows:

[0125]

[0126] in, is the predicted probability distribution, o i is the i-th element of the unnormalized prediction O, k is a vector including several prediction outputs, and the Softmax function makes the prediction output within [0, 1].

[0127] 2.3. ANN flowering period detection model verification method:

[0128] Use accuracy_score to verify the model accuracy in the test set. The accuracy_score calculation formula is as follows:

[0129]

[0130] The ANN flowering period detection model was tested using a flowering period test set constructed from images collected in different years to verify the accuracy and generalization of the model. The accuracy of the ANN flowering period detection model on the flowering period test set was 0.899.

[0131] 3. Online use

[0132] Use a mobile phone to take pictures of tea tree flowers perpendicular to the ground, with a focal length of 0.6 times and a shooting distance of about 26 cm from the outside of the tea row. The image should include the flowers on the side of the tea trees. To ensure that the image accurately reflects the amount of tea tree flowers, collect images evenly at different positions of the tea row, with shots taken every 1-2 m.

[0133] Then, the images and acquisition time collected by the mobile phone were input into the computer, and the size was scaled to 1280×1280. The number of tea buds (bud), open flowers (B flower), and decayed flowers (Wflower) was obtained through the tea flower recognition model (TflosYOLO model), and the number of tea buds (bud) and open flowers (Bflower) with higher accuracy was obtained through the flower quantity linear calibration model;

[0134] The number of withered flowers output by the tea flower recognition model, the number of tea buds and open flowers output by the flower quantity linear calibration model, and the collection time are output by the ANN flowering period detection model to output the flowering period category with the highest probability (beginning of flowering, early flowering period, middle flowering period, late flowering period, and final flowering period).

[0135] experiment:

[0136] 1. Performance comparison of different YOLO models based on the tea tree flower validation set

[0137] The tea flower recognition model (TflosYOLO model) of the present invention is compared with the existing different YOLO models. The experimental data uniformly adopts the tea flower validation set established in step 1.3.1. The experimental results are shown in Table 1, where the accuracy of different models is shown in Table 1. Figure 4 As shown in Figure 5 shown.

[0138] Table 1. Comparative experimental results of tea flower recognition model (TflosYOLO model) and YOLO

[0139]

[0140] Note: The values ​​in the table are the average values ​​of the three types of flowering period recognition effects: buds, open flowers, and withered flowers.

[0141] As can be seen from Table 1, compared with the original YOLO model, the TflosYOLO model of the present invention has a better effect in identifying tea flowers on the basis of smaller computational complexity and model size (between YOLOv5s and YOLOv 5m), and has the highest comprehensive accuracy, recall, and average precision mAP50-95.

[0142] 2. TflosYOLO performance based on the tea tree flower test set:

[0143] The test results of the TflosYOLO model of the present invention on a tea flower test set containing 26 varieties (i.e., the tea flower test set of the TflosYOLO model established in step 1.3.1) are shown in Table 2.

[0144] Table 2. Results of the TflosYOLO model on a test set of 26 tea tree flower varieties

[0145]

[0146]

[0147] As shown in Table 2, the average precision mAP(0.5) of the comprehensive recognition of the three types of flowering stages (bud, blooming flower, withered flower) is 0.874, the precision is 0.802, the recall is 0.854, and the F1 value is 0.827. Among them, the recognition effect of bud is the best, and the recognition effect of withered flower is poor, but the recognition mAP50 of the three types of flowering stages is above 0.82. The test results of the model on the 26 varieties test set show that this model has high accuracy, high robustness and generalization.

[0148] 3. Accuracy of ANN flowering period detection model

[0149] On the flowering period validation set data of the ANN flowering period detection model established in step 2, the accuracy of the ANN flowering period detection model is 0.738, and the confusion matrix results are as follows: Figure 6 As shown. On the flowering period test set data of the ANN flowering period detection model of different years established in step 2, the accuracy of the ANN flowering period detection model is 0.899, and the confusion matrix result of the flowering period test set is as follows Figure 7 The confusion matrix results show that the neural network model has a high accuracy rate in distinguishing the flowering period of tea trees. There is category confusion between adjacent flowering periods, such as the early, middle, and late stages of the peak flowering period. Considering that the flowering period of tea trees is artificially divided and there are many intermediate states that are difficult to clearly divide, a small amount of confusion between adjacent categories is normal.

[0150] Finally, it should be noted that the above examples are only some specific embodiments of the present invention. Obviously, the present invention is not limited to the above embodiments, and there are many variations. All variations that can be directly derived or associated with the content disclosed by a person skilled in the art should be considered as the protection scope of the present invention.

Claims

1. A method for identifying tea tree flowers, estimating flower quantity and flowering period, characterized in that: The method comprises collecting tea tree flower images, then inputting the collected images and the collection time into a computer, obtaining the number of tea tree flower buds, open flowers and decayed flowers through an offline trained tea tree flower recognition model, and then obtaining the number of tea tree flower buds and open flowers with higher accuracy through an offline trained flower quantity linear calibration model; The collection time, the number of decayed flowers output by the tea flower recognition model, and the number of tea buds and open flowers output by the flower quantity linear calibration model are passed through an offline trained ANN flowering period detection model to obtain the flowering period category of the tea flower.

2. The method for identifying tea flowers, estimating flower quantity and estimating flowering period according to claim 1, characterized in that: The tea flower recognition model is based on the YOLOv5 model, and an SE module is added after the third C3 module of the backbone network.

3. The method for identifying tea flowers, estimating flower quantity and estimating flowering period according to claim 2, characterized in that: The formula of the linear calibration model of flower quantity is as follows: Among them, w1 and w2 are weight parameters, x1 is the number of tea tree buds, and x2 is the number of open flowers. To predict the final amount of flowers.

4. The method for identifying tea flowers, estimating flower quantity and estimating flowering period according to claim 3, characterized in that: The ANN flowering period detection model is a multi-layer perceptron, including an input layer, 6 hidden layers and an output layer.

5. The method for tea tree flower identification, flower quantity estimation and flowering period estimation according to claim 4, characterized in that: The offline training process of the tea flower recognition model is as follows: (1) Periodically collect time series images of tea flowers, covering different tea varieties, lighting, backgrounds, and flowering stages; (2) Each image was cropped to a uniform size and randomly divided into a tea flower training set, a tea flower validation set, and a tea flower test set. Then, each sample was subjected to data augmentation and manually labeled with the labels of "tea bud", "open flower", and "decayed flower". (3) During the training process, the loss function is used to calculate various losses, calculate the gradient, and then the model parameters are optimized through back propagation of the SGD optimizer.

6. The method for identifying tea flowers, estimating flower quantity and flowering period according to claim 5, characterized in that: The offline training process of the linear calibration model of flower quantity is as follows: (1) constructing data subsets of the tea flower time series images according to different tea varieties, light and environment, and then using the offline trained tea flower recognition model to detect, outputting the flower quantity data of each data subset, including the number of tea buds and open flowers, using manually observed flower quantity data as labels, and then randomly dividing into a flower quantity calibration training set and a flower quantity calibration verification set; (2) Recollect tea flower images, classify them by tea tree variety, light, and environment, and manually measure the number of tea flowers in each image as the flower quantity calibration test set; (3) The flower quantity calibration training set is input into the flower quantity linear calibration model, and the loss is obtained through the MSE loss function. The SGD optimizer back propagates to continuously update the parameters. After each training cycle (epoch), the performance of the model is evaluated using the flower quantity calibration validation set. After the training, the flower quantity linear calibration model classified by tea tree variety, light and environment is obtained; The flower quantity calibration test set was input into the tea tree flower recognition model to obtain the uncalibrated flower quantity results. Then, the trained flower quantity linear calibration model was used to output the calibrated flower quantity. The calibrated flower quantity was compared with the manually measured flower quantity for R 2 Correlation analysis was performed to further verify the effect of the linear calibration model of flower quantity.

7. The method for identifying tea flowers, estimating flower quantity and estimating flowering period according to claim 6, characterized in that: The offline training process of the ANN flowering period detection model is as follows: (1) detecting the tea tree flower images in the tea tree flower time series images using the offline trained tea tree flower recognition model to obtain the number of tea tree buds, open flowers and decayed flowers corresponding to each image, and then processing each sample, including adding time data, filtering out low-quality data, and averaging every three samples of the same time and variety to generate a new sample; (2) Each new sample is manually labeled with a flowering period category label and divided into a flowering period training set and a flowering period verification set. In addition, tea tree flower images from different years are collected as a flowering period test set. (3) The flowering period training set samples are input into the model for forward propagation. The predicted probability of each flowering period category is output through the softmax function and used together with the flowering period category label to calculate the cross entropy loss. The parameters are updated through back propagation of the Adam optimizer. After each training cycle, the performance of the model is evaluated using the flowering period validation set to obtain the trained ANN flowering period detection model, which is then tested using the flowering period test set to verify the accuracy and generalization of the trained ANN flowering period detection model.

8. The method for identifying tea flowers, estimating flower quantity and estimating flowering period according to claim 7, characterized in that: The method for collecting tea tree flower images is: (1) The shooting angle is perpendicular to the ground; (2) The shooting position is outside the tea row, and the image should include the flowers on the side of the tea tree; (3) Collect images at evenly spaced intervals at different locations along each tea row.

Images