A machine learning based system and method for identifying growth stages of rice

Abstract

The application discloses a kind of identification system and method of rice growth stage based on machine learning;Including rice feature generation module and rice growth stage identification model construction module;Rice feature generation module is used to obtain the fractal dimension of rice according to rice image and the statistics constructed using gray level co-occurrence matrix;Rice growth stage identification model construction module is used to obtain the mapping relationship between the characteristics of rice and the growth stage of rice according to neural network model, so as to construct rice growth stage identification model.The present application introduces the fractal dimension and the gray level co-occurrence matrix for the discrimination of growth cycle, the accuracy of single machine learning model and optimal weighted integrated model is increased by about 2-8% after adding fractal dimension variable, the accuracy of single machine learning model and optimal weighted integrated model is increased by about 2-7% after adding gray level co-occurrence matrix variable, which greatly improves the detection accuracy of rice growth stage.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of rice growth stage monitoring, and particularly relates to a rice growth stage recognition system and method based on machine learning. BACKGROUND

[0002] Rice is one of the most important crops in the world and plays an important role in agricultural production. A full understanding of the growth stage of rice will enable people to take appropriate cultivation and management measures according to the physiological characteristics of rice at different growth stages and their requirements for external conditions, to reasonably meet their growth and development needs, and to use appropriate amounts of water, fertilizers and pesticides, so as to achieve the goal of high and stable yield of rice. At the same time, automatic supervision of different growth stages of crops can make timely and reasonable management decisions at different growth stages of rice, which is the future development direction of agriculture and has important significance for modern farmland management.

[0003] During the entire growth cycle, we can intuitively observe that the external morphological structure of rice has changed significantly, and as the growth stage of rice increases, the self-similarity and scale-free nature of the plant shape are more obvious. In recent years, with the development of agricultural intelligence, computer vision technology has been applied in the detection of rice quality and growth stage. Machine learning and Internet technologies are increasingly applied in agriculture to automatically observe, detect and distinguish different key growth stages of rice, achieve automatic classification and prediction of rice, and thus improve the quantity and quality of rice.

[0004] At present, artificial inspection method is mainly used for recognition, but it is very tedious, time-consuming and laborious, and is often subject to the subjective perception of the observer on the state of rice, and there is a problem that traditional rice contour feature extraction is not easy and not accurate, resulting in inaccurate growth stage judgment. Therefore, it is urgent to study the automatic recognition method of different growth stages of rice to reduce the labor cost, improve the accuracy and real-time of observation, and avoid damage to the plant. However, traditional rice contour feature extraction is not easy and not accurate, thus leading to inaccurate growth stage judgment, and the key growth period stage of rice plant development cannot be identified in time and accurately. SUMMARY

[0005] In order to improve the accuracy of rice growth stage judgment, the present application proposes a rice growth stage recognition system and method based on machine learning.

[0006] The rice growth stage recognition system based on machine learning for achieving one of the purposes of the present application comprises a rice feature generation module and a rice growth stage recognition model construction module.

[0007] The rice feature generation module is used to obtain a plurality of fractal dimensions of rice from a rice image and a plurality of statistics for representing the texture features of rice by using a gray level co-occurrence matrix.

[0008] The rice growth stage recognition model construction module is configured to obtain a mapping relationship between the features of the rice and the growth stage of the rice according to the neural network model, thereby constructing a rice growth stage recognition model; the rice growth stage recognition model is configured to recognize the growth stage of the rice according to the rice image; the features of the rice include a plurality of fractal dimensions of the rice obtained from the rice feature generation module and statistics constructed using a gray level co-occurrence matrix.

[0009] Further, the rice feature generation module includes a first fractal dimension generation module, a second fractal dimension generation module, and a texture feature acquisition module.

[0010] The first fractal dimension generation module is configured to calculate two fractal dimensions D1 and RFD based on the whole rice and the edge of the rice leaf according to the rice image;

[0011] The second fractal dimension generation module is configured to calculate two fractal dimensions D2 and Sandbox based on the whole rice and the circumscribed rectangle according to the rice image;

[0012] The texture feature acquisition module is configured to obtain a gray level co-occurrence matrix according to the correlation of adjacent pixels and the gray level change of the diagonal elements of the co-occurrence matrix, and to construct a plurality of statistics representing the texture features of the rice according to the gray level co-occurrence matrix.

[0013] Further, the statistics include contrast, difference, difference matrix, entropy, correlation, and angular second moment.

[0014] Further, the fractal dimension generation module further includes a gray scale image generation module configured to convert the rice image into a gray scale image before obtaining the fractal dimensions, and the gray scale image is used to obtain the fractal dimensions D1, RFD, D2, and Sandbox of the rice; the fractal dimension generation module further includes a binary image generation module configured to perform edge detection and denoising on the rice image using a Sobel operator and a Gaussian filter to obtain a binary image of the rice image, and the binary image is used to obtain the fractal dimensions D1, RFD, D2, and Sandbox of the rice.

[0015] The calculation method of the fractal dimensions D1 and RFD includes:

[0016] S501, randomly select a pixel point A from the gray scale image of the rice, and record the coordinate value of the pixel point A as (i, j); randomly select a pixel point A' from the binary image of the rice, and record the coordinate value of the pixel point A' as (i', j');

[0017] S502, respectively in the gray image and binary image of rice, the random walk method is used to determine another pixel point B and pixel point B', the coordinate values are respectively recorded as (u, v) and (u', v'); and R = ||(i, j)-(u, v)|| = ||(i', j')-(u', v')||, R is a randomly set value, R = 1, 2,..., n;

[0018] S503, the difference value G of the gray value of A and B two pixel points and the difference value G' of the gray value of A' and B' two pixel points are calculated:

[0019] G = I(i, j)-I(u, v)

[0020] G' = I(i', j')-I(u', v')

[0021] In the formula:

[0022] I(i, j), I(u, v) are the gray values of pixel point A and pixel point B respectively;

[0023] I(i', j'), I(u', v') are the gray values of pixel point A' and pixel point B' respectively;

[0024] S503, repeat steps S501-S502 to obtain a plurality of difference values G and G' of each gray image and binary image, and calculate the average value E(G) of each gray image and the average value E(G') of each binary image according to the difference values G and G';

[0025] S504, two fractal dimensions of each gray image and binary image are calculated according to the following formula:

[0026]

[0027]

[0028] In the formula:

[0029] D1 is the fractal dimension based on the gray image;

[0030] RFD is the fractal dimension based on the binary image;

[0031] C is a constant.

[0032] Further, the calculation method of the fractal dimension D2 and Sandbox further comprises:

[0033] S701, select a MxM grid for division on each gray image and binary image of rice, wherein M is the number of grid boundaries;

[0034] S702, randomly select an (a, b)th grid (a∈[1, M], b∈[1, M]) on each rice gray-scale image; randomly select an (a', b')th grid (a'∈[1, M], b'∈[1, M]) on each rice binary image; the number of frames required to cover each grid is calculated as follows:

[0035]

[0036]

[0037] In the formula:

[0038] n(a, b) represents the number of frames required to cover the (a, b)th grid;

[0039] n(a', b') represents the number of frames required to cover the (a', b')th grid;

[0040] P max represents the maximum value of the pixel values of all pixel points in the (a, b)th grid;

[0041] P min represents the minimum value of the pixel values of all pixel points in the (a, b)th grid;

[0042] P' max represents the maximum value of the pixel values of all pixel points in the (a', b')th grid;

[0043] P' min represents the minimum value of the pixel values of all pixel points in the (a', b')th grid;

[0044] S703, the sum N of the number of frames covered by each grid is calculated as:

[0045]

[0046]

[0047] S704, the two fractal dimensions are calculated according to the following formulae on the gray-scale image and the binary image respectively:

[0048]

[0049]

[0050] In the formula:

[0051] D2 is the fractal dimension based on the gray-scale image;

[0052] Sandbox is the fractal dimension based on the binary image.

[0053] Further, the method for constructing a plurality of statistical quantities for representing the texture features of the rice by using the gray level co-occurrence matrix comprises the following steps:

[0054] S801, dividing each binary image of the rice into L gray levels according to the gray values, each pixel corresponding to a gray level;

[0055] S802, obtaining a gray level co-occurrence matrix p(x, y) according to the gray level of each pixel of each binary image of the rice;

[0056] x, y respectively represent the gray levels of two pixel points, x∈[0, L-1], y∈[0, L-1];

[0057] S803, extracting six texture features from the gray level co-occurrence matrix: contrast, difference, inverse difference moment matrix, entropy, correlation and angular second moment, respectively denoted as Con, DISL, IDM, ENT, Corr and ASM, and the calculation formulae include:

[0058]

[0059]

[0060]

[0061]

[0062]

[0063]

[0064] In the formulae:

[0065] μ x ,μ y : the mean value of the gray levels x and y of two different pixel points;

[0066] σ x ,σ y : the standard deviation of the gray levels x and y of two different pixel points.

[0067] A method for identifying the growth stage of rice based on machine learning to achieve the second object of the present application comprises the following steps:

[0068] S1, obtaining a plurality of fractal dimensions of the rice and a plurality of statistical quantities for representing the texture features of the rice by using the gray level co-occurrence matrix according to the original image of the rice; the fractal dimensions are data characteristic quantities for representing the phenotypic traits of the rice;

[0069] S2, obtaining a mapping relationship between the characteristics of the rice and the growth stage of the rice according to the neural network model, so as to construct a rice growth stage recognition model; the rice growth stage recognition model is used for identifying the growth stage of the rice according to the rice image; the characteristics of the rice include a plurality of fractal dimensions of the rice obtained from the rice characteristic generation module and a plurality of statistics for representing the texture characteristics of the rice constructed by using a gray level co-occurrence matrix.

[0070] Further, the step S1 comprises the following steps:

[0071] calculating two fractal dimensions D1 and RFD based on the whole rice and the edge of the rice leaf according to the rice image;

[0072] calculating two fractal dimensions D2 and Sandbox based on the whole rice and the circumscribed rectangle according to the rice image;

[0073] obtaining a gray level co-occurrence matrix according to the correlation of adjacent pixels and the gray level change of the diagonal elements of the co-occurrence matrix, and constructing a plurality of statistics for representing the texture characteristics of the rice according to the gray level co-occurrence matrix;

[0074] Further, the plurality of statistics include contrast, difference, difference matrix, entropy, correlation and angular second moment;

[0075] Further, the step S2 comprises the following steps:

[0076] S201, combining a plurality of phenotype trait data sets of a plurality of rice samples with a data set containing four fractal dimensions D1, RFD, D2 and Sandbox and six gray level co-occurrence matrices extracted from the gray level image and the binary image, to obtain an initial data set of the rice;

[0077] S202, drawing a heat map of correlation analysis according to all the characteristics of the initial data set of the rice, obtaining the relationship between each characteristic and the growth stage of the rice according to the heat map, and retaining part of the characteristics according to the relationship between the characteristics and the growth stage of the rice;

[0078] S203, combining and cross-validating the remaining characteristics by using a random forest model on the basis of recursive feature elimination, obtaining the importance of the accuracy of different characteristic numbers for the judgment of the growth stage of the rice by calculating the sum of the decision coefficients, and retaining a plurality of characteristic combination numbers to obtain a modeling data set according to the importance;

[0079] S204, performing data preprocessing and normalization processing on the modeling data set and training, to obtain a trained rice growth stage recognition model.

[0080] A non-transitory computer readable storage medium for achieving the third object of the present application, the computer program stored on the non-transitory computer readable storage medium is executed by a processor to achieve the steps of the method for identifying the growth stage of rice based on machine learning.

[0081] Advantages:

[0082] The present application finds that introducing the fractal dimension and the gray level co-occurrence matrix is of positive significance for the discrimination of the growth period, and after adding the fractal dimension variable, the accuracy of the single machine learning model and the optimal weighted integrated model is increased by about 2-8%, and after adding the gray level co-occurrence matrix variable, the accuracy of the single machine learning model and the optimal weighted integrated model is increased by about 2-7%, which greatly improves the detection accuracy of the growth stage of rice. BRIEF DESCRIPTION OF DRAWINGS

[0083] Figure 1 is a flowchart of an embodiment of the method described in the present application;

[0084] Figure 2 is the ROC curve of the six machine learning models in the embodiment;

[0085] Figure 3 is the ROC curve of the optimal weighted integrated model in the embodiment;

[0086] Figure 4 is the confusion matrix of the optimal weighted integrated model in the embodiment;

[0087] Figure 5 is the feature importance result of the input variables of the support vector machine, decision tree, Adaboost and optimal weighted integrated model in the embodiment. DETAILED DESCRIPTION

[0088] The following detailed description is used to explain the technical solutions of the claims of the present application, so that those skilled in the art can understand the claims of the present application. The protection scope of the present application is not limited to the following specific implementation structure. The technical solutions of the claims of the present application which are different from the following specific embodiments and contain the technical solutions of the claims of the present application are also within the protection scope of the present application.

[0089] As Figure 1 shown is a flowchart of an embodiment of the method described in the present application, the machine learning model and integrated model based on the data of the automatic rice phenotype platform proposed by the present application are used to detect the growth stage of rice, and the specific steps are as follows:

[0090] (1) Data extraction

[0091] a. The 28 phenotypic traits of 521 rice varieties, a total of 1094 rice samples, at three different growth stages (tillering stage, jointing stage, and heading stage) were measured by the rice automatic phenotyping platform, as the data set measured by the rice automatic phenotyping platform;

[0092] Table 1 Data characteristics of the rice automatic phenotyping platform

[0093]

[0094]

[0095] b. The rice automatic phenotyping platform uses a visible light industrial camera (AVT Stingray FG504) to take photos of rice, obtaining rice RGB images;

[0096] c. The hierarchical method of the automatic segmentation process based on kernel linear discriminant analysis and Gaussian process regression is used for the rice image, obtaining the gray scale image and the binary image of the rice. Kernel linear discriminant analysis is used to distinguish the target leaf from similar leaves in two steps: one is to roughly detect the entire leaf, and the other is to finely detect the edge of the leaf. The specific method is as follows:

[0097] The first step is to convert the collected rice color image into a gray scale image by using RGB channel gray scale, taking the value of each channel as the gray scale value of the gray scale image, obtaining the gray scale image of the three RGB channels, and drawing the gray scale histogram of the three gray scale images. According to the main information distribution of rice potting, the best rice gray scale image is selected.

[0098] The second step is the kernel linear discriminant analysis based on rough segmentation. A supervised classifier is modeled using kernel linear discriminant analysis to segment the target leaf from similar leaf backgrounds. The target leaf region is cropped and collected from the gray scale image, and the cropped background leaf region is created. After segmenting the target leaf, boundary extraction is performed on the rough segmentation image, and the region containing the target leaf boundary is collected, as well as other regions (such as leaf regions and backgrounds). This method uses the Remove Image Background tool based on Python, Ruby, and deep learning technology, which uses powerful artificial intelligence AI algorithm to automatically identify foreground objects (i.e. target leaves) and backgrounds, thereby realizing large-scale batch image segmentation.

[0099] Third step, after the segmentation by kernel linear discriminant analysis, edge detection is performed on the image. Due to some misclassification of kernel linear discriminant analysis, the edge detected on the boundary region may not be continuous. In order to eliminate these errors, it is necessary to mask the leaf, that is, to cover part of the region with selected objects and perform image processing. Then, the Sobel operator is combined with Gaussian filtering to detect the edge of the picture and remove noise. The basic principle of Gaussian filtering is to weight the average of the image pixels in the sliding window, and the weight coefficient is calculated by using the Gaussian function. The weight value is determined according to the distance between the center pixel point of the sliding window image and other pixel points in the window image, and the weight coefficient will increase with the increase of the distance, otherwise if the distance is smaller, the weight is lower. The formula is:

[0100]

[0101] Where σ 2 represents the variance of the Gaussian function, p and q are the horizontal coordinates, and h(p,q) is the function of the Gaussian filter. At present, the commonly used templates are 3*3 and 5*5, as shown below:

[0102]

[0103] By comparing the results, it is shown that the 3*3 template is the best. Then we change the color of the rice in the gray image to white, and thus obtain the binary image we need.

[0104] d. For the rice gray image and the binary image, the random walk method is used to calculate two fractal dimensions based on the whole rice and the edge of the rice leaf, which specifically includes:

[0105] S501, randomly select a pixel point A and a pixel point A' from the gray image and the binary image of the rice respectively, and the coordinate values are recorded as (i,j) and (i',j') respectively;

[0106] S502, determine another pixel point B and pixel point B' in the gray image and the binary image of the rice respectively by using the random walk method, and the coordinate values are recorded as (u,v) and (u',v') respectively; and R = ||(i,j)-(u,v)|| = ||(i',j')-(u',v')||, R is a randomly set value, R = 1, 2,..., n;

[0107] S503, calculate the difference G between the gray values of the two pixel points A and B and the difference G' between the gray values of the two pixel points A' and B':

[0108] G = I(i,j)-I(u,v)

[0109] G' = I(i',j')-I(u',v')

[0110] In the formula:

[0111] I(i,j) and I(u,v) are the gray values of pixel point A and pixel point B, respectively;

[0112] I(i',j') and I(u',v') are the gray values of pixel point A' and pixel point B', respectively;

[0113] S503, repeat steps S501-S502 to obtain a plurality of difference values G and G' of each gray image and binary image, and calculate the average value E(G) of each gray image and the average value E(G') of each binary image according to the difference values G and G';

[0114] S504, calculate the two fractal dimensions of each gray image and binary image according to the following formula:

[0115]

[0116]

[0117] In the formula:

[0118] D1 is the fractal dimension based on the gray image;

[0119] RFD is the fractal dimension based on the binary image;

[0120] e. For the rice gray image and binary image, the box-counting dimension method is used to calculate two fractal dimensions based on the whole rice and the circumscribed rectangle, specifically including:

[0121] S701, select an MxM grid on each rice gray image and binary image for division, where M is the number of grid boundaries;

[0122] S702, randomly select the (a,b)th grid (a∈[1,M], b∈[1,M]) on each rice gray image; randomly select the (a',b')th grid (a'∈[1,M], b'∈[1,M]) on each rice binary image; the number of frames required to cover each grid is calculated as follows:

[0123]

[0124]

[0125] In the formula:

[0126] n(a,b) represents the number of frames required to cover the (a,b)th grid;

[0127] n(a',b') represents the number of frames required to cover the (a',b')th grid;

[0128] P max : represents the maximum value of pixel values of all pixel points in the (a, b)th grid;

[0129] P min : represents the minimum value of pixel values of all pixel points in the (a, b)th grid;

[0130] P' max : represents the maximum value of pixel values of all pixel points in the (a', b')th grid;

[0131] P' min : represents the minimum value of pixel values of all pixel points in the (a', b')th grid;

[0132] S703, the sum N of the number of covered boxes of each grid is calculated as:

[0133]

[0134]

[0135] S704, the two fractal dimensions are calculated according to the following formula respectively for the gray-scale image and the binary image:

[0136]

[0137]

[0138] In the formula:

[0139] D2 is the fractal dimension based on the gray-scale image;

[0140] Sandbox is the fractal dimension based on the binary image.

[0141] f. For the rice binary image, the correlation between adjacent pixels and the degree of gray level change of the diagonal elements of the co-occurrence matrix are calculated to obtain the statistics of the gray level co-occurrence matrix as the texture features of rice classification. The specific calculation method is as follows:

[0142] According to the gray level of each rice binary image, a gray level co-occurrence matrix p(x, y) is obtained. The specific method for obtaining the gray level co-occurrence matrix p(x, y) is as follows:

[0143] The picture is divided into L levels according to the gray value, and the gray value of each pixel corresponds to a gray level. From a pixel point with an arbitrary gray level x, the probability of reaching a pixel point with a gray level y on a straight line in a direction θ, with a fixed position relationship d=(dx, dy) away from a certain fixed position, all estimated values can be expressed in the form of a matrix, i.e., a gray level co-occurrence matrix; the gray level co-occurrence matrix is represented by p(x, y) (x, y=0, 1, 2...L-1), where L represents the gray level of the image, x and y represent the gray levels of the pixel points, and d represents the spatial position relationship between two pixel points; in this embodiment, d of the gray level co-occurrence matrix is set to 1, and the direction θ is set to ["0", "45", "90", "135"], the values of the four angles are calculated respectively in this embodiment, and then the average value of the four angles is taken; then six texture features are extracted from the gray level co-occurrence matrix: contrast, difference, inverse difference moment matrix, entropy, correlation, and angular second moment (energy), which are denoted as Con, DISL, IDM, ENT, Corr, and ASM respectively, and the specific calculation formulas are as follows:

[0144]

[0145]

[0146]

[0147]

[0148]

[0149]

[0150] wherein μ x ,μ y are the average values of different pixel gray levels x and y, and σ x ,σ y are the standard deviations of different pixel gray levels x and y.

[0151] g. The four fractal dimensions D1, RFD, D2, and Sandbox and the six gray level co-occurrence matrix data are subjected to K-S test, respectively, to determine whether the different stages of rice at each angle are subject to uniform distribution, and the existing autocorrelation is very similar, which indicates that they do not have local specific conditions affected by themselves in growth, and it is indicated that the fractal dimensions of rice and the gray level co-occurrence matrix can be added to the data set as an important feature.

[0152] (2) Feature selection

[0153] a. Combine the 28 phenotype trait data sets of 1094 rice samples measured by the rice automatic phenotype platform with the data sets containing four fractal dimensions and six gray level co-occurrence matrix extracted from the gray scale image and binary image to obtain an initial data set of rice;

[0154] b. Draw a heat map of correlation analysis according to all features of the initial data set of rice, observe the relationship between each feature and the growth stage of rice, and remove features whose relationship with the growth stage of rice is not obvious, in this embodiment, features f1-f12 and features LD1-LD6 shown in Table 1 are removed;

[0155] c. On the basis of recursive feature elimination, a random forest model is used to combine and cross-validate the remaining features, the sum of the decision coefficients is calculated, the importance of the accuracy of different feature numbers for judging the growth stage of rice is finally obtained, and the best feature combination number is retained to obtain a modeling data set; the importance is determined according to actual needs, and in this embodiment, the best feature combination number is a feature combination of 31 features.

[0156] In this step, the recursive feature elimination method is used, which mainly uses a random forest model to train the initial data set of rice multiple times, according to the obtained best feature combination number 31, the features with low weight coefficients are removed according to the weight coefficients after each training, the model is repeatedly constructed, then the best features are selected according to the coefficients, the selected features are extracted, and the remaining features are repeated, until all features are traversed, the best feature combination is selected to obtain a modeling data set, and the rice data set after feature selection using the recursive feature elimination method is the modeling data set.

[0157] (3) Machine learning modeling

[0158] a. Data preprocessing is performed on the modeling data set, including missing value processing, outlier processing and testing whether the data is balanced;

[0159] The testing whether the data is balanced is that the sample number ratio of different label data sets may be unbalanced. Therefore, if the algorithm training is directly used for classification, the training effect may be poor; therefore, the data needs to be tested to make the sample number ratio of the label data set comparable;

[0160] b. Normalization processing is performed on the modeling data set, the purpose is to eliminate the dimension influence of rice features, and the specific calculation formula is as follows:

[0161]

[0162] Wherein, x i represents each feature, represents the mean of the feature, si The standard deviation of this feature

[0163] c. The modeling dataset is divided into a training set and a test set by random sampling at an 8:2 ratio. Each time, 80% of all samples are used for model training, and the remaining 20% ​​are used as the test set to estimate performance metrics. The same random number seed is set for the same model to ensure model consistency.

[0164] d. The established model is trained and validated using the ten-fold cross-validation method. This allows the model to be split into multiple training and test sets for training. The training set is randomly divided into 10 subsets of roughly equal size. One subset is used as the validation set to verify the accuracy of the model, and the other nine subsets are used as the training set to train the model.

[0165] e. The established model is optimized using Bayesian optimization, which employs a Gaussian process, considers previous parameter information, continuously updates priors, and seeks the hyperparameter combination that optimizes the recognition performance of each model classifier.

[0166] f. The initial rice dataset and the rice modeling dataset after feature selection are used to train a machine learning model. The classification labels are "tillering stage", "jointing stage" and "heading stage". A multi-class machine learning model is established. The machine learning classification models used in this embodiment include: support vector machine (SVM), decision tree, random forest, Adaboost, stacked ensemble and optimized weighted ensemble learning classifier.

[0167] The specific calculation method for the stacked integration model is as follows:

[0168] The first layer of the model uses five algorithms—Support Vector Machine, Decision Tree, Random Forest, and AdaBoost—for modeling, fitting, and prediction.

[0169] The second-layer model: This classification model uses the prediction results of the first 5 models as features, the labels of the test set as labels, and the XGBClassifier algorithm as the base classifier for modeling, fitting, and prediction.

[0170] The specific calculation formula for the optimized weighted ensemble model is as follows:

[0171]

[0172]

[0173]

[0174] Among them, w jis the weight corresponding to the base model j (j = 1,..., k), n is the total number of samples, y i is the true value of observation i, is the prediction of observation i by base model j.

[0175] g.The evaluation of the established model performance can be represented by the confusion matrix between the model prediction results and the true results. The confusion matrix function in sklearn.metrics in Python is used to calculate the AUC value and draw the ROC curve to evaluate the classification performance of the model, and the model results are visualized

[0176] h.The evaluation indexes of the model, such as precision, recall, accuracy, F1-score value and kappa coefficient, are obtained by using the precision_score, recall_score, accuracy_score, f1_score, cohen_kappa_score functions in sklearn.metrics in Python, and the specific calculation formulas are as follows:

[0177]

[0178]

[0179]

[0180]

[0181]

[0182]

[0183]

[0184] where TP i is the true positive, indicating the rice correctly classified as the i-th growth stage; FP i is the false positive, indicating the rice incorrectly classified as the i-th growth stage; FN i is the false negative of class i, indicating the rice of the i-th growth stage incorrectly classified as other growth stages. P i and R i are the precision and recall of class i, respectively, n is the number of classes (n = 3 in this study), P w and R w are the precision and recall of weighted F1-score, respectively. p0 is the sum of the number of samples correctly classified in each class divided by the total number of samples, that is, the overall classification accuracy, p eis the sum of the "actual and predicted quantity of the product" of all categories respectively;

[0185] i. the rice modeling data set after the correlation analysis and recursive feature elimination method is used for modeling by multiple machine learning classifiers, and the classification results of the models before and after feature selection are compared according to the evaluation index;

[0186] j. the four fractal dimensions D1, D2, RFD, and Sandbox calculated from the rice grayscale image and binary image, and the six texture features Con, DISL, IDM, ENT, Corr, and ASM obtained by using the gray level co-occurrence matrix are added to the rice modeling data set after feature selection respectively to form new data sets, and then multiple machine learning classifiers are used for modeling, and the classification results of the models with the four fractal dimensions and the six gray level co-occurrence matrix are compared according to the evaluation index; the specific implementation is as follows:

[0187] Example One:

[0188] (1) Read the entire data set of rice and the rice modeling data set after the feature selection respectively, and generate two different data sets: data set 1 is the initial data set of rice, and data set 2 is the modeling data set of rice;

[0189] (2) Set the corresponding classification labels according to the growth stage categories of rice, and the classification labels of the multi-classification model are the tillering stage, the jointing stage, and the heading stage. In each multi-classification model, there are 1094 rice sample data, among which the rice sample data in the tillering stage is labeled as "-1", the rice sample data in the jointing stage is labeled as "0", and the rice sample data in the heading stage is labeled as "1";

[0190] (3) Model the two data sets respectively, use multiple machine learning classifiers such as support vector machine (SVM), decision tree, random forest, Adaboost, stacked ensemble, and optimized weighted ensemble, divide them into training set and test set according to the division ratio of 8:2, and set the same random seed for the same model to ensure the consistency of the model. Use the ten-fold cross-validation method for training and verification, and use the Bayesian optimization method to find the optimal combination of hyperparameters for each model classifier to identify the effect;

[0191] (4) The precision indicators of the six machine learning models in the present embodiment on the entire rice data set are shown in Table 2, and the precision indicators of the rice data set after feature selection are shown in Table 3. It can be seen that the evaluation indicators of the six machine learning classifiers are improved after feature selection, and the best single machine learning model is the Adaboost model, with an accuracy of 93.15% and an F1 score of 0.93, and a Kappa coefficient of 0.91. Compared with the model without feature selection, the accuracy is improved by about 0.5%. Compared with the base model, the ensemble model provides better performance, and the optimized weighted ensemble is the most accurate model, with an accuracy of 94.06% and an F1-score of 0.94, and a Kappa coefficient of 0.92. Compared with the model without feature selection, the accuracy is improved by about 0.6%. Overall, the performance of the machine learning classifier after feature selection is better;

[0192] Table 2 Precision indicators of six models of data set 1

[0193]

[0194] Table 3 Precision indicators of six models of data set 2

[0195]

[0196] (5) The ROC curves of the six machine learning models are shown in Figure 2 , and the ROC curve takes the false positive rate as the horizontal coordinate and the true positive rate as the vertical coordinate. For a data sample of a classification task, the ROC curve needs to calculate the probability that the sample belongs to the correct class. In order to convert the probability into the corresponding class, we need to select a threshold, and the ROC curve is obtained by changing the threshold and shows the performance of each classification model. Due to the high precision of the model, all varieties are closer to the value representing the true positive rate. Compared with the base model, the AUC value of the ensemble model is higher, reaching the level of 0.98, while the AUC values of the single machine learning models based on the random forest model and the Adaboost model also reach about 0.97, indicating that the performance of each classification model is good;

[0197] (6) The ROC curve and confusion matrix of the optimized weighted ensemble model are shown in Figure 3 , 4 , among all the models in the three growth stages, the tillering stage is the most accurate to identify, and it is not easy to confuse, followed by the jointing stage, and finally the heading stage. The reason may be that the stem internodes of rice rapidly elongate upward during the jointing stage, and the top leaves of rice are stretched out with the elongation of the stem, and the overall morphology of rice is not much different.

[0198] Example Two:

[0199] (1) Read the rice modeling data set after the feature selection respectively, and generate three different data sets 3, data set 4 and data set 5: the rice features in data set 3 do not include fractal dimension and gray level co-occurrence matrix; the rice features in data set 4 do not include gray level co-occurrence matrix, including fractal dimension; the rice features in data set 5 do not include fractal dimension, including gray level co-occurrence matrix;

[0200] (2) According to the growth stage category of rice, the corresponding classification label is set, and the classification label of the multi-classification model is tillering stage, jointing stage and heading stage. In each multi-classification model, there are 1094 rice sample data in this embodiment, among which the rice sample data in tillering stage is labeled as "-1"; the rice sample data in jointing stage is labeled as "0"; and the rice sample data in heading stage is labeled as "1";

[0201] (3) Model the three data sets respectively, use six machine learning classifiers including support vector machine (SVM), decision tree, random forest, Adaboost, stacked ensemble and optimized weighted ensemble, divide into training set and test set according to the division ratio of 8:2, and set the same random seed to ensure the consistency of the model. The method of ten-fold cross validation is used for training and verification, and the method of Bayesian optimization is used to find the optimal combination of super parameters of each model classifier to identify the effect;

[0202] (4) The accuracy, F1 score and kappa coefficient indicators of the six kinds of machine learning models in data set 3, 4, 5 are shown in Tables 4, 5 and 6. From the table, it can be found that in different models, on average, when the fractal dimension or gray level co-occurrence matrix feature is introduced, the discrimination effect of the classifier is improved. Overall, after introducing the gray level co-occurrence matrix feature, the evaluation index of each model is improved more, and the accuracy is improved by about 2-7%. After introducing the fractal dimension feature, the decision tree model and the optimized weighted ensemble model are improved more than the model after introducing the gray level co-occurrence matrix feature, and the accuracy is improved by about 3% and 8% respectively. Overall, the single machine learning model improves more, and the introduction of fractal dimension and gray level co-occurrence matrix is of positive significance to the discrimination of growth cycle.

[0203] Table 4 Accuracy indicators of six models of data sets 3, 4 and 5

[0204]

[0205] Table 5 F1 score indicators of six models of data sets 3, 4 and 5

[0206]

[0207] Kappa coefficient index of six models of dataset 3,4,5 of table 6

[0208]

[0209] Example three:

[0210] (1) Read the rice modeling data set after the feature selection respectively, and generate a data set 6, which is the same as data set 2 in example 1.

[0211] (2) According to the setting of the corresponding classification label of rice growth stage category, the classification label of multi-classification model is tillering stage, jointing stage and heading stage respectively. In each multi-classification model, there are 1094 rice sample data, among which the rice sample data in tillering stage is labeled as "-1"; the rice sample data in jointing stage is labeled as "0"; the rice sample data in heading stage is labeled as "1"

[0212] (3) Model the data set 6, use support vector machine (SVM), decision tree, random forest, Adaboost, stacked ensemble and optimized weighted ensemble six machine learning classifiers, respectively adopt random forest method in feature selection tree model to evaluate the importance of features, calculate the importance of each feature to the growth stage category of rice, sort the feature contribution degree, and find the influential feature variables to optimize the model

[0213] (4) The feature importance results of the top 10 input variables are shown in table 7. According to the calculation of input feature importance of six models, the most influential independent variable is found to optimize the model. Table 7 shows the top 10 feature importance results weighted by all 6 models. The weight of parameter SA reaches 0.1359, which is the most important feature. Among the top 10 input variables, there are two texture feature variables extracted from the image, and two fractal dimension variables. Therefore, other auxiliary data derived from the image should be added, because they may lead to higher estimation accuracy. In addition, parameters such as rice relative frequency and structure parameters, which reflect the compactness of the plant, seem to be less important;

[0214] Table 7 Feature importance coefficient and ranking of six models

[0215]

[0216] (5) The feature importance results of support vector machine, decision tree, Adaboost and optimized weighted ensemble model input variables are as follows: Figure 5As shown in the figure, the feature importance of the morphological parameters in the decision tree model is greater, while the feature importance of the texture parameters in the other four models is greater. Therefore, it can be inferred that the texture parameters such as fractal dimension and gray level co-occurrence matrix are very important for the detection of the growth stage of rice. Although the feature importance of different models is not the same, in general, the feature importance of RFD, D2, Sandbox, ENT, ASM and G_g is relatively large in the four models.

[0217] k. According to the machine learning classifier model, the feature importance of the random forest method in the feature selection tree model is evaluated, the importance of each feature for the rice growth stage category is calculated, the feature contribution is sorted, and the influential feature variables are found to optimize the model.

[0218] As can be seen from the embodiments, the optimal single machine learning model for detecting the growth stage of rice is the Adaboost model, with an accuracy of 93.15% and an F1 score of 0.93, and a Kappa coefficient of 0.91. The method for detecting the growth stage of rice according to the present application uses an integrated model, especially an optimized weighted integration after feature selection, which performs the best classification. Compared with the existing single machine learning model, the accuracy is improved by about 1.5%, and the accuracy and F1-score are 94.06% and 0.94, respectively, and the kappa coefficient is 0.92.

[0219] It should be understood that the size of the serial number of each step in the above embodiments does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0220] The present embodiment also provides a computer readable storage medium storing a computer program, the computer program comprising program instructions which, when executed by a processor, implement the steps of the method of the present application, which will not be repeated here.

[0221] The computer readable storage medium can be an internal storage unit of the data transmission device or the computer device provided in any of the preceding embodiments, such as the hard disk or memory of the computer device. The computer readable storage medium can also be an external storage device of the computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc.

[0222] Further, the computer readable storage medium can also include both the internal storage unit of the computer device and the external storage device. The computer readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer readable storage medium can also be used to temporarily store data to be output or data that has been output.

[0223] Those skilled in the art will appreciate that embodiments of the application can be supplied as a method, a system, or a computer program product. Accordingly, the application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer-readable program code.

[0224] The application is described in reference to the flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0225] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0226] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0227] The details of the application described herein are not intended to limit the scope of the application. Rather, the scope of the application is defined by the appended claims.

Claims

1. A machine learning based system for identification of growth stages of rice plants, characterized in that, The method comprises a rice feature generation module and a rice growth stage recognition model construction module. The rice feature generation module is configured to obtain multiple fractal dimensions of the rice and multiple statistics of texture features of the rice based on a rice image. The rice growth stage recognition model construction module is configured to obtain a mapping relationship between features of the rice and a growth stage of the rice based on a neural network model, thereby constructing a rice growth stage recognition model. The rice feature generation module comprises a first fractal dimension generation module, a second fractal dimension generation module, and a texture feature acquisition module. The first fractal dimension generation module is configured to calculate two fractal dimensions D1 and RFD based on the rice as a whole and edges of rice leaves based on a rice image. The second fractal dimension generation module is configured to calculate two fractal dimensions D2 and Sandbox based on the rice as a whole and a circumscribed rectangle based on a rice image. The texture feature acquisition module is configured to obtain a gray level co-occurrence matrix based on correlations of adjacent pixels and gray level changes of diagonal elements of the matrix, and to construct multiple statistics of texture features of the rice based on the matrix. The fractal dimension generation module further comprises a binary image generation module configured to obtain a binary image of the rice by performing edge detection and noise reduction on a gray image of the rice using a Sobel operator and a Gaussian filter. 2.The machine learning based rice growth stage identification system of claim 1, wherein, The statistics include contrast, difference, difference matrix, entropy, correlation, and angular second moment. 3.The machine learning based rice growth stage identification system of claim 1 or 2, wherein, The fractal dimension generation module further comprises a gray image generation module configured to convert the rice image into a gray image before obtaining the fractal dimensions. 4.The machine learning based rice growth stage identification system of claim 1, wherein, The method for calculating the fractal dimensions D1 and RFD comprises the following steps: S501, randomly selecting a pixel point A from a gray image of the rice, and recording the coordinate value of the pixel point A as (i, j); randomly selecting a pixel point A' from a binary image of the rice, and recording the coordinate value of the pixel point A' as (i', j'); S502, determining another pixel point B and a pixel point B' in the gray image and the binary image of the rice respectively using a random walk method, and recording the coordinate values of the pixel point B and the pixel point B' as (u, v) and (u', v') respectively; and R = ||(i, j)-(u, v)|| = ||(i', j')-(u', v')||, R is a randomly set value, R = 1, 2,..., n; S503, calculating a difference G between the gray values of the pixel points A and B and a difference G' between the gray values of the pixel points A' and B': G = I(i, j)-I(u, v) G' = I(i', j')-I(u', v') In the formula: I(i, j) and I(u, v) are the gray values of the pixel points A and B respectively. I(i',j'), I(u',v') are the gray values of the pixel point A' and the pixel point B', respectively; S503, repeat steps S501-S502 to obtain a plurality of difference values G and G' of each gray image and binary image, and calculate the average value E(G) of each gray image and the average value E(G') of each binary image according to the difference values G and G'; S504, calculate the two fractal dimensions of each gray image and binary image according to the following formula: ； ； In the formula: D1 is the fractal dimension based on the gray image; RFD is the fractal dimension based on the binary image; C is a constant. 5.The machine learning based rice growth stage identification system of claim 1, wherein, The calculation method of the fractal dimension D2 and Sandbox includes: S701, select one MxM grid on each rice gray image and binary image for division, wherein M is the number of grid boundaries; S702, randomly select the (a, b)th grid on each rice gray image, a∈[1, M], b∈[1, M]; randomly select the (a', b')th grid on each rice binary image, a'∈[1, M], b'∈[1, M]; the number of frames required to cover each grid is calculated as follows: ； ; In the formula: n(a, b) represents the number of frames required to cover the (a, b)th grid; n(a', b') represents the number of frames required to cover the (a', b')th grid; P max : represents the maximum value of pixel values of all pixel points in the (a, b)th grid; P min : represents the minimum value of pixel values of all pixel points in the (a, b)th grid; P' max : represents the maximum value of pixel values of all pixel points in the (a', b')th grid; P' min : represents the minimum value of pixel values of all pixel points in the (a', b')th grid; S703, calculate the sum N of the number of frames covered by each grid: ; ; S704, calculate the two fractal dimensions according to the following formula: ; ; In the formula: D2 is the fractal dimension based on the gray image; Sandbox is the fractal dimension based on the binary image. 6.The machine learning based rice growth stage identification system of claim 1, wherein, The calculation method of the plurality of statistical quantities for representing the texture features of rice using the gray level co-occurrence matrix includes: S801, divide each rice binary image into L gray levels according to the gray level, and each pixel corresponds to a gray level; S802, obtain a gray level co-occurrence matrix p(x, y) according to the gray level of each pixel of each rice binary image; x and y represent the gray levels of two pixels, x∈[0, L-1], y∈[0, L-1]; S803, extract six texture features from the gray level co-occurrence matrix: contrast, difference, inverse difference moment matrix, entropy, correlation, and angular second moment, respectively denoted as Con, DISL, IDM, ENT, Corr, and ASM, and the calculation formula includes: ; ; ; ； ; ; In the formula: , : mean of gray levels x, y of two different pixels , : Standard deviation of two different pixel gray levels x, y.

7. A method of identifying the growth stage of rice based on the system of claim 1, characterized by, It includes the following steps: S1, obtain a plurality of fractal dimensions of rice and a plurality of statistical quantities for representing the texture features of rice using the gray level co-occurrence matrix from the original image of rice; the fractal dimension is a data feature quantity for representing the phenotype of rice; S2, obtain the mapping relationship between the features of rice and the growth stage of rice according to the neural network model, thereby constructing a rice growth stage recognition model; the rice growth stage recognition model is used to identify the growth stage of rice according to the rice image; the features of rice include a plurality of fractal dimensions of rice and a plurality of statistical quantities for representing the texture features of rice using the gray level co-occurrence matrix obtained from the rice feature generation module.

8. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by a processor, implements the steps of the method for identifying the growth stage of rice based on machine learning according to claim 7.

Images