A method and system for predicting SPAD value based on fresh leaf color identification

Abstract

The application discloses a SPAD value prediction method and system based on fresh leaf color recognition, and the method comprises the following steps: controlling a shooting device to collect fresh leaf real-time samples and uploading the samples; performing sample pretreatment to obtain fresh leaf sample characteristic values; and predicting the SPAD value of the fresh leaf based on a pre-deployed fresh leaf SPAD value prediction model and the sample characteristic values, to obtain the SPAD value prediction of the fresh leaf real-time sample. The application controls the real-time collection and uploading of fresh leaf images on site, and completes sample pretreatment by using image processing technology, and then combines the pre-deployed fresh leaf SPAD value prediction model, so that the low-cost real-time prediction of the SPAD value of the whole fresh leaf in a complex field environment is realized.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of fresh leaf SPAD value prediction, and particularly relates to a SPAD value prediction method and system based on fresh leaf color recognition. BACKGROUND

[0002] Chlorophyll is the basic substance for photosynthesis of green plants, is the main photosynthetic pigment of plant leaves, and is an important index for studying plant growth characteristics, physiological changes and nitrogen nutrition status. The SPAD chlorophyll meter is a portable instrument developed by Japan for measuring the color of crops, and the SPAD value is usually referred to as the leaf color value, has the characteristics of rapidity, convenience and non-destructive monitoring of the object, and is often used to determine the relative content of chlorophyll in living leaves. However, its cost is relatively high and it is difficult to be widely used, and at the same time, the clamping area is small, and the measurement error caused by disease spots or mature spots is large.

[0003] The decrease of chlorophyll content in leaf crops is generally considered to be an important manifestation of nitrogen deficiency or entering the mature and aging period, and the SPAD value objectively reflects the yellow-green degree of the leaf. For example, the mature harvesting of tobacco leaves, numerous studies have found that the SPAD value can quantitatively and accurately reflect the maturity of tobacco leaves. At present, most tobacco farmers in China still rely on sensory judgment based on the appearance characteristics of tobacco leaves, which makes the maturity identification directly linked to the harvesting experience of individual farmers, resulting in uneven quality of tobacco harvesting, which is not conducive to subsequent processing of tobacco leaves. The problems of high purchase cost and small clamping area limit the use of SAPD instrument in tobacco leaves, and at present, a scientific and convenient SPAD value judgment method is urgently needed to realize the objective, accurate and rapid identification of the maturity of fresh tobacco leaves in the field.

[0004] Many studies start from tobacco leaf spectrum analysis, and identify the maturity of tobacco leaves with the help of spectral parameters, but spectral determination needs to rely on special equipment and environment, and farmers cannot use it in the field in real time. With the wide application of machine vision and artificial intelligence technology in crop production, building an intelligent model to identify maturity based on the image features of tobacco leaves has become a research hotspot in the field of tobacco production, but these models cannot be used in the field in real time. Although some research models can be used in the field, these researches only divide the maturity of tobacco leaves according to human experience when collecting data, and the model label is only maturity, so once the data calibration has errors, the practicability of the model will be greatly reduced. In addition, different models of mobile phone cameras have different image optimization algorithms embedded, which limits the application of the model. SUMMARY

[0005] In order to overcome the shortcomings of the prior art, the present application provides a SPAD value prediction method and system based on fresh leaf color recognition, which solves at least one of the above technical problems.

[0006] According to an aspect of the present application, a SPAD value prediction method based on fresh leaf color recognition is provided, comprising:

[0007] The shooting device is controlled to collect fresh leaf real-time samples and upload them;

[0008] Based on the uploaded fresh leaf real-time samples, sample pre-processing is performed to obtain fresh leaf sample characteristic values, and the sample pre-processing includes image target segmentation, image feature extraction, data preprocessing and feature selection;

[0009] Based on the pre-deployed fresh leaf SPAD value prediction model and the sample characteristic values, the SPAD value of the fresh leaf is predicted to obtain the SPAD prediction value of the fresh leaf real-time sample.

[0010] The above technical solution controls the real-time collection and uploading of fresh leaf images, and uses image processing technology to complete sample pre-processing, and then combines the pre-deployed fresh leaf SPAD value prediction model to predict the fresh leaf SPAD value, so as to realize real-time prediction of the fresh leaf SPAD value in the complex field environment through image recognition. Compared with using high-cost SPAD meters to measure the fresh leaf SPAD value, the use cost is reduced.

[0011] As a further technical solution, the image target segmentation further comprises:

[0012] The hue channel grayscale image and the OTSU threshold value algorithm are used for binaryzation segmentation to obtain the main part of the fresh leaf and generate a mask H;

[0013] Based on the threshold values of the channels of the HSV color space, the image target region is segmented to generate a mask I containing the target region;

[0014] Based on the HSV color space, the interference region in the image is extracted to generate a mask E excluded from the target region;

[0015] The complement of the mask E is taken to obtain ~E, and the intersection of the mask H, the mask I and the mask ~E is taken to obtain the final image segmentation mask M.

[0016] As a further technical solution, the image feature extraction further comprises color feature extraction and texture feature extraction, wherein the extracted color features include the channel component values of the RGB and HSV color spaces and the yellowing rate, and the extracted texture features include texture features in four dimensions of energy features, contrast features, correlation features and inverse difference matrix features.

[0017] As a further technical solution, the data preprocessing and feature selection further comprise: data standardization, feature selection and Chinese label numerical encoding operations are performed on the data set to obtain the fresh leaf sample characteristic values, and each group of feature samples contains 20-dimensional image features and SPAD value labels.

[0018] As a further technical solution, the method further comprises: constructing an XGBoost regression model as a fresh leaf SPAD value prediction model; and determining the optimal base evaluator, number of iterations and learning rate of the fresh leaf SPAD value prediction model through parameter tuning.

[0019] According to an aspect of the present application, a SPAD value prediction system based on fresh leaf color recognition is provided, comprising a fresh leaf image acquisition module, a cloud platform and a terminal module; wherein the fresh leaf image acquisition module comprises a portable shooting device for shooting real-time fresh leaf images and uploading them to the cloud platform; the cloud platform is used for sample pre-processing of real-time fresh leaf samples, calling a pre-deployed model for SPAD value prediction, and sending the SPAD prediction value to the terminal module; and the terminal module is used for data display.

[0020] The above technical solution starts from both software and hardware, uses a portable shooting box and a fixed model high-definition camera, designs an image shooting device for the field, and proposes a collection scheme for real fresh leaf images and SPAD values. Then, image processing technology and XGBoost algorithm are used to extract image feature data, build a fresh leaf SPAD value prediction model, and develop a mobile terminal application program to complete data transmission between the shooting device and the model, so as to realize real-time prediction of fresh leaf SPAD values in complex field environments, and assist in judging the maturity of fresh leaves at a low cost.

[0021] As a further technical solution, the sample pre-processing comprises image target segmentation, image feature extraction, data preprocessing and feature selection.

[0022] As a further technical solution, the portable shooting device comprises a shooting device body and a high-definition camera, the shooting device body is selected from a light-tight dark box made of foldable PVC material, a shooting hole is arranged on the upper part of the dark box for mounting a high-definition camera with a communication module of a fixed model, and a direct-current LED scattering light source is mounted on the top and 45° oblique upper side.

[0023] As a further technical solution, the high-definition camera is connected with a mobile terminal, and the high-definition camera collects fresh leaf images through the mobile terminal.

[0024] Compared with the prior art, the present application has the following advantages:

[0025] (1) The present application aims at the problems of expensive and complicated operation of SPAD measuring instruments, real-time use of spectral measurement equipment in the field, field environment interference with fresh leaf image information, and restriction of model use by embedded image optimization algorithm of mobile phone camera, and proposes a SPAD value prediction method based on fresh leaf color recognition. The method first realizes real-time high-quality picture collection in the field through a portable shooting device, and proposes a real-time segmentation method suitable for fresh leaf images in the field, then extracts image target area feature data, uses XGBoost algorithm to build a fresh leaf SPAD value prediction model, and realizes low-cost real-time prediction of fresh leaf SPAD value in the complex environment in the field.

[0026] (2) The present application develops a mobile terminal APP suitable for picking site to complete the transmission of fresh leaf images and prediction results between the shooting device and the cloud model, realizes rapid, objective and accurate prediction of the maturity of fresh leaves in the field, and provides theoretical basis and technical support for accurate picking of crops in the field. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1 It is a method flowchart according to the embodiment of the present application.

[0028] Figure 2 It is an image interference information schematic diagram according to the embodiment of the present application.

[0029] Figure 3 It is an RMIS method processing intermediate result schematic diagram according to the embodiment of the present application.

[0030] Figures 4a-4b It is a schematic diagram of the influence of the base evaluator on the loss function value of the SPAD value prediction model according to the embodiment of the present application.

[0031] Figure 5 It is a schematic diagram of the influence of the learning rate on the mean square error of the SPAD value prediction model according to the embodiment of the present application.

[0032] Figure 6 It is a schematic diagram of the fitting scatter of the SPAD prediction value and the true value according to the embodiment of the present application.

[0033] Figure 7 It is a system schematic diagram according to the embodiment of the present application.

[0034] Figure 8 It is a schematic diagram of a portable shooting device according to the embodiment of the present application. DETAILED DESCRIPTION

[0035] The technical solutions of various embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0036] This invention provides a method for predicting SPAD values ​​based on fresh leaf color recognition, comprising:

[0037] Control the imaging device to collect and upload real-time samples of fresh leaves;

[0038] Based on the uploaded real-time fresh leaf samples, sample preprocessing is performed to obtain the fresh leaf sample feature values. The sample preprocessing includes image target segmentation, image feature extraction, data preprocessing, and feature selection.

[0039] Based on the pre-deployed SPAD value prediction model for fresh leaves and the sample feature values, the SPAD value of fresh leaves is predicted to obtain the SPAD prediction value of real-time fresh leaf samples.

[0040] This invention uses the prediction of SPAD value in fresh tobacco leaves as an example for illustration, such as... Figure 1 As shown, the method mainly consists of two modules: real-time sample preprocessing of fresh leaves and SPAD value prediction. Sample preprocessing is further divided into three parts: image target segmentation, feature extraction, and data preprocessing and feature selection. In the SPAD value prediction module, sample feature values ​​are used to predict the SPAD value, resulting in the predicted SPAD value, NSPAD.

[0041] This invention establishes a standard for classifying the maturity of fresh tobacco leaves based on their appearance characteristics and planting time after transplanting. In addition to appearance characteristics such as leaf color, vein color, and leaf hairs, this invention also introduces the tobacco planting time indicator. Combined with the guidance of professional tobacco harvesters, tobacco leaf data is divided into five grades: underripe, partially ripe, mature, fully ripe, and overripe (M1, M2, M3, M4, M5). The specific standards for maturity classification are shown in Table 1.

[0042] Table 1. Criteria for classifying the maturity of fresh tobacco leaves

[0043]

[0044] There are many uncontrollable factors in the complex field environment, such as light, background and angle, which will interfere with the image feature information, so the application develops a shooting device for field based on a portable shooting dark box and a high-definition camera. The shooting device body selects a foldable PVC material light-tight dark box, which shields external light sources and weather while isolating the cluttered background in the field. A 3cm diameter shooting hole is arranged on the upper part of the dark box, and a fixed type zoom high-definition camera with a communication module is installed to ensure the consistency of the shooting angle. A DC LED diffuse light source is installed on the top and 45° oblique upper side to improve the imaging quality. The length, width and height of the shooting box are all 30cm, and the thickness after folding is not more than 5.5cm, which is convenient to carry and meets the needs of field identification. The specific structure of the shooting device is shown in Figure 8 .

[0045] According to the division standard in table 1, the images of fresh leaves with different maturity degrees are collected in the field.

[0046] XGBoost algorithm is a supervised ensemble machine learning algorithm based on gradient boosting idea. The model constructed is an ensemble model containing multiple tree models (or linear models). Gradient boosting is used for multiple iterations in the model construction process, and a base evaluator is added each time. The base evaluator generated by iteration is trained and fitted to the residual of the established base evaluator. Finally, all base evaluators are integrated into a strong evaluator, that is, the output of all base evaluators is superimposed as the final prediction result of the model. Compared with traditional neural network method, XGBoost algorithm has higher interpretability, prediction accuracy and faster operation speed on small sample structured data set, which meets the requirements of this paper for realizing responsive prediction by using small sample image feature data.

[0047] Taking the base evaluator as a tree model as an example, the integration process of XGBoost algorithm is shown. It is assumed that the given data set has n samples and m features, and the feature vector and real value (label) of the i th sample are represented by x i and y i , respectively. The output of K tree models is superimposed as the prediction result of the final model, which can be represented as

[0048] (1),

[0049] where represents the function space of all tree models in the forest, and each tree model function f k has leaf node number T, and its own tree structure q and corresponding leaf node weight vector ω. In addition, the objective function of supervised learning needs to be defined and minimized in the model training process, and the regularization objective function of XGBoost is defined as

[0050] (2),

[0051] wherein and are loss function and regular term respectively, the loss function is a differentiable convex function, and mean square error, log-likelihood function, etc. can be selected according to the type of the problem to be solved; the regular term is a penalty term of model complexity, which can avoid model overfitting, and the tree structure is made as simple as possible by controlling the number of leaf nodes through γ, and the weight distribution of the leaf node is made smooth by controlling λ.

[0052] Image segmentation refers to extracting low, medium or high-level semantics of an image using digital image processing technology, dividing the image region into mutually disjoint connected parts through specific similarity criteria, and is often used as a pre-processing task for image classification, target detection, etc. The classical image segmentation method mostly uses low-level semantics of the image, such as image color, texture, shape, etc., which includes region growing, clustering, edge detection, thresholding, graph theory, etc. From the perspective of cloud processing and response speed, the operation cost of the classical segmentation method is lower, while the segmentation method based on deep neural network cannot be accepted due to its high algorithm time and space complexity.

[0053] The threshold-based image segmentation method essentially uses the channel value of the digital image pixel, sets a threshold value, and divides the pixels based on this threshold value. The pixels with channel values in the same range are considered to have certain similarity or belong to the same category.

[0054] The simple mathematical description of the threshold segmentation method is as follows. A single-channel grayscale image is represented by a h x w matrix p, where p(i, j) is the grayscale value of the image pixel point (i, j). The pixel points in the input image are divided into two categories, target and non-target, according to the set threshold value th. The change from the input image p to the output image r is

[0055] r ( i , j ) = { 1 , p ( i , j ) ≥ th 0 , p ( i , j ) < th , i ∈ [ 1 , h ], j ∈ [ 1 , w ] (3),

[0056] wherein, represents the image pixels divided into the target region, represents the image pixels divided into the non-target category.

[0057] Through subjective preliminary observation and analysis of the fresh tobacco leaf original image, it is found that the vast majority of images collected in the field using portable shooting devices have good framing and imaging effects. Each tobacco leaf collected in the image is located in the center of the field of view and occupies the main part of the image. The shooting is sufficient and uniform, the leaf color is saturated, the texture is clear and can be identified. Further observation and analysis found that there are some shooting problems in some collected tobacco leaf images, which may interfere with the image target region segmentation and extraction. The main problems are as follows, combined withFigure 2 The analysis is as follows:

[0058] (1) Adhesion of soil. The basal part of some lower leaves adhered to soil, such as Figure 2 indicated by ① in the middle.

[0059] (2) Abnormal spots. Uncontrollable factors during planting resulted in a small number of leaves being mechanically damaged or field-burned, causing black spots on the leaf surface, such as Figure 2 indicated by ② in the middle.

[0060] (3) Background interference. The white inner bottom of the shooting device was included in the background at the edge of the map, such as Figure 2 indicated by ③ in the middle.

[0061] Since the target area of the tobacco leaf and the background have obvious contrast in color richness and lightness in imaging, the simplest and most direct method is to extract the target area of the image based on color threshold segmentation. However, the color of fresh tobacco leaves of different maturity varies from green to yellow, so the color threshold segmentation algorithm is improved in terms of applicability and effectiveness.

[0062] An improved real-time multi-mask intersection segmentation method (RMIS) based on color threshold segmentation is proposed to meet the target segmentation needs of fresh leaf images of different maturity. The main processing flow of this method for tobacco leaf image segmentation is as follows:

[0063] (1) Since the target area of the tobacco leaf image and the background have significant differences in color and lightness, the image is converted to HSV color space representation. In the gray image of the hue H channel, the gray value of the tobacco leaf area is larger, so the H channel gray image and the OTSU threshold algorithm are used for binary segmentation to obtain the main part of the fresh leaf and generate the mask H (Hue mask). The OTSU automatic threshold algorithm is a point-based global threshold algorithm that works well for images with large differences in gray values between targets and non-targets. The algorithm is time-consuming and occupies less memory, which meets the needs of fast processing of fresh leaf images and response results. It should be noted that in digital image processing, the formula for converting from RGB color space to HSV color space is shown in equations (4)-(6), where R, G, and B represent the normalized r, g, and b color channel values, respectively, and R, G, and B ∈ [0, 1];

[0064] (4),

[0065] (5),

[0066] (6);

[0067] (2) The previous step only roughly separates the main part of the tobacco leaf, and the coincidence and accuracy of the separation edge with the real edge are insufficient. The soil and disease spots that may exist in the leaf surface are interference items for subsequent feature extraction. Therefore, the target region of the image is segmented based on the threshold of each channel of the HSV color space. The green, yellow, orange, and red color ranges of the HSV space are used to cover the dynamic color change range of the tobacco leaf during the normal maturation process, and an included mask I is generated which contains the target region;

[0068] (3) The non-target regions such as soil, spots, and white inner bottom are excluded. Several colors such as black, gray, and white in the HSV space are used to extract the interference regions in the image, and an excluded mask E is generated which is excluded from the target region;

[0069] (4) The final mask M used for image segmentation is the intersection of the above three masks. The image masks H, I, and ~E obtained in the previous three steps are intersected, and since the mask E is used to extract the interference region, the complement ~E is taken before intersection. The relationship between the sets is defined as follows:

[0070] (7)。

[0071] For easy and intuitive understanding, the intermediate results of several key steps in the processing of fresh image segmentation using the RMIS method are as follows Figure 3 .

[0072] After the target region of the original fresh leaf image is segmented using the RMIS method, the image features need to be further extracted as structured data for subsequent algorithm processing. The color change of tobacco leaf during harvesting is the main feature representing its maturity, so the color features of the target region are extracted, including the channel component values of the RGB and HSV color spaces. Digital images are stored in the form of RGB component values in the computer, and the conversion formula to the HSV color space is given in formulas (4)-(6). The channel values of the two types of color spaces are not enough to represent the color distribution and range of the flue-cured tobacco image, so the third-order color moments of the image are statistically analyzed based on them to obtain more comprehensive color features. Among them, the first-order moment μ i is the color mean value, representing the overall brightness; the second-order moment σ i is the color standard deviation, representing the size of the color distribution range; and the third-order moment s i is the color deviation, representing the symmetry of the color distribution. Their calculation formulas are

[0073] (8),

[0074] (9),

[0075] (10),

[0076] wherein i represents the i-th component of the color space, p ij represents the component value of the j-th pixel point on the i-th component, and the total number of image pixel points is N.

[0077] The degree of yellowing of the tobacco leaf is also an important reference index for judging maturity, and therefore the "yellowing rate" feature is extracted from the target region of the image. In the HSV color space, the pixel points of the target region are divided into three categories of yellow, orange and other colors, and the number of pixel points in each category is represented by N Y , N O , and N * , respectively, and the yellowing rate p Y is defined as

[0078] (11).

[0079] In addition, the morphological features of the leaf surface, leaf tip and leaf edge of tobacco leaves of different maturity also have obvious differences, and therefore the texture features of the image are calculated using OpenCV to describe the shape change and degree of withering of the tobacco leaf. Based on the gray level co-occurrence matrix, the energy (ENE) features, contrast (CON) features, correlation (CORR) features and inverse difference matrix (HOM) features at angles of 0°, 45°, 90° and 135° are calculated, and the mean value of the calculation results at the four angles is taken to represent the information of different texture features of the image.

[0080] In summary, the present application extracts 19-dimensional color feature data such as the third-order color moments of each component in the RGB and HSV color spaces and the yellowing rate, and 4-dimensional texture feature data such as ENE, CON, CORR and HOM from the target region of the image, totaling 23-dimensional image feature data.

[0081] The present application extracts color feature and texture feature data from the image by means of machine vision technology, and the image is derived from tobacco leaves in the field, and the entire process is basically free of data abnormality phenomena caused by unstable operation of components and current pulses, and therefore data standardization, feature selection and Chinese label numerical encoding operations are performed on the data set.

[0082] The data magnitudes and distributions of different types of image features are different. Directly importing the original data into the model training will make the model spend more computational cost in a huge range of data magnitudes, leading to the problem of gradient explosion. The present application adopts Z-Score standardization method to complete the conversion of different magnitudes of data to the same magnitude and the standardization operation of different distributed data to a specific distribution. The mean of the transformed sample data is 0 and the variance is 1, which facilitates the rapid convergence of the model in subsequent research. The specific formula is

[0083] (12),

[0084] Where x and x * are the sample data before and after standardization respectively, μ is the mean of the original data, and σ is the standard deviation of the original data.

[0085] HSV is the uniform chromaticity system, and the brightness information is contained in the luminance V component. To avoid the interference of brightness on the recognition of color information, the luminance V component and its corresponding color moment features are excluded from the color features, and the remaining 20-dimensional image feature data is selected as the model training data.

[0086] Each sample data in this paper has a true label SPAD value. The label value of SPAD value is a numerical label.

[0087] In summary, after RMIS and sample preprocessing, the data set for model training is constructed. In each feature sample, 20-dimensional image features and SPAD value labels are included.

[0088] The computer used in the experiment is configured as follows: Intel Core i5-6300HQ CPU @ 2.30GHz; NVIDIA GeForce GTX 960M; 8G RAM; Windows 10 Pro 64-bit. The development environment is Python 3.8.10, PyTorch 1.9.1; the learning model adopts the framework py-xgboost 1.5.1 and scikit-learn 1.0.1, and the image processing is realized by OpenCV-Python 4.5.3.56.

[0089] The data set is divided into training set and test set to avoid overfitting. The former trains the model and the latter validates the model performance. The present application randomly selects 20% of the data as the test set and 80% of the data as the training set.

[0090] (1) Fresh tobacco leaf SPAD value prediction

[0091] From the definition of SPAD value, with the maturity of tobacco, SPAD value is a continuous and decreasing variable, so the prediction of fresh tobacco SPAD value is a regression task, and XGBoost regression model should be designed. The input of this regression task is a 20-dimensional image feature, so the input sample i can be represented by a 20-dimensional feature vector x i , where y represents the corresponding true label value i , that is, the SPAD target value corresponding to sample i. The loss function used in model training is mean square error, and its function definition is

[0092] (13),

[0093] where, represents the predicted value of sample i, and n represents the number of samples.

[0094] XGBoost algorithm is based on gradient boosting idea when building the model, which controls the learning ability of the model through the number of iterations, and controls how much the model learns from the previous model through the learning rate, so too large or too small parameter values will affect the fitting effect and learning rate of the model. In addition, the choice of base estimator will also affect the convergence effect of the model. Therefore, through the parameter tuning process, the best base estimator, iteration number and learning rate of the fresh leaf SPAD value prediction model are selected.

[0095] The model training process is shown in Figures 4a-4b . The dashed line represents the change of the training set loss function value, and the solid line represents the change of the test set loss function value. By observing and comparing, when the base estimator is a tree model, the training set loss function value and the test set loss function value decrease faster, and get smaller loss function value in less iteration number, achieving better fitting effect; when the iteration number is 80, the test set loss function value of the model no longer decreases, indicating that the XGBoost model converges, and then tends to be stable, without overfitting problem. Therefore, for the SPAD value prediction model, this paper selects tree model as the base estimator, and the iteration number is 80.

[0096] Figures 4a-4b The influence of base estimator on the loss function value of SPAD value prediction model. Figure 4a In , the base estimator of the model is a linear model. Figure 4b In , the base estimator of the model is a tree model.

[0097] By adjusting the learning rate of the model, the learning rate and fitting effect of the model are further improved. As shown in Figure 5As shown, due to the limitation of the number of iterations, a too small learning rate cannot guarantee the complete fitting of the model to the training set, and the prediction result produces a large mean square error, so the learning rate is set to 0.09, which ensures sufficient learning of the model to the training set and minimizes the prediction error.

[0098] By selecting multiple different evaluation indicators to evaluate the prediction model of the application, the performance of the algorithm is verified, and model integration and deployment are completed to realize real-time prediction in the field.

[0099] (1) Fresh leaf SPAD value prediction model evaluation

[0100] For the SPAD value prediction model, the application selects the mean absolute error (Mean Absolute Error, MAE), root mean squared error (Root Mean Squared Error, RMSE) and goodness of fit (R-squared Score, r 2 ) as evaluation indicators. MAE and RMSE can show the difference between the predicted value and the true value of the model from different aspects, and r 2 can show the fitting degree of the predicted value and the true value, which is defined as

[0101] (15),

[0102] (16),

[0103] (17),

[0104] wherein, and respectively represent the true value and the predicted value of sample i, and n is the number of samples. .

[0105] The SPAD value prediction model is verified on the test set, and the results of each evaluation indicator are shown in Table 2.

[0106] Table 2 Test results of SPAD value prediction model

[0107]

[0108] Among the results of various indicators, MAE is 0.4703, combined with the SPAD value of fresh tobacco leaf of different maturity (the SPAD value of each fresh tobacco leaf collected is measured by SPAD502 chlorophyll meter, the left and right 2 positions of the tip, middle and base of each tobacco leaf are selected for measurement, and the average of 6 measurements is taken as the final SPAD value of the tobacco leaf), the change of fresh tobacco leaf SPAD value in the whole maturity process is 18 units, and the error accounts for only 2.61%; RMSE is slightly larger than MAE, indicating that there are some prediction results in the results that differ slightly from the true value; 2 0.9392, close to 1, indicating that the fitting degree of the predicted value and the true value is high.

[0109] The SPAD prediction value and the true value of all samples in the test set are plotted. As shown in Figure 6 , most of the scatter points are close to the reference line , the linear fitting result of the scatter points is a straight line , and the fitting degree relative to the reference line is high, indicating that the model built has excellent prediction performance for SPAD value.

[0110] The SPAD value prediction model after parameter optimization is integrated according to Figure 1 the steps, and a cloud platform and a database are built, a mobile APP is developed, and model deployment is completed. As shown in Figure 7 , the mobile APP controls the portable shooting device to collect fresh leaf images through a local area network, and uploads the images to the cloud platform database through WiFi or 4G. The cloud platform calls the deployed model to perform fresh leaf real-time sample preprocessing and SPAD value prediction, and saves the extracted tobacco leaf image feature data and model prediction results to the database, only transmits the SPAD value to the mobile APP for data display, providing harvesting reference for tobacco farmers and technicians.

[0111] This paper researches from both software and hardware aspects, and proposes a SPAD value prediction method based on fresh leaf color recognition. The main conclusions are as follows:

[0112] (1) The real-time segmentation method RMIS of multiple masks intersection can quickly divide the target area and various interference information in the tobacco leaf image according to the different color channel values in the HSV space, improve the target area segmentation accuracy, and adapt to the application of real-time harvesting scene in the field. The third moment color feature, texture feature and yellowing rate feature of the target area can provide rich tobacco leaf information for maturity prediction, increase the number of available features, improve the model anti-interference ability, and provide technical support for crop precision harvesting.

[0113] (2) By introducing the time dimension, based on planting time and tobacco appearance to develop scientific maturity classification standard, using the chlorophyll tester to measure SPAD value, to ensure the accuracy and objectivity of sample data calibration.

[0114] (3) The portable shooting device can shield the influence of field environment and weather, fix the shooting background, angle, light and camera parameters, eliminate the possibility of modifying the image by the mobile phone camera, ensure the consistency of sample data, improve the universality of the model. The developed mobile phone APP can control the shooting device, obtain the field tobacco image, and transmit the image to the cloud model for prediction, to ensure the real-time and responsiveness of the model in the field.

[0115] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application, and not to limit it; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solution deviate from the technical solutions of the embodiments of the present application.

Claims

1. A SPAD value prediction method based on fresh leaf color recognition, characterized in that, The method comprises the following steps: controlling the shooting device to collect real-time samples of fresh leaves and upload them; performing sample pretreatment based on the uploaded real-time samples of fresh leaves to obtain characteristic values of the fresh leaf samples, wherein the sample pretreatment comprises image target segmentation, image feature extraction, data preprocessing and feature selection; the image target segmentation further comprises: performing binaryzation segmentation by using a hue channel grayscale image and an OTSU threshold algorithm to obtain a main part of the fresh leaves and generate a mask H; covering the dynamic change range of colors in the normal maturation process of tobacco leaves based on the green, yellow, orange and red color ranges of the HSV color space to generate a mask I contained in the target area; extracting the interference area in the image based on the black, gray and white of the HSV color space to generate a mask E excluded from the target area; taking the complement of the mask E to obtain ~E; and taking the intersection of the mask H, the mask I and the mask ~E to obtain the final image segmentation mask M; predicting the SPAD value of the fresh leaves based on a pre-deployed fresh leaf SPAD value prediction model and the characteristic values of the samples to obtain the SPAD prediction value of the real-time samples of fresh leaves.

2. The SPAD value prediction method based on fresh leaf color recognition according to claim 1, characterized in that, the image feature extraction further comprises color feature extraction and texture feature extraction, wherein the extracted color features include channel component values of the RGB and HSV color spaces and the yellowing rate, and the extracted texture features include texture features in four dimensions of energy features, contrast features, correlation features and inverse difference moment features.

3. The SPAD value prediction method based on fresh leaf color recognition according to claim 2, characterized in that, the data preprocessing and feature selection further comprise: performing data standardization, feature selection and Chinese label numerical encoding operations on the data set to obtain the characteristic values of the fresh leaf samples, and each group of characteristic samples contains 20-dimensional image features and a SPAD value label.

4. The SPAD value prediction method based on fresh leaf color recognition according to claim 1, characterized in that, the training of the fresh leaf SPAD value prediction model comprises: constructing an XGBoost regression model as the fresh leaf SPAD value prediction model; and determining the best base evaluator, the number of iterations and the learning rate of the fresh leaf SPAD value prediction model through parameter tuning.

5. A SPAD value prediction system based on fresh leaf color recognition, characterized by, The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps:

6. The SPAD value prediction system based on fresh leaf color recognition according to claim 5, characterized in that, The portable shooting device comprises a shooting device body and a high-definition camera, the shooting device body is selected from a light-tight dark box made of foldable PVC material, a shooting hole is arranged on the upper part of the dark box for mounting a high-definition camera of a fixed model and having a communication module, and a direct-current LED scattered light source is mounted on the top and 45 DEG oblique upper side.

7. The SPAD value prediction system based on fresh leaf color recognition according to claim 6, characterized in that, The high-definition camera is connected with a mobile terminal, and the mobile terminal is used for controlling the high-definition camera to collect fresh leaf images.

Images