A comprehensive verification method and system for a fresh tobacco leaf quality evaluation model

Abstract

The application discloses a kind of fresh tobacco leaf quality evaluation model comprehensive verification method and system, including steps S1: standard verification sample set is constructed;Step S2: the prediction result of model to be verified is obtained;Step S3: calculate basic accuracy rate index;Step S4: multidimensional cross validation;Step S5: expert consistency test;Step S6: actual curing verification;Step S7: comprehensive accuracy rate assessment;By associating the model prediction result with the actual equal tobacco proportion of cured tobacco leaf, the key quality index such as homogenization level, the guiding value of the model to actual production is verified;In addition, by preset weight calculation comprehensive accuracy rate score, the quantitative evaluation and access standardization of model performance are realized.The method has universality and scalability, and can effectively verify the performance of traditional machine learning models such as XGBoost and random forest, and deep learning models such as ResNet and MobileViT, providing a scientific and comprehensive evaluation standard for intelligent evaluation of fresh tobacco leaf quality.

Description

Technical Field

[0001] This invention relates to the field of tobacco leaf processing technology, specifically to a comprehensive verification method and system for evaluating the quality of fresh tobacco leaves. Background Technology

[0002] Tobacco curing is a crucial processing step in transforming fresh tobacco leaves from the field into raw materials for the cigarette industry. The quality of cured tobacco leaves is influenced by a variety of factors, including the quality of the fresh leaves and the curing process parameters. The quality of fresh tobacco leaves refers to their inherent characteristics, manifested in aspects such as tissue structure, pigment content, enzyme activity, and chemical composition, directly determining the curing characteristics and the overall quality of the cured leaves. Scientifically evaluating the quality of fresh tobacco leaves is a key prerequisite for achieving precise curing and improving the homogenization of the quality of cured tobacco leaves.

[0003] In recent years, with the development of artificial intelligence and machine vision technologies, automatic evaluation methods for fresh tobacco leaf quality based on image recognition and machine learning have gradually attracted attention. For example, some studies have used the ResNet-50 model to predict the maturity level of tobacco leaves, achieving 100% accuracy in predicting 10 categories of tobacco leaves on a test set; others have constructed a BP neural network model based on leaf color skewness parameters to determine the maturity of fresh tobacco leaves; still others have used near-infrared spectroscopy combined with Fisher's discriminant method to identify the maturity of fresh tobacco leaves for cigars. In addition, existing patents have disclosed methods for determining the maturity of fresh tobacco leaves based on information fusion, methods for determining the regreening of tobacco leaves based on dual models, and methods for establishing standards for judging the grade and quality of fresh tobacco leaves.

[0004] However, existing technologies have the following shortcomings:

[0005] First, most existing fresh tobacco quality evaluation technologies focus only on the construction of the model itself and lack systematic methods for verifying model accuracy. Existing verification methods usually only report a single classification accuracy indicator, such as "test set accuracy 100%" or "identification accuracy 95.85%", which is difficult to comprehensively reflect the model's generalization ability and practical application effect under different varieties, different parts, and different maturity conditions.

[0006] Second, the evaluation of fresh tobacco quality involves multiple dimensions (appearance, physicochemical properties, harvest maturity, etc.), and the accuracy of the evaluation model is difficult to verify effectively using a single indicator. Although existing studies have used cross-validation methods to validate the model, these validation dimensions are fragmented and singular, lacking a systematic and comprehensive validation framework.

[0007] Third, existing verification methods do not correlate model predictions with actual curing quality. The ultimate goal of evaluating the quality of fresh tobacco leaves is to guide the selection of tobacco curing processes, but existing technologies cannot verify the guiding value of model predictions for actual curing production, resulting in insufficient reliability of the models in practical applications.

[0008] In conclusion, there is an urgent need for a method and system that can systematically and objectively verify the accuracy of fresh tobacco leaf quality evaluation models. Summary of the Invention

[0009] The purpose of this invention is to provide a comprehensive verification method and system for a fresh tobacco leaf quality evaluation model, so as to solve the problems mentioned in the background art.

[0010] To achieve the above objectives, the present invention provides the following technical solution: a verification method for a fresh tobacco leaf quality evaluation model, comprising the following steps:

[0011] Step S1: Construct a standard verification sample set; collect fresh tobacco leaf samples of different varieties, parts, and maturity levels, measure various indicators according to the preset fresh tobacco leaf quality evaluation index system, and generate a standard verification sample set containing real quality labels.

[0012] Step S2: Obtain the prediction results of the model to be validated; input the fresh tobacco leaf images from the standard validation sample set into the fresh tobacco leaf quality evaluation model to be validated, and obtain the predicted quality category or predicted quality score output by the model.

[0013] Step S3: Calculate the basic accuracy index; compare the prediction results with the true quality labels, and calculate at least one of the following: overall classification accuracy, category average accuracy, root mean square error, and coefficient of determination;

[0014] Step S4: Multi-dimensional cross-validation; Group the samples for validation from multiple dimensions, and use K-fold cross-validation to evaluate the model's generalization ability;

[0015] Step S5: Expert Consistency Test; Compare the model prediction results with the consensus labels of experts in the field of tobacco curing, and calculate the Kappa coefficient to assess consistency;

[0016] Step S6: Actual roasting verification; Based on the model prediction results, match the roasting process to carry out actual roasting, measure the quality of the roasted tobacco leaves, and calculate the correlation coefficient between the predicted quality and the actual quality.

[0017] Step S7: Overall accuracy assessment; Based on the verification results of steps S3 to S6, calculate the overall accuracy score according to preset weights, and determine whether the model passes the verification based on the score.

[0018] Preferably, step S1 specifically includes:

[0019] S1-1: Among the collected fresh tobacco leaf samples, the varieties included Yunyan 87, K326, and Hongda; the parts included upper leaves, middle leaves, and lower leaves; and the maturity levels included underripe, still ripe, moderately ripe, and overripe.

[0020] S1-2: The fresh tobacco leaf quality evaluation index system includes primary indicators and secondary indicators:

[0021] The primary indicators include appearance and morphology indicators, physicochemical and physiological indicators, and harvest maturity indicators.

[0022] The secondary indicators under the appearance and morphology indicators include: color characteristics, leaf length, leaf width, leaf shape index, midrib color, and degree of pubescence loss;

[0023] The secondary indicators under the physicochemical and physiological indicators include: water content, relative chlorophyll content, total sugar content, starch content, protein content, polyphenol oxidase activity, and peroxidase activity.

[0024] The secondary indicators under the harvest maturity index include: the degree of yellowing of leaf color, the degree of whitening of the midrib, and the degree of hair loss;

[0025] S1-3: Associate the multidimensional index data obtained in step S1-2 with the true label value of fresh tobacco leaf quality to form a standard verification sample set labeled with true quality category or true quality score.

[0026] S1-4: Divide the standard validation sample set into a validation set and a test set according to a preset ratio. The validation set is used for parameter fine-tuning during the model validation process, and the test set is used for independent evaluation of the final accuracy. The preset ratio is 7:3 or 8:2.

[0027] Preferably, step S2 specifically includes:

[0028] S2-1: Input the fresh tobacco leaf images from the standard verification sample set into the fresh tobacco leaf quality evaluation model to be verified. The model is based on a machine vision imaging system to obtain the color value and key morphological parameters of the fresh tobacco leaves.

[0029] S2-2: Record the predicted quality category or predicted quality score output by the model to be validated for each validation sample.

[0030] Preferably, the fresh tobacco leaf quality evaluation model to be verified uses at least one of the following methods for quality identification and classification:

[0031] Method 1: Extract key feature points from fresh tobacco leaf images based on the SIFT feature extraction algorithm, combine them with the CCA feature dimensionality reduction method, and then use the XGBoost classification model for classification.

[0032] Method 2: Classification or detection based on deep learning neural network models, including ResNet series networks, VGG series networks, MobileViT network, Faster-RCNN algorithm, YOLO algorithm, and SSD algorithm;

[0033] Method 3: Classification based on a random forest classification model;

[0034] Method 4: Classification based on Fisher's linear discriminant function model.

[0035] Preferably, step S3 specifically includes:

[0036] S3-1: Compare the model prediction results obtained in step S2 with the true label values ​​of the standard validation sample set obtained in step S1 on a sample-by-sample basis.

[0037] S3-2: Calculate the following basic accuracy metrics respectively:

[0038] Overall classification accuracy (OA):

[0039] in, To predict the number of samples whose class matches the true class, To verify the total number of samples;

[0040] Average Accuracy (AA) for each category:

[0041] in, The total number of quality categories, Let be the number of samples in class i that were correctly predicted. This represents the number of samples in class i that were incorrectly predicted as other classes.

[0042] Root mean square error (RMSE) of quality score prediction:

[0043] in, Predict the quality score for the j-th sample using the model. This represents the true quality score of the j-th sample.

[0044] Coefficient of determination (R²) for predicting quality scores:

[0045] in, This represents the average of the true quality scores of all validation samples.

[0046] Preferably, step S4 specifically includes:

[0047] S4-1: Variety-based cross-validation: Group the validation samples by flue-cured tobacco variety, calculate the model accuracy for each variety, and evaluate the model's generalization ability among different varieties.

[0048] S4-2: Part-based cross-validation: Group the validation samples according to tobacco leaf parts, calculate the model accuracy for each part, and evaluate the model's generalization ability across different parts.

[0049] S4-3: Cross-validation of maturity dimension: Group the validation samples according to the harvest maturity level, calculate the model accuracy for each maturity level, and evaluate the model's ability to distinguish between different maturity levels;

[0050] S4-4: K-fold cross-validation: The standard validation sample set is randomly divided into K subsets of similar size. One subset is taken as the test set and the remaining K-1 subsets are taken as the training set for K independent validations. The average of the K validation accuracies is taken as the final accuracy evaluation result.

[0051] Preferably, step S5 specifically includes:

[0052] S5-1: Select at least 3 experts in the field of tobacco curing, and have the experts independently evaluate the quality of fresh tobacco samples in the verification sample set. The majority consensus of the experts' evaluation results shall be used as the expert consensus label.

[0053] S5-2: Compare the model prediction results with the expert consensus labels, and calculate the consistency coefficient (Kappa coefficient) between the model predictions and the expert consensus:

[0054] in, To observe the consistency rate, Expected consistency rate;

[0055] S5-3: When the Kappa coefficient is ≥0.80, the prediction results of the judgment model are highly consistent with the expert consensus.

[0056] Preferably, step S6 specifically includes:

[0057] S6-1: Based on the evaluation results of the fresh tobacco leaf quality of the verification sample set in step S1, match the corresponding baking process in the preset tobacco leaf precision baking process curve database.

[0058] S6-2: Perform actual baking on the verification samples according to the matched baking process;

[0059] S6-3: Determine the quality indicators of flue-cured tobacco leaves, including: the proportion of high-grade tobacco, the level of homogenization, and the curing loss rate;

[0060] S6-4: Establish a correlation model between the predicted results of the fresh tobacco leaf quality evaluation model and the actual quality of the cured tobacco leaves, and calculate the correlation coefficient between the predicted quality category and the actual cured quality:

[0061] in, Encode the quality category predicted by the model for the j-th sample. The actual baking quality score for the j-th sample;

[0062] S6-5: When the correlation coefficient r ≥ 0.85, it is determined that the model prediction results are strongly correlated with the actual baking quality, thus verifying the effectiveness of the model in practical applications.

[0063] Preferably, in step S7, based on the verification results of steps S3 to S6, a comprehensive accuracy score is calculated according to the corresponding weight allocation; when the comprehensive accuracy score is ≥85 points, the fresh tobacco leaf quality evaluation model to be verified is determined to have passed the verification and can be used for quality evaluation and process matching in tobacco leaf curing production.

[0064] A validation system for a fresh tobacco leaf quality evaluation model, comprising:

[0065] The sample collection module is used to collect fresh tobacco leaf samples of different varieties, parts, and maturity levels, and to record the variety information, part information, and harvest maturity level of the samples.

[0066] The index measurement module is used to measure the appearance and morphology, physicochemical and physiological properties, and harvest maturity of the samples according to the preset fresh tobacco leaf quality evaluation index system.

[0067] The standard validation sample set construction module is used to correlate the measurement results with the true competency label values ​​to form a standard validation sample set.

[0068] The model prediction module is used to input fresh tobacco leaf images into the model to be validated and obtain the predicted quality category or predicted quality score.

[0069] The basic accuracy calculation module is used to calculate the overall classification accuracy, the average accuracy of each category, the root mean square error of the quality score prediction, and the coefficient of determination.

[0070] The cross-validation module is used to perform grouped validation and K-fold cross-validation based on variety, part, and maturity dimensions.

[0071] The consistency verification module is used to calculate the Kappa coefficient between the model prediction results and the expert consensus labels;

[0072] The actual roasting verification module is used to match and execute the roasting process, determine the quality of the roasted tobacco leaves, and calculate the correlation coefficient between the predicted results and the actual quality.

[0073] The comprehensive evaluation module is used to calculate the overall accuracy score and determine whether the model passes validation based on the score.

[0074] Compared with the prior art, the beneficial effects of the present invention are:

[0075] This invention establishes a quality evaluation index system for fresh tobacco leaves covering three dimensions: appearance, physicochemical morphology, and harvest maturity. It realizes multi-dimensional quantitative characterization of fresh tobacco leaf quality and provides standardized real labels for model verification.

[0076] This invention proposes a multi-dimensional and multi-level model accuracy verification method, covering multiple dimensions such as overall accuracy, category average accuracy, score prediction accuracy, cross-validation, expert consistency test and actual baking verification. It overcomes the shortcomings of existing technologies with single verification dimensions and realizes a comprehensive evaluation of the performance of fresh tobacco quality evaluation models.

[0077] This invention correlates model validation results with actual baking quality, establishing a closed-loop validation system of "evaluation-validation-baking" to ensure that the model validation results have practical application value and solves the technical problem that existing technologies cannot verify the guiding value of model prediction results for actual baking production.

[0078] This invention achieves quantitative evaluation of model accuracy through a preset weight allocation and comprehensive scoring mechanism, providing an objective standard for comparing the merits of fresh tobacco quality evaluation models and for practical application access.

[0079] The verification method of this invention is applicable to various types of fresh tobacco leaf quality evaluation models, including traditional machine learning models (such as XGBoost, Random Forest, SVM, etc.) and deep learning models (such as ResNet, VGG, MobileViT, etc.), and has good versatility and scalability. Attached Figure Description

[0080] Figure 1 This is a schematic diagram of the overall process for verifying the fresh tobacco leaf quality evaluation model of the present invention;

[0081] Figure 2 This is a schematic diagram of the structure of the fresh tobacco leaf quality evaluation index system of the present invention;

[0082] Figure 3 This is a flowchart illustrating the multi-dimensional cross-validation process of the present invention;

[0083] Figure 4 This is a structural block diagram of the verification system for the fresh tobacco leaf quality evaluation model of the present invention. Detailed Implementation

[0084] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0085] Please see Figures 1-4 This invention provides a technical solution: a verification method for a fresh tobacco leaf quality evaluation model, comprising the following steps:

[0086] Step S1: Construct a standard validation sample set

[0087] S1-1: Collect fresh tobacco leaf samples of different varieties, parts, and maturity levels. Record the variety information, part information, and harvest maturity level of each sample. The varieties include, but are not limited to, Yunyan 87, K326, and Hongda. The parts include upper leaves, middle leaves, and lower leaves. The maturity level includes underripe, slightly ripe, moderately ripe, and overripe.

[0088] S1-2: For each fresh tobacco leaf sample, conduct comprehensive index measurements according to the preset fresh tobacco leaf quality evaluation index system to obtain the true label value of fresh tobacco leaf quality for each sample.

[0089] The evaluation index system for the quality of fresh tobacco leaves includes primary indicators and secondary indicators:

[0090] The primary indicators include: appearance and morphology indicators, physicochemical and physiological indicators, and harvest maturity indicators;

[0091] The secondary indicators under appearance and morphology include: color characteristics (RGB color space parameters and / or HSV color space parameters), leaf length, leaf width, leaf shape index, midrib color, and degree of pubescence loss;

[0092] Secondary indicators under physicochemical and physiological indicators include: water content, relative chlorophyll content (SPAD value), total sugar content, starch content, protein content, polyphenol oxidase (PPO) activity, and peroxidase (POD) activity;

[0093] The secondary indicators under the harvest maturity index include: the degree of yellowing of leaf color, the degree of whitening of the midrib, and the degree of loss of pubescence.

[0094] S1-3: Associate the multidimensional index data obtained in step S1-2 with the true label value of fresh tobacco leaf quality to form a standard verification sample set labeled with true quality category or true quality score.

[0095] S1-4: Divide the standard validation sample set into a validation set and a test set according to a preset ratio. The validation set is used for parameter fine-tuning during the model validation process, and the test set is used for independent evaluation of the final accuracy. The preset ratio is 7:3 or 8:2.

[0096] Step S2: Obtain the prediction results of the model to be validated.

[0097] S2-1: Input the fresh tobacco leaf images from the standard verification sample set into the fresh tobacco leaf quality evaluation model to be verified. The model is based on a machine vision imaging system to obtain key parameters such as the color value and morphology of the fresh tobacco leaves.

[0098] S2-2: The fresh tobacco leaf quality evaluation model uses at least one of the following methods for quality identification and classification:

[0099] Method 1: Extract key feature points from fresh tobacco leaf images based on SIFT (Scale Invariant Feature Transform) feature extraction algorithm, combine CCA (Canonical Correlation Analysis) feature dimensionality reduction method for feature reduction, and use XGBoost classification model for fresh tobacco leaf quality classification;

[0100] Method 2: Classify the quality of fresh tobacco leaves based on deep learning neural network models, including but not limited to ResNet series networks, VGG series networks, MobileViT network, Faster-RCNN object detection algorithm, YOLO object detection algorithm, and SSD object detection algorithm;

[0101] Method 3: Classifying the quality of fresh tobacco leaves based on a random forest classification model;

[0102] Method 4: Classify the quality of fresh tobacco leaves based on the Fisher linear discriminant function model.

[0103] S2-3: Record the predicted quality category or predicted quality score output by the model to be validated for each validation sample.

[0104] Step S3: Calculate the baseline accuracy index

[0105] S3-1: Compare the model prediction results obtained in step S2 with the true label values ​​of the standard validation sample set obtained in step S1 on a sample-by-sample basis.

[0106] S3-2: Calculate the following basic accuracy metrics respectively:

[0107] Overall classification accuracy (OA):

[0108] in, To predict the number of samples whose class matches the true class, To verify the total number of samples;

[0109] Average Accuracy (AA) for each category:

[0110] in, The total number of quality categories, Let be the number of samples in class i that were correctly predicted. This represents the number of samples in class i that were incorrectly predicted as other classes.

[0111] Root mean square error (RMSE) of quality score prediction:

[0112] in, Predict the quality score for the j-th sample using the model. This represents the true quality score of the j-th sample.

[0113] Coefficient of determination (R²) for predicting quality scores:

[0114] in, This represents the average of the true quality scores of all validation samples.

[0115] Step S4: Multi-dimensional cross-validation

[0116] S4-1: Variety-based cross-validation: Group the validation samples by flue-cured tobacco variety, calculate the model accuracy for each variety, and evaluate the model's generalization ability among different varieties.

[0117] S4-2: Part-based cross-validation: Group the validation samples according to tobacco leaf parts, calculate the model accuracy for each part, and evaluate the model's generalization ability across different parts.

[0118] S4-3: Cross-validation of maturity dimension: Group the validation samples according to the harvest maturity level, calculate the model accuracy for each maturity level, and evaluate the model's ability to distinguish between different maturity levels;

[0119] S4-4: K-fold cross-validation: The standard validation sample set is randomly divided into K subsets of similar size. One subset is selected as the test set and the remaining K-1 subsets are selected as the training set for K independent validations. The average of the K validation accuracies is taken as the final accuracy evaluation result. The value of K is in the range of 5 to 10.

[0120] Step S5: Expert Consistency Check

[0121] S5-1: Select at least 3 experts in the field of tobacco curing, and have the experts independently evaluate the quality of fresh tobacco samples in the verification sample set. The majority consensus of the experts' evaluation results shall be used as the expert consensus label.

[0122] S5-2: Compare the model prediction results with the expert consensus labels, and calculate the consistency coefficient (Kappa coefficient) between the model predictions and the expert consensus:

[0123] in, To observe the consistency rate, Expected consistency rate;

[0124] S5-3: When the Kappa coefficient is ≥0.80, the prediction results of the judgment model are highly consistent with the expert consensus.

[0125] Step S6: Actual baking verification

[0126] S6-1: Based on the evaluation results of the fresh tobacco leaf quality of the verification sample set in step S1, match the corresponding baking process in the preset tobacco leaf precision baking process curve database.

[0127] S6-2: Perform actual baking on the verification samples according to the matched baking process;

[0128] S6-3: Determine the quality indicators of flue-cured tobacco leaves, including: the proportion of high-grade tobacco, the level of homogenization (characterized by the standard deviation of the main quality indicators), and the curing loss rate.

[0129] S6-4: Establish a correlation model between the predicted results of the fresh tobacco leaf quality evaluation model and the actual quality of the cured tobacco leaves, and calculate the correlation coefficient between the predicted quality category and the actual cured quality:

[0130] in, Encode the quality category predicted by the model for the j-th sample. The actual baking quality score for the j-th sample;

[0131] S6-5: When the correlation coefficient r ≥ 0.85, it is determined that the model prediction results are strongly correlated with the actual baking quality, thus verifying the effectiveness of the model in practical applications.

[0132] Step S7: Overall Accuracy Assessment

[0133] S7-1: Based on the verification results of steps S3 to S6, calculate the overall accuracy score according to the following weighting:

[0134] Overall classification accuracy (OA) 20% Category Average Precision (AA) 15% Predicted RMSE based on competence score (converted to a percentage scale: 100 - RMSE) 10% Quality score prediction R² (converted to a percentage: R²×100) 10% Inter-varietal generalization ability (average OA of each variety) 5% Inter-part generalization ability (average OA of each part) 5% Maturity discrimination ability (average of OA at each maturity level) 5% Expert consensus (Kappa coefficient × 100) 15% Correlation of actual baking quality (r×100) 15%

[0135] S7-2: When the overall accuracy score is ≥85, the fresh tobacco leaf quality evaluation model to be verified is deemed to have passed the verification and can be used for the intelligent matching of fresh tobacco leaf quality evaluation and curing process in actual tobacco curing production.

[0136] A validation system for a fresh tobacco leaf quality evaluation model, comprising:

[0137] The sample collection module is used to collect fresh tobacco leaf samples of different varieties, parts, and maturity levels.

[0138] The index measurement module is used to measure all the indexes of the fresh tobacco leaf quality evaluation index system for each fresh tobacco leaf sample, including the appearance and morphology index measurement unit, the physicochemical and physiological index measurement unit, and the harvest maturity index measurement unit.

[0139] The standard validation sample set construction module is used to associate the indicator measurement results with the real quality label values ​​to form a standard validation sample set.

[0140] The model prediction module is used to input fresh tobacco leaf images into the fresh tobacco leaf quality evaluation model to be validated, and obtain the predicted quality category or predicted quality score output by the model.

[0141] The basic accuracy calculation module is used to calculate the overall classification accuracy, the average accuracy of each category, the RMSE of the quality score prediction, and the R² of the quality score prediction.

[0142] The cross-validation module is used to perform cross-validation based on variety, part of body, maturity, and K-fold cross-validation.

[0143] The consensus verification module is used to perform expert consensus verification and calculate the Kappa coefficient between the model prediction and the expert consensus.

[0144] The actual roasting verification module is used to match the roasting process with the model prediction results and carry out actual roasting, measure the quality indicators of the roasted tobacco leaves, and calculate the correlation coefficient between the prediction results and the actual quality.

[0145] The comprehensive evaluation module is used to calculate the comprehensive accuracy score based on the verification results of each dimension according to preset weights, and to determine whether the model passes the verification.

[0146] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a verification method for the fresh tobacco leaf quality evaluation model as described in any of the preceding claims.

[0147] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a verification method for a fresh tobacco leaf quality evaluation model as described in any of the preceding claims.

[0148] Example 1: Validation of Fresh Tobacco Leaf Quality Evaluation Based on SIFT+CCA+XGBoost Model

[0149] 1. Construct a standard validation sample set

[0150] A total of 3,000 fresh tobacco leaf samples were collected from tobacco-growing areas in Qujing City, Yunnan Province, during the same growing season.

[0151] Variety distribution: Yunyan 87 (1200 tablets), K326 (1000 tablets), Hongda (800 tablets).

[0152] Distribution of parts: lower leaves (1000), middle leaves (1000), upper leaves (1000).

[0153] Maturity distribution: underripe (600 pieces), moderately ripe (800 pieces), moderately ripe (900 pieces), overripe (700 pieces).

[0154] For each tobacco leaf, the true label value was obtained using measuring equipment:

[0155] Appearance and morphology: RGB and HSV color parameters were obtained using a hyperspectral imager, the length and width were measured using digital calipers to calculate the leaf shape index, and the color of the midrib and the degree of trichome loss were observed manually.

[0156] Physicochemical and physiological methods: Moisture content was determined by drying method, chlorophyll content was determined by SPAD instrument, starch content was determined by anthrone colorimetric method, and PPO and POD activities were determined by ultraviolet spectrophotometry.

[0157] Maturity: The maturity level was determined by three senior baking technicians according to the YC / T311-2009 standard.

[0158] The aforementioned multidimensional data was correlated with the ground truth labels to form a standard dataset. 2400 samples were randomly selected at an 8:2 ratio as the validation set, and 600 samples as the test set.

[0159] 2. Obtain the prediction results of the model to be validated.

[0160] Images of fresh tobacco leaves from the validation set are input into the model to be validated. This model uses Method 1:

[0161] Extracting scale-invariant feature points from images using the SIFT algorithm;

[0162] The extracted feature vectors are dimensionality reduced using CCA (canonical correlation analysis) to remove redundant information;

[0163] The dimensionality-reduced features are input into the trained XGBoost classification model, which outputs the predicted quality category (e.g., "mature") and the corresponding predicted quality score for each tobacco leaf.

[0164] 3. Calculate the basic accuracy index

[0165] Compare the model output with the true labels on the validation set:

[0166] According to statistics, N correct For 2160, N total It is 2400.

[0167] Overall classification accuracy (OA) = 2160 / 2400 × 100% = 90.0%.

[0168] Category average accuracy (AA): The accuracy rates for under-ripe, moderately ripe, moderately ripe, and over-ripe categories were 88.3%, 89.5%, 92.1%, and 87.6%, respectively, with an average AA of 89.375%.

[0169] RMSE and R²: The RMSE for the quality score prediction is 0.12 and the R² is 0.91, indicating a high model fit.

[0170] 4. Multi-dimensional cross-validation

[0171] Variety-based accuracy: Yunyan 87 achieved 89.2% accuracy, K326 achieved 91.5% accuracy, and Hongda achieved 88.7% accuracy. The accuracy of the Hongda variety was slightly lower due to its unique leaf characteristics; the model still needs further optimization for these specific varieties.

[0172] Location dimension: lower leaves 87.5%, middle leaves 92.0%, upper leaves 89.8%. The middle leaves have obvious features and the highest recognition rate.

[0173] Maturity dimension: The recognition rate of properly mature tobacco leaves was the highest (92.1%), while the recognition rate of over-mature tobacco leaves was 87.6% because they were easily confused with abnormal aging.

[0174] K-fold cross-validation: With K=10, 10 cross-validations were performed. The average accuracy was 89.8%, and the standard deviation was less than 1%, indicating that the model performance was stable.

[0175] 5. Expert Consistency Check

[0176] Five tobacco processing technicians certified by the State Tobacco Monopoly Administration were invited to conduct blind evaluations of 2,400 samples. The majority opinion of the experts (≥3 consistent opinions) was defined as the expert consensus label.

[0177] Calculate the observed consistency rate p0 = 0.92, and the expected consistency rate p e =0.28.

[0178] The Kappa coefficient κ = (0.92 − 0.28) / (1 − 0.28) = 0.89.

[0179] Since κ>0.80, the prediction results of the judgment model are highly consistent with the expert consensus.

[0180] 6. Actual baking verification

[0181] 300 middle leaves (Yunyan 87) that were determined to be "suitable for ripening" by the model were randomly selected from the validation set:

[0182] Based on the quality score output by the model, the "medium temperature, medium humidity, slow yellowing" process curve is matched in the baking process database.

[0183] This process is used for intensive baking in a drying oven.

[0184] Post-curing tests showed that the proportion of high-quality tobacco reached 78.5%, the coefficient of variation (homogenization level) of major quality indicators such as single leaf weight and stem content was 8.5%, and the curing loss rate (including ash residue and steaming) was 7.2%.

[0185] The correlation coefficient r between the predicted quality category code and the actual baking quality score was calculated to be 0.89.

[0186] Since r > 0.85, it is verified that the model prediction results are strongly correlated with the actual baking quality.

[0187] 7. Overall Accuracy Assessment

[0188] Weighting criteria: Base accuracy 30%, cross-validation 20%, expert consensus 20%, actual baking validation 30%.

[0189] Overall score = 90×0.3 + 89.8×0.2 + 89×0.2 + 89×0.3 = 89.56 points.

[0190] Since 89.56 points is greater than or equal to 85 points, the SIFT+CCA+XGBoost fresh tobacco leaf quality evaluation model is deemed to have passed the verification and can be put into actual production.

[0191] Example 2: Validation based on the MobileViT deep learning model

[0192] 1. Construct a standard validation sample set

[0193] Same as in Example 1, maintain data consistency for comparison.

[0194] 2. Obtain the prediction results of the model to be validated.

[0195] Method two is used, loading a pre-trained MobileViT model. Fresh tobacco leaf images are input into the model, which automatically extracts multi-scale features and outputs predicted quality categories.

[0196] 3. Calculate the basic accuracy index

[0197] The overall classification accuracy (OA) reached 93.5%.

[0198] RMSE and R²: RMSE is 0.09, R² 2 It is 0.94.

[0199] 4. Multi-dimensional cross-validation

[0200] Variety dimension: The accuracy of the Hongda variety was improved to 91.2% (significantly improved compared to Example 1), indicating that the deep learning model has a stronger ability to learn morphological features.

[0201] K-fold cross-validation: The average accuracy of 10-fold cross-validation is 92.9%.

[0202] 5. Expert Consistency Check

[0203] The calculated Kappa coefficient is 0.92, indicating extremely high consistency.

[0204] 6. Actual baking verification

[0205] The correlation coefficient r = 0.91.

[0206] The proportion of premium cigarettes increased to 82.0%.

[0207] 7. Overall Accuracy Assessment

[0208] Overall score = 93.5×0.3 + 92.9×0.2 + 92×0.2 + 91×0.3 = 92.43 points.

[0209] The MobileViT model was validated and found to outperform traditional machine learning models in complex contexts.

[0210] Example 3: Validation based on the random forest model

[0211] 1. Construct a standard validation sample set

[0212] Same as Example 1.

[0213] 2. Obtain the prediction results of the model to be validated.

[0214] Method three was adopted, using a random forest classification model. The model input consisted of manually extracted color features, texture features, and physical size features.

[0215] 3. Calculate the basic accuracy index

[0216] The overall classification accuracy (OA) was 87.5%.

[0217] RMSE and R²: RMSE is 0.15, R² 2 It is 0.86.

[0218] 4. Multi-dimensional cross-validation

[0219] It performed well on the K326 variety (89.0%), but only moderately on the Hongda variety (84.5%).

[0220] 5. Expert Consistency Check

[0221] The Kappa coefficient is 0.81, barely meeting the standard of high consistency.

[0222] 6. Actual baking verification

[0223] The correlation coefficient r = 0.86.

[0224] The proportion of high-quality cigarettes was 75.0%.

[0225] 7. Overall Accuracy Assessment

[0226] The overall score is 86.2.

[0227] The random forest model has been validated and is suitable for edge computing scenarios with high real-time requirements and limited hardware costs.

[0228] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for validating a fresh tobacco leaf quality evaluation model, characterized in that, Includes the following steps: Step S1: Construct a standard verification sample set; collect different fresh tobacco leaf samples, measure various indicators according to the preset fresh tobacco leaf quality evaluation index system, and generate a standard verification sample set containing real quality labels. Step S2: Obtain the prediction results of the model to be validated; input the fresh tobacco leaf images from the standard validation sample set into the fresh tobacco leaf quality evaluation model to be validated, and obtain the predicted quality category or predicted quality score output by the model. Step S3: Calculate the basic accuracy index; compare the prediction results with the true quality labels, and calculate at least one of the following: overall classification accuracy, category average accuracy, root mean square error, and coefficient of determination; Step S4: Multi-dimensional cross-validation; Group the samples for validation from multiple dimensions, and use K-fold cross-validation to evaluate the model's generalization ability; Step S5: Expert Consistency Test; Compare the model prediction results with the consensus labels of experts in the field of tobacco curing, and calculate the Kappa coefficient to assess consistency; Step S6: Actual roasting verification; Based on the model prediction results, match the roasting process to carry out actual roasting, measure the quality of the roasted tobacco leaves, and calculate the correlation coefficient between the predicted quality and the actual quality. Step S7: Overall accuracy assessment; Based on the verification results of steps S3 to S6, calculate the overall accuracy score according to preset weights, and determine whether the model passes the verification based on the score.

2. The verification method for the fresh tobacco leaf quality evaluation model according to claim 1, characterized in that, Step S1 specifically includes: S1-1: Among the collected fresh tobacco leaf samples, the varieties included Yunyan 87, K326, and Hongda; the parts included upper leaves, middle leaves, and lower leaves; and the maturity levels included underripe, still ripe, moderately ripe, and overripe. S1-2: The fresh tobacco leaf quality evaluation index system includes primary indicators and secondary indicators: The primary indicators include appearance and morphology indicators, physicochemical and physiological indicators, and harvest maturity indicators. The secondary indicators under the appearance and morphology indicators include: color characteristics, leaf length, leaf width, leaf shape index, midrib color, and degree of pubescence loss; The secondary indicators under the physicochemical and physiological indicators include: water content, relative chlorophyll content, total sugar content, starch content, protein content, polyphenol oxidase activity, and peroxidase activity. The secondary indicators under the harvest maturity index include: the degree of yellowing of leaf color, the degree of whitening of the midrib, and the degree of hair loss; S1-3: Associate the multidimensional index data obtained in step S1-2 with the true label value of fresh tobacco leaf quality to form a standard verification sample set labeled with true quality category or true quality score. S1-4: Divide the standard validation sample set into a validation set and a test set according to a preset ratio. The validation set is used for parameter fine-tuning during the model validation process, and the test set is used for independent evaluation of the final accuracy. The preset ratio is 7:3 or 8:

2.

3. The verification method for the fresh tobacco leaf quality evaluation model according to claim 1, characterized in that, Step S2 specifically includes: S2-1: Input the fresh tobacco leaf images from the standard verification sample set into the fresh tobacco leaf quality evaluation model to be verified. The model is based on a machine vision imaging system to obtain the color value and key morphological parameters of the fresh tobacco leaves. S2-2: Record the predicted quality category or predicted quality score output by the model to be validated for each validation sample.

4. The verification method for the fresh tobacco leaf quality evaluation model according to claim 3, characterized in that, The fresh tobacco leaf quality evaluation model to be verified uses at least one of the following methods for quality identification and classification: Method 1: Extract key feature points from fresh tobacco leaf images based on the SIFT feature extraction algorithm, combine them with the CCA feature dimensionality reduction method, and then use the XGBoost classification model for classification. Method 2: Classification or detection based on deep learning neural network models, including ResNet series networks, VGG series networks, MobileViT network, Faster-RCNN algorithm, YOLO algorithm, and SSD algorithm; Method 3: Classification based on a random forest classification model; Method 4: Classification based on Fisher's linear discriminant function model.

5. The verification method for the fresh tobacco leaf quality evaluation model according to claim 1, characterized in that, Step S3 specifically includes: S3-1: Compare the model prediction results obtained in step S2 with the true label values ​​of the standard validation sample set obtained in step S1 on a sample-by-sample basis. S3-2: Calculate the following basic accuracy metrics respectively: Overall classification accuracy (OA): in, To predict the number of samples whose class matches the true class, To verify the total number of samples; Average Accuracy (AA) for each category: in, The total number of quality categories, Let be the number of samples in class i that were correctly predicted. This represents the number of samples in class i that were incorrectly predicted as other classes. Root mean square error (RMSE) of quality score prediction: in, Predict the quality score for the j-th sample using the model. This represents the true quality score of the j-th sample. Coefficient of determination (R²) for predicting quality scores: in, This represents the average of the true quality scores of all validation samples.

6. The verification method for the fresh tobacco leaf quality evaluation model according to claim 1, characterized in that, Step S4 specifically includes: S4-1: Variety-based cross-validation: Group the validation samples by flue-cured tobacco variety, calculate the model accuracy for each variety, and evaluate the model's generalization ability among different varieties. S4-2: Part-based cross-validation: Group the validation samples according to tobacco leaf parts, calculate the model accuracy for each part, and evaluate the model's generalization ability across different parts. S4-3: Cross-validation of maturity dimension: Group the validation samples according to the harvest maturity level, calculate the model accuracy for each maturity level, and evaluate the model's ability to distinguish between different maturity levels; S4-4: K-fold cross-validation: The standard validation sample set is randomly divided into K subsets of similar size. One subset is taken as the test set and the remaining K-1 subsets are taken as the training set for K independent validations. The average of the K validation accuracies is taken as the final accuracy evaluation result.

7. The verification method for the fresh tobacco leaf quality evaluation model according to claim 1, characterized in that, Step S5 specifically includes: S5-1: Select at least 3 experts in the field of tobacco curing, and have the experts independently evaluate the quality of fresh tobacco samples in the verification sample set. The majority consensus of the experts' evaluation results shall be used as the expert consensus label. S5-2: Compare the model prediction results with the expert consensus labels, and calculate the consistency coefficient (Kappa coefficient) between the model predictions and the expert consensus: in, To observe the consistency rate, Expected consistency rate; S5-3: When the Kappa coefficient is ≥0.80, the prediction results of the judgment model are highly consistent with the expert consensus.

8. The verification method for the fresh tobacco leaf quality evaluation model according to claim 1, characterized in that, Step S6 specifically includes: S6-1: Based on the evaluation results of the fresh tobacco leaf quality of the verification sample set in step S1, match the corresponding baking process in the preset tobacco leaf precision baking process curve database. S6-2: Perform actual baking on the verification samples according to the matched baking process; S6-3: Determine the quality indicators of flue-cured tobacco leaves, including: the proportion of high-grade tobacco, the level of homogenization, and the curing loss rate; S6-4: Establish a correlation model between the predicted results of the fresh tobacco leaf quality evaluation model and the actual quality of the cured tobacco leaves, and calculate the correlation coefficient between the predicted quality category and the actual cured quality: in, Encode the quality category predicted by the model for the j-th sample. The actual baking quality score for the j-th sample; S6-5: When the correlation coefficient r ≥ 0.85, it is determined that the model prediction results are strongly correlated with the actual baking quality, thus verifying the effectiveness of the model in practical applications.

9. The verification method for the fresh tobacco leaf quality evaluation model according to claim 1, characterized in that, In step S7, based on the verification results of steps S3 to S6, the comprehensive accuracy score is calculated according to the corresponding weight allocation; when the comprehensive accuracy score is ≥85 points, the fresh tobacco leaf quality evaluation model to be verified is determined to have passed the verification and can be used for quality evaluation and process matching in tobacco curing production.

10. A verification system for a fresh tobacco leaf quality evaluation model, characterized in that, include: The sample collection module is used to collect fresh tobacco leaf samples of different varieties, parts, and maturity levels, and to record the variety information, part information, and harvest maturity level of the samples. The index measurement module is used to measure the appearance and morphology, physicochemical and physiological properties, and harvest maturity of the samples according to the preset fresh tobacco leaf quality evaluation index system. The standard validation sample set construction module is used to correlate the measurement results with the true competency label values ​​to form a standard validation sample set. The model prediction module is used to input fresh tobacco leaf images into the model to be validated and obtain the predicted quality category or predicted quality score. The basic accuracy calculation module is used to calculate the overall classification accuracy, the average accuracy of each category, the root mean square error of the quality score prediction, and the coefficient of determination. The cross-validation module is used to perform grouped validation and K-fold cross-validation based on variety, part, and maturity dimensions. The consistency verification module is used to calculate the Kappa coefficient between the model prediction results and the expert consensus labels; The actual roasting verification module is used to match and execute the roasting process, determine the quality of the roasted tobacco leaves, and calculate the correlation coefficient between the predicted results and the actual quality. The comprehensive evaluation module is used to calculate the overall accuracy score and determine whether the model passes validation based on the score.

Images