A method and system for evaluating similarity of tobacco leaf quality based on near-infrared spectroscopy

Abstract

The present application belongs to the technical field of tobacco quality evaluation, and discloses a tobacco quality similarity evaluation method and system based on near-infrared spectroscopy, which comprises the following steps: scanning tobacco samples by using a near-infrared spectrometer to obtain spectral data, reducing the dimension of the spectral data by using principal component analysis to obtain the first D principal components and their scores whose cumulative variance contribution rate is greater than 0.99, calculating the cosine distance and Euclidean distance between samples based on the principal component scores corresponding to the target sample set, converting the two-dimensional vector composed of the above two parameters into a one-dimensional scalar by introducing a kernel function and L2 regularization, and finally obtaining the similarity value, so as to realize the digital evaluation of the quality similarity between target samples. The present application provides a new technical idea for tobacco raw material substitution, leaf group formula auxiliary design, and cigarette product quality stability evaluation.

Description

Technical Field

[0001] This invention belongs to the field of tobacco quality evaluation technology, and in particular relates to a method and system for evaluating the similarity of tobacco quality based on near-infrared spectroscopy. Background Technology

[0002] Tobacco leaf quality evaluation is a crucial step in raw material control and cigarette formulation design in the tobacco industry, and its accuracy directly affects the sensory quality stability and production consistency of cigarette products. Traditional tobacco leaf quality evaluation methods mainly rely on manual sensory evaluation and chemical component detection. Sensory evaluation is highly dependent on experience and subjective, while conventional physicochemical testing methods suffer from long testing cycles, high costs, and difficulty in achieving rapid batch analysis. With the continuous improvement of informatization and digitalization in the tobacco industry, how to utilize efficient and objective technical means to achieve rapid evaluation and similarity analysis of tobacco leaf quality has become an important research direction in the field of tobacco quality control.

[0003] Near-infrared spectroscopy (NIR) has been widely used in tobacco quality testing due to its advantages of being rapid, non-destructive, and requiring no complex pretreatment. Studies have shown that NIR spectroscopy can reflect various chemical components and structural information in tobacco leaves. By analyzing spectral data using chemometric methods, functions such as tobacco variety identification, origin determination, and quality evaluation can be achieved. In recent years, researchers have commonly used multivariate analysis methods such as principal component analysis (PCA), partial least squares (PLS), and support vector machines to reduce the dimensionality and extract features from near-infrared spectral data of tobacco leaves to establish tobacco quality discrimination or classification models. By mapping spectral data to a low-dimensional feature space and then judging based on the differences in Euclidean distance or projection values ​​between samples, quantitative analysis of tobacco style or quality grade can be performed. Related studies have shown that principal component analysis can extract the main variation features of spectral data and use the distance relationship between samples to classify and evaluate tobacco style or quality. ([SciEngine][1]) However, in existing technologies, tobacco quality evaluation methods based on near-infrared spectroscopy mostly focus on classification or quantitative prediction. Their similarity analysis typically employs a single distance metric, such as Euclidean distance or a simple similarity index, to measure the overall relationship between samples. When processing high-dimensional spectral feature data, these methods often describe sample differences only from a spatial distance perspective, neglecting the structural differences reflected by the directional information of sample feature vectors. This may result in the similarity evaluation results not comprehensively representing the sample distribution structure. Furthermore, because different metrics differ in numerical scale and trends, a unified quantification method is often lacking when conducting comprehensive evaluations, which affects the stability and consistency of the evaluation results.

[0004] Therefore, in the field of tobacco quality similarity analysis based on near-infrared spectral data, how to more comprehensively describe the differences between samples in a low-dimensional feature space and obtain stable and uniform similarity measurement results remains a technical problem that needs further research and improvement. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention provides a method and system for evaluating the similarity of tobacco leaf quality based on near-infrared spectroscopy.

[0006] This invention is implemented as follows: A method for evaluating the similarity of tobacco leaf quality based on near-infrared spectroscopy includes: Step 1: Obtain the principal component score vector of the target tobacco leaf sample after principal component analysis; Step 2: Based on the principal component score vector, calculate the angular difference and spatial distance difference between samples respectively; Step 3: Convert the spatial distance difference into a proximity measure using an exponential decay method; Step 4: Perform a monotonic uniform transformation on the proximity measure to make it consistent with the angle difference in terms of numerical range and trend. Step 5: Based on the angle difference and the affinity measure after the standardization process, a one-dimensional similarity scalar is calculated using the L2 norm to characterize the similarity of tobacco leaf quality.

[0007] Furthermore, the angular difference is the cosine distance calculated based on the inner product of the principal component score vectors of the two samples and the vector magnitude, and its value ranges from 0 to 2.

[0008] Furthermore, the affinity metric is obtained by mapping the Euclidean distance between the sample principal component score vectors using a Gaussian function. The smaller the Euclidean distance, the larger the corresponding affinity metric value.

[0009] Furthermore, the monotonic uniformity transformation performs a linear inverse mapping on the affinity metric, such that the smaller the value, the higher the sample similarity, and the range of the mapped value is consistent with the angular difference.

[0010] Another object of the present invention is to provide a sample feature construction method for calculating near-infrared spectral similarity of tobacco leaves, comprising: Near-infrared spectral data from multiple tobacco leaf samples were centrally processed. Singular value decomposition is performed on the centered spectral data to obtain the corresponding set of eigenvectors. Based on the magnitude of the eigenvalues, select the first few eigenvectors with a cumulative variance contribution rate greater than 0.99; By using the selected feature vectors to perform a linear transformation on the centered spectral data, a principal component score vector space for sample similarity calculation is constructed.

[0011] Furthermore, the dimension of the principal component score vector is determined by the number of selected feature vectors and is less than the number of original spectral variables.

[0012] Furthermore, the near-infrared spectral data undergoes at least one or more of the following processes before being processed by principal component analysis: first derivative processing, multivariate scattering correction processing, or standard normal variable transformation processing.

[0013] Another objective of this invention is to provide a method and system for evaluating the similarity of tobacco leaf quality based on near-infrared spectroscopy, comprising: The feature construction module is used to convert near-infrared spectral data of tobacco leaves into principal component score vectors that satisfy the variance contribution rate constraint. The dual difference calculation module is used to calculate the angular difference and spatial distance difference based on the principal component score vector, respectively. A consistency processing module is used to convert the spatial distance difference into a difference that is consistent with the numerical range and trend of the angle difference. The similarity calculation module is used to calculate a one-dimensional similarity scalar based on the angle difference and the uniformized difference.

[0014] Furthermore, the dual difference calculation module includes a cosine distance calculation unit and an Euclidean distance calculation unit, which are used to characterize the differences of the sample in the orientation dimension and the amplitude dimension, respectively.

[0015] Furthermore, the similarity calculation module performs a L2 norm operation based on the angle difference and the uniformized difference, and the smaller the output similarity scalar value, the higher the similarity of tobacco leaf quality.

[0016] Based on the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solution to be protected by this invention are as follows: This invention utilizes near-infrared technology to rapidly extract tobacco leaf quality information. Based on this, it calculates the cosine distance and Euclidean distance, which characterize the similarity of tobacco leaf quality. By introducing kernel functions and L2 norms, the two-dimensional vector composed of cosine distance and Euclidean distance is transformed into a one-dimensional scalar. Finally, a similarity algorithm is constructed for the digital evaluation of tobacco leaf quality similarity.

[0017] This invention proposes an algorithm for evaluating the similarity of tobacco leaf quality based on near-infrared spectroscopy. Near-infrared spectroscopy is used to rapidly scan and extract tobacco leaf quality information. Principal component analysis (PCA) is used to reduce the dimensionality of the spectral data, obtaining principal component eigenvalues ​​and scores. The top D principal components with a cumulative variance contribution rate greater than 0.99 and their scores are selected to replace the original spectra as new data characterizing tobacco leaf quality. Based on the principal component scores, the cosine and Euclidean distances between each target sample are calculated. By introducing kernel functions and the L2 norm, the cosine and Euclidean distances are effectively coupled, transforming the two-dimensional vector into a one-dimensional scalar. Finally, a similarity algorithm is constructed to evaluate the similarity of tobacco leaf quality.

[0018] (1) The expected benefits and commercial value of the technical solution of this invention after transformation are as follows: Evaluation of tobacco leaf quality similarity is an important technical means to improve raw material utilization and reduce costs and increase efficiency in the tobacco industry. This invention is based on near-infrared spectral data and supplemented by a constructed similarity model to achieve rapid evaluation of tobacco leaf quality similarity. It can be applied to tobacco leaf raw material substitution, auxiliary formula design and quality stability evaluation in the cigarette manufacturing process, and has high feasibility and practicality.

[0019] (2) The technical solution of this invention fills a technical gap in the industry both domestically and internationally: Cosine distance and Euclidean distance are commonly used algorithms in machine learning to measure the similarity of research objects. This invention successfully couples the two effectively, avoiding the limitations and one-sidedness of using them alone, making the similarity evaluation more reasonable and comprehensive.

[0020] (3) The technical solution of the present invention solves a technical problem that people have long wanted to solve but have never been able to solve successfully: Cosine distance and Euclidean distance differ in their dimensions and function monotonicity when evaluating similarity, so they cannot be directly coupled. This invention overcomes the contradiction between the two in terms of dimensions and monotonicity by introducing a Gaussian kernel function to transform Euclidean distance into affinity. Then, the two-dimensional parameter vector is transformed into a one-dimensional scalar by introducing the L2 norm. Attached Figure Description

[0021] Figure 1 This is a flowchart of the tobacco leaf quality similarity evaluation method based on near-infrared spectroscopy provided in an embodiment of the present invention.

[0022] Figure 2 This is a structural block diagram of a tobacco leaf quality similarity system based on near-infrared spectroscopy provided in an embodiment of the present invention.

[0023] Figure 3 This is a schematic diagram of a similarity measurement based on spectral angle and Euclidean distance provided in an embodiment of the present invention.

[0024] Figure 4 The images shown are the original near-infrared spectra (a) and first derivative (b) of the formulation samples for 2022 and 2023 provided in this embodiment of the invention.

[0025] Figure 5 This is a heatmap of the sensory score difference matrix between formulation samples in 2022 and 2023, provided in an embodiment of the present invention.

[0026] Figure 6 This is a heatmap of similarity values ​​between formulation samples from 2022 and 2023, provided in an embodiment of the present invention. Figure 7 This is a heatmap of the similarity values ​​of single-grade tobacco leaves from various production areas in Shandong Province from 2022 to 2024, provided by an embodiment of the present invention.

[0027] Figure 8 This is a heatmap of similarity values ​​of re-dried tobacco samples from different provinces provided in an embodiment of the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0029] like Figure 1 As shown in the figure, the method for evaluating the similarity of tobacco leaf quality based on near-infrared spectroscopy provided in this embodiment of the invention includes the following steps: S101, Obtain the principal component score vector of the target tobacco leaf sample after principal component analysis; S102, based on the principal component score vector, calculate the angular difference and spatial distance difference between samples respectively; S103, the spatial distance difference is converted into a proximity measure by exponential decay; S104, Perform a monotonically uniform transformation on the proximity measure to make it consistent with the angle difference in terms of numerical range and trend of change. S105, based on the angle difference and the affinity measure after the standardization process, a one-dimensional similarity scalar is calculated using the L2 norm to characterize the similarity of tobacco leaf quality.

[0030] This invention provides a method for evaluating the similarity of tobacco leaf quality based on near-infrared spectroscopy. It leverages the ability of near-infrared spectroscopy to reflect the internal chemical composition and structural characteristics of tobacco leaves. By reducing the dimensionality of spectral information and constructing a multi-dimensional feature difference measurement mechanism, a quantitative evaluation of the similarity of tobacco leaf quality is achieved. The basic principle is to comprehensively consider the directional and distance relationships between samples in a low-dimensional feature space, and obtain a stable one-dimensional similarity index through unified dimensions and fusion calculation, thereby more accurately reflecting the similarity of tobacco leaf quality.

[0031] In step S101, principal component analysis (PCA) is performed on the acquired near-infrared spectral data of tobacco leaves. Since raw spectral data typically has high dimensionality and strong correlations between variables, direct similarity analysis in the raw spectral space is easily affected by noise and redundant information. PCA extracts several principal components reflecting the main spectral variation characteristics and maps the raw spectral data to the principal component feature space, thus obtaining the principal component score vector of the target tobacco leaf sample. This vector can retain the main structural features of the spectral information at a lower dimension, providing a stable feature representation for subsequent sample difference analysis.

[0032] In step S102, the principal component score vector is used as the sample feature representation, and the angular difference and spatial distance difference between samples are calculated respectively. The angular difference reflects the degree of directional difference between the feature vectors of two samples, describing the changes in the sample's feature structure; the spatial distance difference reflects the degree of overall positional difference between the two samples in the feature space, describing the absolute difference between the samples. By simultaneously introducing both directional and distance differences, the quality differences between tobacco leaf samples can be characterized from different dimensions, making the similarity evaluation more comprehensive.

[0033] In step S103, the calculated spatial distance difference is converted into a proximity metric using an exponential decay method. This conversion process maps distance information to a similarity index with decay characteristics, so that the smaller the sample distance, the higher the corresponding proximity metric value, thereby converting the distance quantity that originally represented the degree of difference into a similarity index that represents the degree of proximity.

[0034] In step S104, the obtained affinity metric undergoes a monotonically consistent transformation to ensure that its numerical range and trend are consistent with the angular difference metric. This process ensures that metrics from different sources maintain consistency in terms of dimensions and variation patterns, thereby avoiding imbalances in the comprehensive calculation results caused by scale differences between different metrics and providing a foundation for subsequent unified fusion calculations.

[0035] In step S105, the angular difference and the homogenized proximity measure are used as joint measurement parameters, and a one-dimensional similarity scalar is calculated using the L2 norm. This one-dimensional scalar comprehensively reflects the differences between samples in both feature direction and spatial location, and can be used to characterize the degree of similarity between tobacco leaf qualities. Through the above processing flow, this invention can achieve stable and uniform tobacco leaf quality similarity evaluation in a low-dimensional feature space, improving the application effect of near-infrared spectral data in tobacco leaf quality analysis.

[0036] The angular difference provided in this embodiment of the invention is the cosine distance calculated based on the inner product of the principal component score vectors of two samples and the vector magnitude, and its value ranges from 0 to 2.

[0037] The affinity metric provided in this embodiment of the invention is obtained by mapping the Euclidean distance between the principal component score vectors of the samples using a Gaussian function. The smaller the Euclidean distance, the larger the corresponding affinity metric value.

[0038] The monotonic uniformity transformation provided in this embodiment of the invention performs a linear inverse mapping of the affinity metric, such that the smaller the value, the higher the sample similarity, and the range of values ​​after mapping is consistent with the angular difference.

[0039] The sample feature construction method for calculating near-infrared spectral similarity of tobacco leaves provided in this embodiment of the invention includes: Near-infrared spectral data from multiple tobacco leaf samples were centrally processed. Singular value decomposition is performed on the centered spectral data to obtain the corresponding set of eigenvectors. Based on the magnitude of the eigenvalues, select the first few eigenvectors with a cumulative variance contribution rate greater than 0.99; By using the selected feature vectors to perform a linear transformation on the centered spectral data, a principal component score vector space for sample similarity calculation is constructed.

[0040] The dimension of the principal component score vector provided in this embodiment of the invention is determined by the number of selected feature vectors and is less than the number of original spectral variables.

[0041] The near-infrared spectral data provided in this embodiment of the invention undergoes at least one or more of the following processes before entering principal component analysis: first derivative processing, multivariate scattering correction processing, or standard normal variable transformation processing.

[0042] like Figure 2 As shown in the figure, an embodiment of the present invention provides a method system for evaluating the similarity of tobacco leaf quality based on near-infrared spectroscopy, comprising: The feature construction module is used to convert near-infrared spectral data of tobacco leaves into principal component score vectors that satisfy the variance contribution rate constraint. The dual difference calculation module is used to calculate the angular difference and spatial distance difference based on the principal component score vector, respectively. A consistency processing module is used to convert the spatial distance difference into a difference that is consistent with the numerical range and trend of the angle difference. The similarity calculation module is used to calculate a one-dimensional similarity scalar based on the angle difference and the uniformized difference.

[0043] Furthermore, the dual difference calculation module includes a cosine distance calculation unit and an Euclidean distance calculation unit, which are used to characterize the differences of the sample in the orientation dimension and the amplitude dimension, respectively.

[0044] Furthermore, the similarity calculation module performs a L2 norm operation based on the angle difference and the uniformized difference, and the smaller the output similarity scalar value, the higher the similarity of tobacco leaf quality.

[0045] An embodiment of the present invention provides a method for evaluating the similarity of tobacco leaf quality based on near-infrared spectroscopy, comprising the following steps: (1): The target tobacco leaf sample was scanned using a near-infrared spectrometer to obtain near-infrared spectral data, and the spectral data was preprocessed. (2): Using the spectral data of each sample as a row vector, the spectral data of all samples form a matrix X, and principal component analysis is performed on X; (3): Calculate the cosine distance between the target samples ( Cosine distance , d cos ); (4): Calculate the affinity between target samples ( Degree of affinity , d aff ); (5): Calculate the degree of similarity between target samples. d sim ).

[0046] The preprocessing method provided in this embodiment of the invention includes: first derivative, second derivative, multivariate scattering correction (MSC), and standard normal variable transformation (SNV).

[0047] The present invention provides a method where the spectral data of each sample is used as a row vector, and the spectral data of all samples form a matrix X. Principal component analysis is then performed on X, as follows: First, analyze the spectral matrix ( X The variables of ) are centered to obtain X c Then to X cPerform singular value decomposition:

[0048] Where V is X c eigenmatrix S For a diagonal matrix, the elements on the main diagonal are... s j With eigenvalues λ j The following relationship exists:

[0049] As can be seen from the above, the spectral matrix ( X The principal component score matrix corresponding to () Z )for:

[0050] Based on the characteristic values ​​of each principal component λ j The top D principal components with a cumulative variance contribution rate greater than 0.99 were selected as the new principal component score matrix. Z n×D ) is used to represent sample information, where n represents the number of samples and D represents the number of variables (dimensions) of the sample.

[0051] The present invention provides a method for calculating the cosine distance between target samples. Cosine distance , d cos ): For matrix Z n×D = [ z 1 , z 2 , …, z n ] T Any two sample vectors in z i and z j Their cosine distance ( d cos ) is defined as:

[0052] The cosine distance ranges from [0,2]. The smaller the value, the smaller the angle between the vectors of the two samples in space, i.e., the higher the similarity; conversely, the larger the value, the lower the similarity.

[0053] The present invention provides a method for calculating the affinity between target samples. Degree of Affinity , d aff): First, calculate the Euclidean distance. d euc For any two sample vectors z i and z j Their Euclidean distance is defined as:

[0054] The Euclidean distance ranges from [0, +∞). The smaller the value, the shorter the distance between the two samples in space, i.e., the higher the similarity; conversely, the larger the value, the lower the similarity. To facilitate effective coupling of the two parameters above, the ranges of values ​​for cosine distance and Euclidean distance must be unified. Therefore, a Gaussian kernel (RBF kernel) is introduced to transform the Euclidean distance into "proximity":

[0055] κ RBF The value range is (0,1]. z i and z j When the distance is 0, the value is 1; when the distance is +∞, the value approaches 0. γ needs to be reasonably selected based on the Euclidean distance between all samples to ensure that their... κ RBF The values ​​should be distributed as evenly as possible within the interval (0,1). Since the cosine distance ranges from [0,2], and a smaller value indicates higher similarity, this is consistent with... κ RBF They show opposite trends; in order to unify the range of values ​​and the monotonicity of the functions, the affinity must be transformed as follows:

[0056] From the above formula, we can see that d aff The value range is [0, 2), and the smaller the value, the higher the similarity. This is consistent with... d cos The range of values ​​and monotonicity are consistent.

[0057] The present invention provides a method for calculating the degree of similarity between target samples. d sim ): As can be seen from the above, any two samples ( z i , zj The similarity between them can be determined by d cos and d aff While similarity is represented using two dimensions, it remains a two-dimensional vector, which is not conducive to comparing the similarity between samples. Therefore, the L2 norm is introduced to "reduce the dimensionality" of similarity, transforming it into a one-dimensional similarity scalar.

[0058] d sim The value range is [0, The smaller the value, the higher the similarity between the two samples, and vice versa.

[0059] Specific implementation of the present invention: like Figure 3 As shown, among the three sample points A, B, and C in two-dimensional space, based on the spectral angle, the similarity between A and B is higher than that between A and C (θ). AC >θ AB According to the Euclidean distance, A and C are higher than A and B (d). AB >d AC ).

[0060] This invention discloses four grades (B2F, C2F, C3F, X2F) of tobacco leaves from a certain production area in 2022 and 2023, respectively, blended according to a uniform formula design. Fifteen formula samples were obtained for each year. Based on the near-infrared data of the formula samples from these two years... Figure 4 ) and the sensory score difference matrix among samples ( Figure 5 According to the present invention, a similarity matrix between formulation samples was calculated. Figure 6 A correlation analysis was conducted between the results and the corresponding sensory score differences (Table 1), the details of which are as follows: Table 1. Correlation analysis of similarity values ​​and sensory score differences between each formulation sample and other samples in 2022 and 2023.

[0061] Theoretically, the lower the similarity between two samples, the greater the corresponding difference in sensory quality. That is, there is a positive correlation between the similarity value of each formulation sample and the difference in sensory scores with other samples in an overall trend. To investigate whether the above assumption is accurate, this embodiment conducted a correlation analysis on the similarity value and the difference in sensory scores between each formulation sample and other samples in 2022 and 2023 (Table 1). For 2022, except for September and November 2022, the similarity values ​​and sensory score differences among the remaining samples were positively correlated. Among them, 10 samples from March to August 2022, October 2022, and March to November 2022 showed a significant positive correlation (p<0.05), accounting for 66.7% of the total samples. For 2023, except for August and September 2023, the similarity values ​​and sensory score differences among the remaining samples were positively correlated. Among them, 9 samples from January 2023, February 2023, April 2023, June 2023, July 2023, and October to November 2023 showed a significant positive correlation (p<0.05), accounting for 60.0% of the total samples. This indicates that the similarity values ​​and sensory score differences among the samples have good consistency, that is, the lower the similarity (higher), the greater the sensory difference (smaller).

[0062] Example 1: Implementation Method for Basic Similarity Calculation of Standard Tobacco Leaf Samples Multiple tobacco leaf samples from consistent sources and with known grades were selected. Each sample was scanned using a near-infrared spectrometer under uniform conditions to obtain raw near-infrared spectral data. After centering the spectral data, a spectral sample matrix was constructed, and eigenvectors were extracted using singular value decomposition. Principal components with a cumulative variance contribution rate greater than 0.99 were selected based on the eigenvalue magnitude, forming a low-dimensional principal component score vector space. The angular and spatial distance differences between samples were calculated in this space. Spatial distance was mapped to a proximity metric using an exponential decay method, and numerical interval and monotonicity were standardized to obtain a one-dimensional similarity scalar, enabling precise differentiation of tobacco leaf quality.

[0063] Figure 7 A heatmap of the similarity values ​​between pairs of 76 single-grade tobacco leaves from various production areas in Shandong Province from 2022 to 2024 is presented. Similarity values ​​range from [0,2], but the vast majority of pairs have values ​​less than 1.0. The 2022-Lanling-X2F, 2024-Qishan-X2F, 2024-Gaozhuang-B1F, and 2024-Gaozhuang-B2F samples show higher similarity values ​​with other samples, indicating lower similarity between them. Table 2 lists the top ten samples with the highest similarity to each single-grade tobacco leaf sample, sorted from highest to lowest similarity.

[0064] Table 2. List of the top 10 samples with the highest similarity to single-grade tobacco leaves from different years and origins.

[0065] Example 2: Robust Implementation Method Introducing Multiple Spectral Preprocessing Techniques Based on Example 1, near-infrared spectral data were processed using first-order derivative processing, standard normal variable transformation, and multivariate scattering correction to eliminate the influence of particle size, surface state, and optical path variation on the spectrum. Spectral data obtained from different preprocessing methods were then subjected to the same principal component analysis and similarity calculation process. The results show that under different preprocessing conditions, the relative ranking of the similarity scalar in sample quality grouping remained stable, indicating that the aforementioned similarity calculation mechanism has good robustness to spectral perturbations.

[0066] Example 3: Implementation of Cross-Source Similarity Determination for Tobacco Leaves from Multiple Origin Regions Multiple batches of tobacco leaf samples from different origins were selected to establish a sample set containing significant differences in chemical composition. Using a unified principal component space construction method, tobacco leaves from different origins were mapped to the same feature space. Within this space, both angular difference and uniformized spatial difference were introduced for joint measurement. This effectively distinguishes the internal similarity of samples from the same origin from the overall differences between samples from different origins, avoiding the sensitivity imbalance problem of a single distance metric to scale or directional changes.

[0067] Figure 8 A heatmap showing the similarity value matrix of re-dried tobacco samples from different provinces, blended at multiple grades, is presented. For example... Figure 8 As shown, near the main diagonal, the similarity of re-dried tobacco samples from the same province is higher, while the similarity between different provinces is lower. It is worth noting that the similarity between Hubei Burley tobacco and flue-cured tobacco from other provinces is significantly different. Table 3 lists the top 10 tobacco samples with the highest similarity to each tobacco sample.

[0068] Table 3. List of the top 10 samples with the highest similarity to re-dried tobacco sheets from different provinces.

[0069] Example 4: Implementation of Adaptive Principal Component Dimension Selection Under different sample sizes, the principal component eigenvalues ​​are dynamically calculated, and the number of principal components is determined based on the cumulative variance contribution rate. When the sample size is small, the principal component dimensionality is automatically reduced; as the sample size increases, the dimensionality is expanded accordingly. The principal component score vector constructed in this way retains the main spectral information while avoiding interference from high-dimensional noise in similarity calculation, ensuring consistent discriminative ability of the similarity scalar across different data sizes. In this algorithm, the principle for selecting the number of principal components is: sorting them by eigenvalue size and selecting the top few principal components with a cumulative variance contribution rate greater than 0.99. Therefore, a cumulative variance contribution rate greater than 0.99 is the fundamental principle for ultimately selecting the number of principal components. The sample size may affect the number of principal components, but the impact is not significant.

[0070] Example 5: The effect of affinity uniformity transformation on similarity stability. Implementation method. While keeping the calculation method of angular difference unchanged, spatial distance difference was subjected to direct normalization and exponential decay-based standardization. The results show that the unstandardized spatial difference has a dominant influence on the final similarity under extreme distance conditions. However, after standardization, the two types of difference remain synergistic in terms of numerical range and trend, enabling the final similarity to comprehensively reflect both direction and amplitude information, significantly improving the stability and rationality of similarity judgment.

[0071] Example 6: Implementation of Batch Sample Similarity Application under Systematic Deployment Conditions This system integrates feature construction, dual-difference calculation, standardization, and similarity calculation into a single computational system for automated processing of large batches of tobacco leaf samples. Upon receiving new tobacco leaf spectral data, the system automatically performs feature mapping and similarity calculation, outputting the similarity results with historical samples. Practical operation demonstrates that the system maintains a consistent similarity scale under continuous sample input conditions, making it suitable for tobacco leaf quality grading, sample comparison, and quality control scenarios.

[0072] It should be noted that embodiments of the present invention can be implemented in hardware, software, or a combination of both. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by a suitable instruction execution system, such as a microprocessor or dedicated-design hardware. Those skilled in the art will understand that the above-described devices and methods can be implemented using computer-executable instructions and / or included in processor control code, for example, such code provided on a carrier medium such as a disk, CD, or DVD-ROM, a programmable memory such as read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuitry such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field-programmable gate arrays, programmable logic devices, etc., or by software executed by various types of processors, or by a combination of the above-described hardware circuitry and software, such as firmware.

[0073] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for evaluating the similarity of tobacco leaf quality based on near-infrared spectroscopy, characterized in that, include: Step 1: Obtain the principal component score vector of the target tobacco leaf sample after principal component analysis; Step 2: Based on the principal component score vector, calculate the angular difference and spatial distance difference between samples respectively; Step 3: Convert the spatial distance difference into a proximity measure using an exponential decay method; Step 4: Perform a monotonic uniform transformation on the proximity measure to make it consistent with the angle difference in terms of numerical range and trend. Step 5: Based on the angle difference and the affinity measure after the standardization process, a one-dimensional similarity scalar is calculated using the L2 norm to characterize the similarity of tobacco leaf quality.

2. The method as described in claim 1, characterized in that, The angle difference is the cosine distance calculated based on the inner product of the principal component score vectors of the two samples and the vector magnitude, and its value ranges from 0 to 2.

3. The method as described in claim 1, characterized in that, The affinity metric is obtained by mapping the Euclidean distance between the sample principal component score vectors using a Gaussian function. The smaller the Euclidean distance, the larger the corresponding affinity metric value.

4. The method as described in claim 1, characterized in that, The monotonically uniform transformation performs a linear inverse mapping of the affinity metric, such that the smaller the value, the higher the sample similarity, and the range of the mapped value is consistent with the angular difference.

5. The method as described in claim 1, characterized in that, Sample feature construction methods for calculating near-infrared spectral similarity of tobacco leaves include: Near-infrared spectral data from multiple tobacco leaf samples were centrally processed. Singular value decomposition is performed on the centered spectral data to obtain the corresponding set of eigenvectors. Based on the magnitude of the eigenvalues, select the first few eigenvectors with a cumulative variance contribution rate greater than 0.99; By using the selected feature vectors to perform a linear transformation on the centered spectral data, a principal component score vector space for sample similarity calculation is constructed.

6. The method as described in claim 5, characterized in that, The dimension of the principal component score vector is determined by the number of selected feature vectors and is less than the number of original spectral variables.

7. The method as described in claim 5, characterized in that, The near-infrared spectral data undergoes at least one or more of the following processes before being processed by principal component analysis: first derivative processing, multivariate scattering correction processing, or standard normal variable transformation processing.

8. A system for evaluating the similarity of tobacco leaf quality based on near-infrared spectroscopy, implementing the method for evaluating tobacco leaf quality based on any one of claims 1-4, characterized in that, include: The feature construction module is used to convert near-infrared spectral data of tobacco leaves into principal component score vectors that satisfy the variance contribution rate constraint. The dual difference calculation module is used to calculate the angular difference and spatial distance difference based on the principal component score vector, respectively. A consistency processing module is used to convert the spatial distance difference into a difference that is consistent with the numerical range and trend of the angle difference. The similarity calculation module is used to calculate a one-dimensional similarity scalar based on the angle difference and the uniformized difference.

9. The system as described in claim 8, characterized in that, The dual-difference calculation module includes a cosine distance calculation unit and an Euclidean distance calculation unit, which are used to characterize the differences of the sample in the direction dimension and the amplitude dimension, respectively.

10. The system as described in claim 8, characterized in that, The similarity calculation module performs a L2 norm operation based on the angle difference and the uniformized difference. The smaller the output similarity scalar value, the higher the similarity of the tobacco leaf quality.

Images