Tobacco replacement determination method based on multi-modal hybrid fusion

Abstract

Disclosed in the present invention is a tobacco replacement determination method based on multi-modal hybrid fusion. The method comprises the following steps: (1) obtaining a "static" quality feature matrix X of tobacco in a cigarette formula and "static" quality feature matrices Y of inventory tobacco; (2) calculating a Mahalanobis distance value d1, and performing normalization processing, so as to obtain a "static" quality similarity value a1; (3) obtaining a "dynamic" quality feature matrix U of the tobacco in the cigarette formula and "dynamic" quality feature matrices V of different inventory tobacco; (4) calculating a Mahalanobis distance value d2, and performing normalization processing, so as to obtain a "dynamic" quality similarity value a2; and (5) integrating a1 and a2, substituting same into f(a) for calculation, and finally determining whether the inventory tobacco can replace the tobacco in the formula, and a replacement risk probability.

Description

A tobacco leaf substitution judgment method based on multimodal hybrid fusion Technical Field

[0001] The present invention belongs to the technical field of cigarettes, and in particular relates to a tobacco leaf substitution judgment method based on multimodal hybrid fusion. Background Art

[0002] The quality and style of cigarettes are primarily determined by product designers through the blending of tobacco leaves from different origins, varieties, and grades. When a particular tobacco leaf runs out, manual selection of similar-quality leaves from hundreds of stockpiled grades is necessary, relying on recipe experience and sensory evaluation. This process is highly subjective and labor-intensive, and due to the lack of objective, scientific, and efficient technical means, it is difficult to fully guarantee the stability of cigarette quality and style. Using instrumental testing methods, objective data, and scientific techniques to assist with blend substitution would be crucial for improving cigarette quality stability.

[0003] The quality and distinctive style of cigarettes are a reflection of the chemical composition of tobacco leaves, which is composed of various raw materials, during combustion, affecting the human sensory organs. Therefore, the inherent chemical composition and content of tobacco leaves determine, to a certain extent, the sensory quality of the product. For example, compounds such as sugars, nitrogen, and nicotine in tobacco leaves are closely related to characteristics such as aroma, smoke, and mouthfeel. Domestic and international researchers and industry experts have devoted considerable effort to the application of tobacco leaf chemical composition indicators and formulation maintenance, and have achieved considerable research and application results.

[0004] In recent years, near-infrared technology, due to its rapidity, high efficiency, and rich quality information, has been widely used in tobacco leaf quality analysis and evaluation, as well as formulation design and maintenance, due to its ability to effectively characterize the quality information of tobacco leaf's intrinsic chemical composition. However, intrinsic chemical composition and near-infrared spectroscopy only consider the correlation between tobacco leaf quality and leaf formulation under "static conditions," ignoring the heat transfer behavior under combustion conditions during cigarette consumption. This does not truly represent the smoking quality of tobacco leaves, that is, the quality characteristics of the smoke. Therefore, formulation replacement technologies based on intrinsic chemical composition and near-infrared spectroscopy have certain limitations.

[0005] Thermal analysis technology is one of the most widely used methods in the industry to bridge the gap between tobacco leaves and smoke. Thermogravimetric analysis (TG) measures the relationship between sample weight (mass) and sample temperature or time under programmed temperature control and in different atmospheres. This technique indirectly reflects the pyrolysis and combustion behavior of different tobaccos and is a concrete manifestation of the "dynamic" quality characteristics of tobacco leaves under combustion conditions. Applications of thermal analysis technology to identify tobacco thermal conversion characteristics, conduct tobacco substitution experiments in formulations, and evaluate the batch quality stability of reconstituted tobacco and cigarette paper have been reported.

[0006] Multimodal fusion, also known as multi-source information fusion or multi-sensor fusion, refers to the process of combining information from two or more modalities for prediction. During prediction, a single modality typically cannot contain all the valid information required to produce accurate predictions. Multimodal fusion combines information from two or more modalities to complement each other, broaden the coverage of the input data, improve the accuracy of predictions, and enhance the robustness of the prediction model. Multimodal fusion methods include early fusion, late fusion, and hybrid fusion. Early fusion aims to alleviate inconsistencies between the raw data of each modality by first extracting feature representations from each modality and then fusing them at the feature level (i.e., feature fusion). Late fusion, also known as decision-level fusion, first trains different models for different modalities and then fuses the outputs of multiple models. Hybrid fusion methods combine the advantages of both early and late fusion methods, but also increase the model complexity and training difficulty.

[0007] To comprehensively characterize the smoking quality of cigarette blends and tobacco leaf raw materials, this present invention utilizes objective data and scientific techniques, based on chemical composition indicators and near-infrared spectral data, combined with thermogravimetric analysis data, to replace the traditional tobacco leaf substitution method in cigarette blends that relies on sensory evaluation and expert experience.

[0008] Summary of the Invention

[0009] The present invention adopts a multimodal hybrid fusion algorithm from three dimensions: intrinsic chemical composition indicators, near-infrared spectroscopy, and thermogravimetric analysis of cigarette combustion, to conduct a comprehensive quality similarity measurement and calculation between tobacco leaves in the formula and tobacco leaves in stock, so as to achieve objective judgment and intelligent recommendation on whether tobacco leaves in stock can replace tobacco leaves in cigarette formula.

[0010] The technical solutions of the present invention are as follows:

[0011] A tobacco leaf substitution judgment method based on multimodal hybrid fusion, as shown in FIG1 , includes the following steps:

[0012] (1) The intrinsic chemical component index data and near-infrared characteristic spectrum data of tobacco leaves in cigarette formula are fused using a multimodal feature fusion algorithm based on convolutional neural networks to obtain the "static" quality feature matrix X of tobacco leaves in cigarette formula; the intrinsic chemical component index data and near-infrared characteristic spectrum data of different stock tobacco leaves are fused using a multimodal feature fusion algorithm based on convolutional neural networks to obtain the "static" quality feature matrix Y of stock tobacco leaves;

[0013] (2) Calculate the Mahalanobis distance d1 between matrices X and Y and perform normalization to obtain the “static” quality similarity a1 between the tobacco leaves in the recipe and the tobacco leaves in stock;

[0014] (3) According to the main heat transfer weight loss stages during tobacco leaf combustion, the characteristic spectrum segments of the tobacco leaf thermogravimetric curve are screened to form the "dynamic" quality characteristic matrix U of tobacco leaves in cigarette formula and the "dynamic" quality characteristic matrix V of different stock tobacco leaves;

[0015] (4) Calculate the Mahalanobis distance d2 between matrices U and V, perform normalization, and obtain the “dynamic” quality similarity a2 between the tobacco leaves in the recipe and the tobacco leaves in stock;

[0016] (5) Based on the similarity value a1 of the “static” quality characteristics and the similarity value a2 of the “dynamic” quality characteristics of tobacco leaves, a decision fusion method based on fuzzy comprehensive evaluation is used to determine whether the tobacco leaves in stock can be substituted for the tobacco leaves in the formula, as well as the replacement risk probability.

[0017] Preferably, the chemical component index data of tobacco leaves selected in step (1) are 70 kinds, and the near-infrared characteristic spectrum of tobacco leaves is: 4350-4450cm -1 , 4500-4700cm -1 , 4800-4900cm -1 , 5050-5100cm -1 , 5220-5300cm -1 , 6050-6250cm -1 , 6400-6600cm -1 , 6950-7220cm -1 The data of 70 chemical component indicators of tobacco leaves are shown in Table 1. The 70 intrinsic chemical component indicators were classified and counted according to bases, sugars, acids, and Amadori compounds. Combined with the absorption peak positions of each molecular functional group in the near-infrared spectrum, the correlation between the chemical component indicators of tobacco leaves and the near-infrared spectrum was extracted, and the near-infrared characteristic spectrum of tobacco leaves was screened, as shown in Figure 2.

[0018] Preferably, the steps of using the multimodal feature fusion algorithm based on convolutional neural network to perform fusion are as shown in FIG3 : first, the intrinsic chemical composition index full connection layer f m and the near-infrared characteristic spectrum fully connected layer f n Multiply together to form a two-dimensional feature fusion transformation matrix A m×n ; Then, for A m×n Perform maximum pooling to form m feature nodes as the new fusion feature layer B m ; Finally, apply the softmax layer to B mLogarithmic normalization is performed to compress the variables and classify them between [0, 1], thereby obtaining the final fusion feature layer; among them, the fusion feature matrix X of tobacco leaves in the formula and the fusion feature matrix Y of tobacco leaves in stock are obtained.

[0019] Preferably, the calculation formula of the Mahalanobis distance value d1 in step (2) is: Where X is the “static” quality fusion feature matrix of tobacco leaves in cigarette formula, Y is the “static” quality fusion feature matrix of tobacco leaves in stock, and Σ is the covariance matrix of X and Y. The normalization formula is: After normalization, the data is transformed into a value between 0 and 1, and the distance value (difference value) is transformed into a similarity value.

[0020] Preferably, the characteristic spectrum segments of the thermogravimetric curve of the tobacco leaves in step (3) are: 190°C-240°C, 320°C-360°C; which respectively constitute the "dynamic" quality characteristic matrix U of the tobacco leaves in the formula and the "dynamic" quality characteristic matrix V of the tobacco leaves in stock.

[0021] Preferably, the Mahalanobis distance calculation formula in step (4) is: Among them, U is the “dynamic” quality characteristic matrix of tobacco leaves in cigarette formula, V is the “dynamic” quality characteristic matrix of tobacco leaves in stock, and Σ is the covariance matrix of U and V. The normalization formula is: After normalization, the data is transformed into a value between 0 and 1, and the distance value (difference value) is transformed into a similarity value.

[0022] Preferably, the judgment method of step (5) is: substitute a1 and a2 into the formula r=0.3*f(a1)+0.7*f(a2) for calculation; if r>0.5, the tobacco leaves in stock can replace the tobacco leaves in the cigarette formula, where the replacement risk probability is 1-r; if r≤0.5, the tobacco leaves in stock cannot replace the tobacco leaves in the cigarette formula; wherein f(a) is the set membership function:

[0023] Table 1 Chemical composition indexes of tobacco leaves

[0024] Beneficial effects of the present invention:

[0025] The present invention's tobacco leaf substitution judgment method, based on multimodal hybrid fusion, uses a multimodal hybrid fusion algorithm to objectively characterize the complete quality characteristics of tobacco leaves in cigarette formulas and inventory tobacco leaves from three dimensions: intrinsic chemical composition indicators, near-infrared spectroscopy, and cigarette combustion thermogravimetric analysis. This method then performs a comprehensive quality similarity measurement on the tobacco leaves in the formula and inventory tobacco leaves, enabling objective judgment and intelligent recommendation of whether inventory tobacco leaves can replace tobacco leaves in cigarette formulas. Furthermore, it provides a formula replacement risk probability for these replaceable tobacco leaves. The present invention's judgment method is objective, scientific, and efficient, enabling judgment of whether inventory tobacco leaves can replace tobacco leaves in cigarette formulas and intelligent recommendation of replacements, fully ensuring the stability of cigarette quality and style. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a schematic diagram of the tobacco leaf substitution judgment method in cigarette formula based on multimodal hybrid fusion.

[0027] FIG2 is a diagram showing the screening of characteristic spectrum segments of near-infrared spectrum emitted by tobacco leaves according to the present invention.

[0028] FIG3 is a schematic diagram of the tobacco leaf “static” quality feature fusion algorithm of the present invention. DETAILED DESCRIPTION

[0029] The present invention will be further described in detail below through specific embodiments, but this should not be construed as limiting the scope of the present invention to the following examples. Various substitutions and modifications made according to common technical knowledge and customary means in the art without departing from the spirit of the present invention are intended to be included within the scope of the present invention.

[0030] Example: Determination and recommendation of whether stock tobacco leaves can be substituted for tobacco leaves in cigarette formulations.

[0031] The information of tobacco leaves in the recipe selected for this example and the tobacco leaves in stock are shown in Table 2.

[0032] Table 2 Information on tobacco leaves and tobacco leaves in stock in the recipe selected in this embodiment

[0033] The specific steps for determining whether stockpiled tobacco leaves can replace tobacco leaves in cigarette formulations are as follows:

[0034] (1) The tobacco leaf sample in the cigarette formula is evenly divided into two parts; one part is used for near-infrared spectroscopy analysis, and the other part is used for thermogravimetric analysis.

[0035] (2) Near infrared spectroscopy analysis of tobacco leaves in the formula and in stock, specifically:

[0036] The tobacco leaf sample was dried in a 40℃ oven for 2 hours, crushed by a cyclone mill and passed through a 40-mesh sieve. 3 g of tobacco leaf powder was weighed and placed in a sample cup. Scanning was performed at 4000-9000 cm-1 The spectrum of each band was repeated 10 times for each sample and the average value of the spectrum was taken.

[0037] Combined with the absorption peak position of each molecular functional group in the near infrared spectrum, the correlation between the chemical composition index of tobacco leaves and the near infrared spectrum was extracted, and the near infrared characteristic spectrum of tobacco leaves was screened. As shown in Figure 2, the characteristic spectrum of tobacco leaves near infrared spectrum was screened: 4350-4450cm -1 , 4500-4700cm -1 , 4800-4900cm -1 , 5050-5100cm -1 , 5220-5300cm -1 , 6050-6250cm -1 , 6400-6600cm -1 , 6950-7220cm -1 .

[0038] The near-infrared prediction model of 70 intrinsic chemical component indicators of tobacco leaves was used to obtain the intrinsic chemical component data of tobacco leaves, as shown in Table 3.

[0039] Table 3 Data on 70 intrinsic chemical components of tobacco leaves (taking samples 1#-5# as an example)

[0040] (3) Aiming at the “static” quality characteristics of tobacco leaves, a multimodal feature fusion algorithm based on convolutional neural network is used to fuse the intrinsic chemical component index data and near-infrared characteristic spectrum data of tobacco leaves in the formula and in stock. First, the near-infrared characteristic spectrum fully connected layer f m and the fully connected layer f of the characteristic spectrum of the thermogravimetric curve n Multiply together to form a two-dimensional feature fusion transformation matrix A m×n ; Then, for A m×n Perform maximum pooling to form m feature nodes as the new fusion feature layer B m ; Finally, apply the softmax layer to B m Logarithmic normalization is performed to compress the variables and classify them between [0, 1], thereby obtaining the final formula tobacco leaf fusion feature matrix X and inventory tobacco leaf fusion feature matrix Y, as shown in Table 4.

[0041] Table 4 Fusion feature matrix X of tobacco leaves in recipe and fusion feature matrix Y of tobacco leaves in stock

[0042] (3) The Mahalanobis distance d1 of matrices X and Y was calculated and normalized to obtain the similarity value a1 of the “static quality” between the tobacco leaves in the formula and the tobacco leaves in stock. The results are shown in Table 5.

[0043] Table 5 Similarity values ​​a1 of the “static” quality feature fusion matrix of tobacco leaves in the recipe and tobacco leaves in stock

[0044] (5) Thermogravimetric analysis of tobacco leaves: Using a TGA / DSC 1LF thermogravimetric analyzer from Mettler Toledo, Germany, (5.00±0.05) mg of sample was weighed and placed in a thermogravimetric alumina crucible. The sample was heated from 30°C to 900°C at a heating rate of 10°C / min under a nitrogen flow rate of 30 mL / min to obtain the thermogravimetric curve of the tobacco leaves. Based on the main heat transfer weight loss stages during tobacco leaf combustion, the characteristic spectrum of the thermogravimetric curve of the tobacco leaves was screened: 190°C-240°C and 320°C-360°C. The characteristic spectrum of the thermogravimetric curve of the tobacco leaf samples in the formula and the characteristic spectrum of the thermogravimetric curve of the tobacco leaf samples in stock were selected to form the "dynamic" quality characteristic matrix U of the tobacco leaf in the formula and the "dynamic" quality characteristic matrix V of the tobacco leaf in stock, respectively.

[0045] (6) The Mahalanobis distance d2 of matrices U and V is calculated and normalized to obtain the similarity value a2 of the “dynamic quality” between the tobacco leaves in the formula and the tobacco leaves in stock. The results are shown in Table 6.

[0046] Table 6 Similarity values ​​a2 of the “dynamic” quality feature fusion matrix of tobacco leaves in the recipe and tobacco leaves in stock

[0047] (7) For each stock tobacco leaf’s “static” quality feature similarity value a1 and “dynamic” quality feature similarity value a2, a decision fusion method based on fuzzy comprehensive evaluation is used to calculate the final judgment value r of whether the stock tobacco leaf can replace the tobacco leaf in the formula, and the formula replacement risk probability 1-r is given for the replaceable tobacco leaf. A list of whether all stock tobacco leaves can be substituted for the formula is given, and the list is sorted from large to small according to the r value and recommended to the formula maker, as shown in Table 7. That is: substitute a1 and a2 into the formula r=0.3*f(a1)+0.7*f(a2) for calculation. If r>0.5, the stock tobacco leaf can replace the tobacco leaf in the cigarette formula, where the replacement risk probability is 1-r; if r≤0.5, the stock tobacco leaf cannot replace the tobacco leaf in the cigarette formula; where f(a) is the set membership function:

[0048] Table 7: List of whether tobacco leaves in stock can be substituted for formulation

Claims

1. A tobacco leaf substitution judgment method based on multi-modal hybrid fusion, characterized in that: The steps include: (1) The intrinsic chemical component index data and near-infrared characteristic spectral data of tobacco leaves in cigarette formula are fused using a multimodal feature fusion algorithm based on a convolutional neural network to obtain a "static" quality feature matrix X of tobacco leaves in cigarette formula; the intrinsic chemical component index data and near-infrared characteristic spectral data of different stock tobacco leaves are fused using a multimodal feature fusion algorithm based on a convolutional neural network to obtain a "static" quality feature matrix Y of stock tobacco leaves; (2) Calculate the Mahalanobis distance d1 between matrices X and Y and perform normalization to obtain the "static" quality similarity a1 between the tobacco leaves in the formula and the tobacco leaves in stock; (3) According to the main heat transfer weight loss stages during tobacco leaf combustion, the characteristic spectrum segments of the tobacco leaf thermogravimetric curve are screened to form the "dynamic" quality characteristic matrix U of tobacco leaves in cigarette formula and the "dynamic" quality characteristic matrix V of different stock tobacco leaves; (4) Calculate the Mahalanobis distance d2 of the matrices U and V, and perform normalization processing to obtain the "dynamic" quality similarity value a2 between the tobacco leaves in the formula and the tobacco leaves in stock; (5) Based on the similarity value a1 of the "static" quality characteristics of tobacco leaves and the similarity value a2 of the "dynamic" quality characteristics of tobacco leaves, it is determined whether the tobacco leaves in the stock can be substituted for the tobacco leaves in the formula, as well as the probability of substitution risk.

2. The tobacco leaf substitution judgment method based on multi-modal hybrid fusion according to claim 1 is characterized in that: The chemical composition index data of tobacco leaves selected in step (1) are 70 kinds, and the near-infrared characteristic spectrum of tobacco leaves is: 4350-4450cm -1 , 4500-4700cm -1 , 4800-4900cm -1 , 5050-5100cm -1 , 5220-5300cm -1 , 6050-6250cm -1 , 6400-6600cm -1 , 6950-7220cm -1 .

3. The tobacco leaf substitution judgment method based on multi-modal hybrid fusion according to claim 2 is characterized in that: The steps of fusion using the multimodal feature fusion algorithm based on convolutional neural network are as follows: first, the intrinsic chemical composition index fully connected layer f m and the near-infrared characteristic spectrum fully connected layer f n Multiply them together to form a two-dimensional feature fusion transformation matrix A m×n ; Then, for A m×n Perform maximum pooling to form m feature nodes as the new fusion feature layer B m ; Finally, apply the softmax layer to B m Logarithmic normalization is performed to compress the variables and classify them between [0,1], so as to obtain the final fusion feature layer; among them, the fusion feature matrix X of tobacco leaves in the formula and the fusion feature matrix Y of tobacco leaves in stock.

4. The tobacco leaf substitution judgment method based on multi-modal hybrid fusion according to claim 1 is characterized in that: Step (2) The calculation formula of the Mahalanobis distance value d1 is: Among them, X is the "static" quality fusion feature matrix of tobacco leaves in cigarette formula, Y is the "static" quality fusion feature matrix of stock tobacco leaves, and Σ is the covariance matrix of X and Y; the normalization formula is:

5. The tobacco leaf substitution judgment method based on multi-modal hybrid fusion according to claim 1 is characterized in that: The characteristic spectrum of the thermogravimetric curve of the tobacco leaf in step (3) is: 190°C-240°C, 320°C-360°C.

6. The tobacco leaf substitution judgment method based on multi-modal hybrid fusion according to claim 1 is characterized in that: Step (4) Mahalanobis distance calculation formula: Among them, U is the "dynamic" quality characteristic matrix of tobacco leaves in cigarette formula, V is the "dynamic" quality characteristic matrix of tobacco leaves in stock, and Σ is the covariance matrix of U and V; the normalization formula is:

7. The tobacco leaf substitution judgment method based on multi-modal hybrid fusion according to claim 1 is characterized in that: The judgment method of step (5) is: substitute a1 and a2 into the formula r=0.3*f(a1)+0.7*f(a2) for calculation. If r>0.5, the stock tobacco leaves can replace the tobacco leaves in the cigarette formula, where the replacement risk probability is 1-r; if r≤0.5, the stock tobacco leaves cannot replace the tobacco leaves in the cigarette formula; where f(a) is:

Images