Baked tobacco leaf state prediction method and system based on space-time sequence algorithm

Abstract

The invention discloses a flue-cured tobacco leaf state prediction method and system based on a space-time sequence algorithm, and relates to the technical field of tobacco production. The flue-cured tobacco leaf state prediction method based on the space-time sequence algorithm comprises the following steps: carrying out color calibration on a flue-cured tobacco leaf image by adopting black field calibration and white balance calibration; performing data set segmentation on the tobacco leaf image after color calibration according to a preset proportion, and performing data cleaning on the segmented data set to generate a baking image database; constructing a baking image data set with time sequence features based on the optimized baking image database; and utilizing a space-time sequence prediction algorithm. According to the invention, in the baking process, technicians need to continuously observe the form changes such as yellowing and water loss of tobacco leaves through an observation window, and dynamically control the operations such as temperature rise, temperature stabilization and moisture removal; the mode depends on subjective experience of visual perception of baking personnel, has relatively large subjectivity and limitation, and is difficult to ensure accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of tobacco production, and particularly to a method and system for predicting the state of baked tobacco leaves based on a spatio-temporal sequence algorithm. Background Art

[0002] The baking of tobacco leaves is a key link in tobacco leaf production, and its quality directly affects the final quality of tobacco leaves. As the main place for tobacco leaf baking, the bulk curing barn is equipped with a baking controller as the core automatic control device. The precise control of the baking process is crucial. As the industry proverb goes: "Well-baked tobacco leaves are treasures in the barn, while poorly-baked ones are just a pile of grass." The yellowing degree and the degree of water loss of tobacco leaves are the "golden yardsticks" for measuring the quality of the baking process. At present, the baking of tobacco leaves mainly adopts a three-stage process in the bulk curing barn, including the yellowing stage, the color-fixing stage, and the stem-drying stage, which can be further divided into the initial yellowing period, the yellowing period, the yellowing and withering period, the vein-changing period, the dry leaf period, the stem-drying period, etc. During the baking process, the state of tobacco leaves can be quantitatively evaluated through three core indicators: the yellowing degree (7 levels), the water loss degree (9 levels), and the browning degree (6 levels).

[0003] In the existing tobacco leaf baking process, the automatic control mainly relies on the controller to collect the temperature and humidity in the curing barn and combine with the pre-set baking curve, supplemented by the adjustment of baking process parameters by baking technicians according to the changes of tobacco leaves. The baking process parameters include parameters such as heating time, constant temperature time, heating rate, and constant temperature. During the baking process, technicians need to continuously observe the morphological changes such as yellowing and water loss of tobacco leaves through the observation window and dynamically control operations such as heating, constant temperature, and moisture exhaust. This mode relies on the subjective experience of baking personnel's visual perception, has great subjectivity and limitations, and it is difficult to ensure accuracy. Summary of the Invention

[0004] Object of the Invention: To provide a method and system for predicting the state of baked tobacco leaves based on a spatio-temporal sequence algorithm to at least solve one of the problems existing in the above-mentioned prior art.

[0005] Technical Solution: A method for predicting the state of baked tobacco leaves based on a spatio-temporal sequence algorithm includes:

[0006] Performing color calibration on the tobacco leaf images during baking by using black field calibration and white balance calibration;

[0007] Splitting the dataset of the color-calibrated tobacco leaf images according to a preset ratio, and performing data cleaning on the split dataset to generate a baking image database;

[0008] Based on the optimized baking image database, constructing a baking image dataset with time series characteristics;

[0009] Using a spatio-temporal sequence prediction algorithm and inputting the tobacco leaf baking images collected at the current moment to generate the baking state images at future moments, so as to visually predict the tobacco leaf baking process.

[0010] Preferably, black field calibration and white balance calibration are used to perform color calibration on the tobacco leaf images during baking, including:

[0011] By pre-calibrating the black field data of the sensor at different temperatures and integrating the temperature and humidity sensors, the black field correction is automatically linked according to the temperature data to ensure the stability of color restoration at different temperatures;

[0012] Use the Spydercolor24 standard 24-color colorimetric card for camera white balance calibration, automatically track the green light spectral response curve, and call the corresponding white balance parameters in real time to ensure the color restoration of the images collected during the entire tobacco leaf baking process.

[0013] Preferably, the tobacco leaf images after color calibration are segmented into a dataset according to a preset ratio, and the segmented dataset is cleaned to generate a baking image database, including:

[0014] Randomly segment the baking time series dataset into a training set, a validation set and a test set according to the ratio of 7:2:1, and scale each image to a preset size;

[0015] Check the image monitoring data of each furnace, and eliminate abnormal images affected by camera abnormalities, no tobacco leaves after the baking room door is opened, personnel operations after the baking room door is opened, and light. Eliminate the tobacco leaf images at the end of the previous furnace baking included in the current furnace image due to inaccurate recognition of the furnace start point.

[0016] Preferably, based on the optimized baking image database, a baking image dataset with time series characteristics is constructed, including:

[0017] During the whole process of each baking furnace, the cameras installed in the baking room continuously collect flue-cured tobacco images at a preset frequency, synchronously collect temperature and humidity data, time information and baking room furnace information, and upload them to the database for storage to construct a baking image database;

[0018] Extract the complete baking data of one furnace from the baking image database, and extract 20 images in chronological order at a sampling frequency of one image every 7 images. The time interval between images is 7 * 20 minutes. The 20 images form 1 sample data of the spatio-temporal sequence dataset; among them, each sample data contains: 10 images as Input and 10 images as GroundTruth, and the time span of Input and Ground Truth is 7 * 20 minutes * 10;

[0019] The above sampling process is performed in a sliding window manner with a sampling step size of 1 and a sliding window width of 7*19, and samples with less than 20 images at the end are discarded;

[0020] The baked time series dataset is organized into a Moving MNIST dataset structure to construct a time series image dataset. The Moving MNIST dataset structure includes: clips.npy, dims.npy, and input_raw_data.npy files. Among them, clips.npy contains the image location information of the input frame and ground truth frame, dims.npy contains the size information of a single image, and input_raw_data.npy contains the image data of each frame, stored in numpy.ndarray type.

[0021] As a preference, the baked time series dataset is organized into the Moving MNIST dataset structure. After constructing the time series image dataset, it also includes:

[0022] Normalize and standardize the baked time series dataset of the Moving MNIST dataset structure to accelerate model convergence;

[0023] The accumulated temperature and humidity data corresponding to the images at 20 time points in each sample are collected to form temperature and humidity sequence data, which corresponds one-to-one to the image sequence.

[0024] Preferably, a spatiotemporal sequence prediction algorithm is used, and tobacco leaf baking images collected at the current moment are input to generate baking state images at future moments, so as to enable a visual prediction of the tobacco leaf baking process, including:

[0025] When the Encoder module extracts spatial features, it incorporates the current temperature and humidity. t (t∈[1,2,...,10]), the accumulated temperature and humidity data [∑T t ,∑H t ] as auxiliary input, and extract temperature and humidity data [∑T t ,∑H t ] as a condition to generate the normalized parameters of the current time step t, including the scaling parameter γ t and the translation parameter β t , the normalization parameter γ at the same time step t t and β t Shared in all Encoder layers;

[0026] Before modeling the time evolution in the Translator module, fuse the trends of historical temperature and humidity sequences, and explicitly introduce the temperature and humidity trend data by adding FiLM layers; after extracting the feature map x, the dimension of x is (T, C, H, W), where T represents the time step, C represents the channel, and H and W represent the height and width of the feature map respectively. The Translator module convolves T×C channels on (H, W). The temperature and humidity sequence data are two scalar values at each time step, and the data dimension is (T, 2). Before the convolution operation of the Translator module, add FiLM layers;

[0027] When generating future frames in the Decoder module, inject the predicted accumulated temperature / humidity conditions at future times, and encode the accumulated temperature and humidity sequences at each future time step into a temporal conditional vector c through an MLP t (t ∈ [11, 12,..., 20]), use c when generating the future frame at time step t t Generate independent scaling parameter γt and translation parameter βt for each predicted time step t. The temporal conditional vector at the same predicted time step t is shared among all Decoder layers to ensure that the influence of temperature and humidity on the image state change rate conforms to global consistency;

[0028] Based on the SimVP algorithm, modify the algorithm by introducing environmental temperature and humidity information through conditional normalization and FiLM layers, and construct a future frame prediction model based on temperature and humidity and spatio-temporal sequence image data.

[0029] Preferably, after constructing a future frame prediction model based on the SimVP algorithm by introducing environmental temperature and humidity information through conditional normalization and FiLM layers, it further includes:

[0030] Train an auxiliary temporal LSTM model, input the historical temperature and humidity sequences and the tobacco leaf state values, predict the accumulated temperature ∑T and accumulated humidity ∑H at each future time, and use them as the conditional information for the Decoder module.

[0031] Preferably, after using the spatio-temporal sequence prediction algorithm and inputting the tobacco leaf baking image collected at the current moment to generate the baking state image at the future moment to visualize the prediction of the tobacco leaf baking process, it further includes:

[0032] Input the predicted future baking image into an image classification model based on Vision Transformer to accurately distinguish the key indicators of the yellowing degree, water loss degree, and browning degree of the tobacco leaves.

[0033] Preferably, the predicted future baking images are input into an image classification model based on Vision Transformer to accurately distinguish the key indicators of the yellowing degree, water loss degree, and browning degree of tobacco leaves, including:

[0034] In the model training step, the model generated 10 frames of sequence images at future time t 11 -t 20 For each generated image, the image classification model for identifying the state of baked tobacco leaves is called to identify the yellowing state level, water loss state level, and browning state level of the tobacco leaves. The top 2 levels with the highest confidence probability are output for each state level, indicating that the state is between two levels;

[0035] For each real image, the image classification model for identifying the state of baked tobacco leaves is also called;

[0036] Compare the tobacco leaf state identified from the real image with the tobacco leaf state identified from the generated image frame by frame, and calculate the state value similarity S; if the top 2 levels or top 1 level are the same, S is recorded as 1; if 1 of the top 2 levels overlaps, S is recorded as 0.5; if the top 2 levels are completely different, S is recorded as 0. Calculate the average value of S for all test set samples at each time step.

[0037] To achieve the above object, according to another aspect of the present application, a baking tobacco leaf state prediction system based on a spatio-temporal sequence algorithm is provided.

[0038] The baking tobacco leaf state prediction system based on the spatio-temporal sequence algorithm according to the present application includes:

[0039] A calibration module for color calibrating the tobacco leaf images during baking by using black field calibration and white balance calibration;

[0040] A splitting and cleaning module for splitting the dataset of the color-calibrated tobacco leaf images according to a preset ratio and cleaning the split dataset to generate a baking image database;

[0041] A construction module for constructing a baking image dataset with time series characteristics based on the optimized baking image database;

[0042] A prediction module for using a spatio-temporal sequence prediction algorithm and inputting the tobacco leaf baking image collected at the current moment to generate a baking state image at a future moment to enable visual prediction of the tobacco leaf baking process.

[0043] To achieve the above object, according to another aspect of the present application, there is provided an electronic device, the electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the method for predicting the state of tobacco leaves during baking based on the spatio-temporal sequence algorithm according to any one of the present invention.

[0044] To achieve the above object, according to another aspect of the present application, there is provided a computer-readable storage medium, in which computer instructions are stored, and the computer instructions are used to implement the method for predicting the state of tobacco leaves during baking based on the spatio-temporal sequence algorithm according to any one of the present invention when executed by a processor.

[0045] Beneficial effects: In the embodiments of the present application, a method based on the spatio-temporal sequence algorithm is adopted. The color of the tobacco leaf image during baking is calibrated by using black field calibration and white balance calibration; the calibrated tobacco leaf image is segmented into a data set according to a preset ratio, and the segmented data set is cleaned to generate a baking image database; based on the optimized baking image database, a baking image data set with time series characteristics is constructed; the spatio-temporal sequence prediction algorithm is used, and the tobacco leaf baking image collected at the current moment is input to generate the baking state image at a future moment, so as to visually predict the tobacco leaf baking process, achieving the purpose of predicting the tobacco leaf baking state at a future moment, thereby realizing the technical effects of providing forward-looking guidance for process parameter adjustment and improving the quality and efficiency of tobacco leaf baking, and further solving the problems of the existing tobacco leaf baking process, which mainly uses a controller to collect the temperature and humidity of the baking room and automatically control it in combination with a preset baking curve, and supplemented by baking technicians to adjust the baking process parameters according to the changes of tobacco leaves. The baking process parameters include parameters such as heating time, constant temperature time, heating rate, and constant temperature temperature; during the baking process, technicians need to continuously observe the morphological changes such as yellowing and water loss of tobacco leaves through the observation window, and dynamically control operations such as heating, constant temperature, and moisture discharge; this mode relies on the subjective experience of baking personnel's visual perception, has great subjectivity and limitations, and is difficult to ensure accuracy. Description of the Drawings

[0046] Figure 1 is the overall technical solution flowchart of the method for predicting the state of tobacco leaves during baking based on the spatio-temporal sequence algorithm according to the embodiments of the present application;

[0047] Figure 2 is the improved SimVP model structure diagram of the method for predicting the state of tobacco leaves during baking based on the spatio-temporal sequence algorithm according to the embodiments of the present application;

[0048] Figure 3It is an example of a visualization prediction sequence diagram of a method for predicting the state of baked tobacco leaves based on a spatio-temporal sequence algorithm according to an embodiment of the present application;

[0049] Figure 4 It is a schematic flowchart of a method for predicting the state of baked tobacco leaves based on a spatio-temporal sequence algorithm according to an embodiment of the present application;

[0050] Figure 5 It is a schematic flowchart of another method for predicting the state of baked tobacco leaves based on a spatio-temporal sequence algorithm according to an embodiment of the present application;

[0051] Figure 6 It is a schematic structural diagram of a system for predicting the state of baked tobacco leaves based on a spatio-temporal sequence algorithm according to an embodiment of the present application; and

[0052] Figure 7 It is a schematic structural diagram of an electronic device for a method for predicting the state of baked tobacco leaves based on a spatio-temporal sequence algorithm according to an embodiment of the present application. Detailed implementation manners

[0053] In order to enable those skilled in the art to better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0054] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so as to implement the embodiments of the present application described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0055] In addition, the terms "install", "set", "provided with", "connected", "connected to", "socketed" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral structure; it can be a mechanical connection or an electrical connection; it can be directly connected, or indirectly connected through an intermediate medium, or there is internal communication between two devices, components or parts. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0056] It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0057] According to an embodiment of the present invention, a method for predicting the state of baked tobacco leaves based on a spatio-temporal sequence algorithm is provided. As Figure 4-5 shown, the method includes the following steps S101 to S105:

[0058] Step S101: Perform color calibration on the tobacco leaf images during baking using black field calibration and white balance calibration;

[0059] Good image color calibration effects can be achieved, thereby providing a basis and guarantee for subsequent accurate data judgment.

[0060] According to an embodiment of the present invention, preferably, performing color calibration on the tobacco leaf images during baking using black field calibration and white balance calibration includes:

[0061] By pre-calibrating the black field data of the sensor at different temperatures and integrating a temperature and humidity sensor at the same time, the black field correction is automatically linked according to the temperature data to ensure the stability of color restoration at different temperatures;

[0062] Use the Spydercolor24 standard 24-color colorimetric card to perform camera white balance calibration, automatically track the green light spectral response curve, and call the corresponding white balance parameters in real time to ensure the color restoration degree of the images collected during the entire tobacco leaf baking process.

[0063] Specifically, to solve the problem that during the baking process, the temperature changes greatly, the black field of the sensor itself will drift gradually with the increase of temperature, resulting in a large color cast phenomenon in the pictures, and the color cast is more serious at higher temperatures. By pre-calibrating the black field data of the sensor at different temperatures and integrating a temperature and humidity sensor in the device at the same time, the black field correction is automatically linked according to the temperature data to ensure the stability of color restoration at different temperatures.

[0064] Chlorophyll has a small reflection peak near 550nm. To solve the problem that during the baking process, as chlorophyll degrades and becomes inactivated, the spectral reflection characteristics of the tobacco leaves will mutate, resulting in serious color imbalance, use the Spydercolor24 standard 24-color colorimetric card to perform camera white balance calibration, automatically track the green light spectral response curve, and call the corresponding white balance parameters in real time to ensure the color restoration degree of the images collected during the entire tobacco leaf baking process.

[0065] Step S102: Split the dataset of the color-calibrated tobacco leaf images according to a preset ratio, and perform data cleaning on the split dataset to generate a baked image database;

[0066] It can achieve good dataset splitting effect, and at the same time clean the dataset, so as to ensure the accuracy of the data, and then generate an accurate baking image database.

[0067] According to an embodiment of the present invention, preferably, the tobacco leaf images after color calibration are split into datasets according to a preset ratio, and the split datasets are cleaned to generate a baking image database, including:

[0068] Randomly split the baking time series dataset into a training set, a validation set and a test set according to the ratio of 7:2:1, and scale each image to a preset size;

[0069] Check the image monitoring data of each furnace, and eliminate abnormal images affected by abnormal cameras, no tobacco leaves after the baking room door is opened, personnel operations after the baking room door is opened, and light. Eliminate the tobacco leaf images at the end of the previous furnace baking included in the current furnace image due to inaccurate recognition of the furnace start point.

[0070] Step S103: Based on the optimized baking image database, construct a baking image dataset with time series characteristics;

[0071] The required baking image dataset can be obtained, thereby providing a data basis for subsequent spatio-temporal sequence prediction algorithms.

[0072] According to an embodiment of the present invention, preferably, based on the optimized baking image database, construct a baking image dataset with time series characteristics, including:

[0073] During the whole process of each baking furnace, the cameras installed in the baking room continuously collect flue-cured tobacco images at a preset frequency, synchronously collect temperature and humidity data, time information and baking room furnace information, and upload them to the database for storage to construct a baking image database;

[0074] Extract the complete baking data of one furnace from the baking image database, and extract 20 images in chronological order at a sampling frequency of one image every 7 images. The time interval between images is 7 * 20 minutes. The 20 images form 1 sample data of the spatio-temporal sequence dataset; among them, each sample data contains: 10 images as Input and 10 images as GroundTruth, and the time span of Input and Ground Truth is 7 * 20 minutes * 10;

[0075] It should be noted that the time span is 7 * 20 minutes * 10, that is, about 23.3 hours, which is about one day; in the present invention, the sampling frequency is taken as 7, the number of Input frames and the number of Ground Truth frames are taken as 10, that is, there are 20 time points, denoted as t1 - t 20It is used to represent 20 time points, which can be flexibly adjusted according to requirements in actual applications. The selection of the sampling frequency depends on the rate of change of images at different time points.

[0076] Perform the above sampling process in a sliding window manner with a sampling step of 1 and a sliding window width of 7 * 19, and discard the samples with less than 20 images at the end.

[0077] It should be noted that, for example: after 20 images with serial numbers [1, 8, 17,..., 134] in a furnace of baking images form a sample, 20 images with serial numbers [2, 9, 18,..., 135] form the next sample, and so on.

[0078] Organize the baking time series dataset into the structure of the Moving MNIST dataset to construct a time series image dataset; the structure of the Moving MNIST dataset includes: clips.npy, dims.npy, and input_raw_data.npy files; among them, clips.npy contains the picture position information of the Input frame and the ground truth frame, dims.npy contains the size information of a single image, and input_raw_data.npy contains the picture data of each frame, and the storage type is numpy.ndarray.

[0079] According to the embodiment of the present invention, preferably, after organizing the baking time series dataset into the structure of the Moving MNIST dataset to construct a time series image dataset, it further includes;

[0080] Normalize and standardize the baking time series dataset with the structure of the Moving MNIST dataset to accelerate the convergence of the model.

[0081] Collect the accumulated temperature and accumulated humidity data corresponding to the images at 20 time points in each sample to form temperature and humidity sequence data, which corresponds one-to-one with the image sequence.

[0082] Step S104, use the spatio-temporal sequence prediction algorithm, and input the tobacco leaf baking image collected at the current moment to generate the baking state image at the future moment, so as to visually predict the tobacco leaf baking process.

[0083] It can achieve accurate prediction effects of tobacco leaf baking images, thereby ensuring good visualization effects.

[0084] Specifically, the SimVP model consists of an encoder, a translator, and a decoder. The encoder is used to extract spatial features, the translator learns temporal evolution, and the decoder is responsible for restoring the temporal features processed by the Translator into future frame data. The Encoder part stacks N s ConvNormReLU blocks (Conv2d + GroupNorm + LeakyReLU) to extract spatial features. The Translator part uses N t Inception modules to learn temporal evolution. The Decoder part uses N s unConvNormReLU blocks (ConvTranspose2d + GroupNorm + LeakyReLU) to reconstruct the Ground Truth frames. The SimVP model is completely based on CNN and is trained using an end-to-end MSE loss. Without introducing any additional tricks and complex strategies, it achieves SOTA results. At the same time, its low computational cost makes it easy to scale to more scenarios.

[0085] From the perspective of physical laws, temperature and humidity affect the rate of yellowing and water loss of flue-cured tobacco through cumulative effects (accumulated temperature, accumulated humidity), and their influence on the state of tobacco leaves is global rather than local. Therefore, the temperature and humidity sequence needs to be fused with the image sequence as the model input. In order to build a future frame prediction model based on temperature, humidity, and spatio-temporal sequence image data, and to more effectively extract the effective information in the flue-curing spatio-temporal sequence dataset, the present invention makes structural modifications to the SimVP algorithm. When the Encoder module extracts spatial features, the temperature and humidity at the current moment are incorporated. When the Translator module models temporal evolution, the trend of the historical temperature and humidity sequence is fused. When the Decoder module generates future frames, the predicted accumulated temperature / accumulated humidity conditions at future moments are injected. To achieve visual prediction of the future state of tobacco leaves during the baking process. The improved model structure is as Figure 2 shown.

[0086] According to an embodiment of the present invention, preferably, a spatio-temporal sequence prediction algorithm is used, and the flue-cured tobacco baking image collected at the current moment is input to generate the flue-cured tobacco baking state image at a future moment, so as to realize visual prediction of the flue-cured tobacco baking process, including:

[0087] When the Encoder module extracts spatial features, the temperature and humidity at the current moment are incorporated. For each input image I t (t ∈ [1, 2,..., 10]), the cumulative temperature and humidity data [∑T t , ∑H t at the time of image acquisition at the current time step t is used as auxiliary input, and the temperature and humidity data [∑Tt , ∑H t Using the information in [] as a condition, generate the normalization parameters for the current time step t, including the scaling parameter γ t and the translation parameter β t , the normalization parameters γ t and β t for the same time step t are shared across all Encoder layers;

[0088] It should be noted that the original GroupNorm normalization performs group normalization in the time dimension, but the normalization parameters for each time step are network layer parameters learned through backpropagation during the training process, and these parameters do not reflect the environmental temperature and humidity differences during image acquisition at each time step. Therefore, replace the GroupNorm in the original ConvNormReLU block with Conditional GroupNorm, abbreviated as C-GroupNorm.

[0089] Fuse the trend of the historical temperature and humidity sequence before the Translator module models the time evolution, and explicitly introduce the temperature and humidity trend data by adding a FiLM layer; after extracting the feature map x, the dimension of x is (T, C, H, W), where T represents the time step, C represents the channel, and H and W represent the height and width of the feature map respectively. The Translator module convolves T×C channels on (H, W). The temperature and humidity sequence data is two scalar values for each time step, and the data dimension is (T, 2). Before the convolution operation of the Translator module, add a FiLM layer;

[0090] Inject the predicted accumulated temperature / humidity conditions at future times when the Decoder module generates future frames, and encode the accumulated temperature and humidity sequences at future time steps into a temporal conditional vector c t (t ∈ [11, 12,..., 20]), and use c when generating the future frame at time step t t Generate independent scaling parameter γt and translation parameter βt for each predicted time step t. The temporal conditional vector for the same predicted time step t is shared across all Decoder layers to ensure that the influence of temperature and humidity on the image state change rate conforms to global consistency;

[0091] It should be noted that for the problem of blurry generated images, since the transposed convolution layer is prone to introducing checkerboard artifacts, resulting in blurry generated images and loss of texture details, and the Decoder module is stacked by transposed convolution modules, therefore, a combination of transposed convolution and sub-pixel convolution is used to replace the transposed convolution layer to reduce the checkerboard effect.

[0092] The original GroupNorm in the Decoder module performs group normalization in the time dimension. However, the normalization parameters for each time step are network layer parameters learned through backpropagation during the training process, and these parameters do not reflect the differences in accumulated temperature and humidity of images at future time steps. Therefore, the GroupNorm in the Decoder module is replaced with Conditional GroupNorm.

[0093] Based on the SimVP algorithm, the algorithm is modified by introducing environmental temperature and humidity information through conditional normalization and the FiLM layer to construct a future frame prediction model based on temperature and humidity and spatio-temporal sequence image data.

[0094] Specifically, the model input is a sequence of 10 frames of images from time t1 to t 10 moments, and the auxiliary input is the temperature and humidity sequence from time t1 to t 10 moments. The output is a sequence of 10 frames of generated images from time t 11 -t 20 moments, and the output is visualized. An example is shown Figure 3 as follows. The generated future moment images are compared with the ground truth frame images, and the SSIM (structural similarity) and MSE (mean squared error) are calculated to evaluate the model performance.

[0095] According to an embodiment of the present invention, preferably, after constructing a future frame prediction model based on temperature and humidity and spatio-temporal sequence image data by modifying the algorithm through conditional normalization and the FiLM layer based on the SimVP algorithm, the following is further included:

[0096] Train an auxiliary temporal LSTM model. Input the historical temperature and humidity sequence and the tobacco leaf state value to predict the accumulated temperature ∑T and accumulated humidity ∑H at future moments, and use them as the conditional information for the Decoder module.

[0097] Specifically, in the model inference stage, the accumulated temperature ∑T and accumulated humidity ∑H at future moments can also accept manual input from front-end baking technicians. Technicians can set different temperature and humidity combinations to view the future frame output and select the most suitable temperature and humidity combination to assist in adjusting the baking process parameters. The model is deployed based on the centOS system, and the flask framework is used to deploy the algorithm service to provide access interfaces for mobile and web applications.

[0098] After step S105, using the spatio-temporal sequence prediction algorithm and inputting the tobacco leaf baking image collected at the current moment to generate the baking state image at the future moment to visualize the prediction of the tobacco leaf baking process, the following is further included:

[0099] Input the predicted future-time baking image into the Vision Transformer-based image classification model to accurately determine the key indicators of the yellowing degree, water loss degree, and browning degree of tobacco leaves.

[0100] It can achieve the effect of accurately determining the baking state of tobacco leaves, thus ensuring that the tobacco leaves are in a good processing state.

[0101] According to an embodiment of the present invention, preferably, inputting the predicted future-time baking image into the Vision Transformer-based image classification model to accurately determine the key indicators of the yellowing degree, water loss degree, and browning degree of tobacco leaves includes:

[0102] In the model training step, the model generates a sequence of 10 frames of images at future time t 11 -t 20 For each generated image, call the image classification model for identifying the baking state of tobacco leaves to identify the yellowing state level, water loss state level, and browning state level of the tobacco leaves. For each state level, output the top 2 levels with the highest confidence probability, indicating that the state is between two levels;

[0103] For each real image, also call the image classification model for identifying the baking state of tobacco leaves;

[0104] Compare frame by frame the tobacco leaf state identified from the real image with the tobacco leaf state identified from the generated image, and calculate the state value similarity S; if the top 2 levels or the top 1 level are the same, S is recorded as 1; if one of the top 2 levels overlaps, S is recorded as 0.5; if the top 2 levels are completely different, S is recorded as 0. Respectively, calculate the average value of S for all test set samples at each time step.

[0105] From the above description, it can be seen that the present application has achieved the following technical effects:

[0106] In the embodiments of the present application, a method based on a spatio-temporal sequence algorithm is adopted to perform color calibration on the tobacco leaf images during baking by using black field calibration and white balance calibration; the tobacco leaf images after color calibration are segmented into a data set according to a preset ratio, and the segmented data set is cleaned to generate a baking image database; based on the optimized baking image database, a baking image data set with time series characteristics is constructed; the spatio-temporal sequence prediction algorithm is used, and the tobacco leaf baking image collected at the current moment is input to generate a baking state image at a future moment, so as to visually predict the tobacco leaf baking process, achieving the purpose of predicting the tobacco leaf baking state at a future moment, thereby realizing the technical effects of providing forward-looking guidance for process parameter adjustment and improving the quality and efficiency of tobacco leaf baking, and further solving the problems of the existing tobacco leaf baking process, which mainly automatically regulates by the controller collecting the temperature and humidity of the baking room and combining with a preset baking curve, and supplemented by the baking technicians adjusting the baking process parameters according to the changes of the tobacco leaves. The baking process parameters include parameters such as heating time, constant temperature time, heating rate, and constant temperature temperature; during the baking process, technicians need to continuously observe the morphological changes such as yellowing and water loss of the tobacco leaves through the observation window, and dynamically control operations such as heating, constant temperature, and moisture exhaust; this mode relies on the subjective experience of the baking personnel's visual perception, has great subjectivity and limitations, and is difficult to ensure accuracy.

[0107] The present invention has the following beneficial effects:

[0108] Key point 1: Construct a large-scale self-owned database. During the baking process, baking monitoring data is collected in real time through Internet of Things collection devices and data gateways, including: dry and wet bulb temperature data of the tobacco leaf baking microenvironment, video image monitoring data, and information such as baking region, baking room number, furnace number, and image acquisition time is synchronously recorded, constructing a complete baking spatio-temporal sequence database.

[0109] Key point 2: Visually predict future frames based on the spatio-temporal sequence algorithm. Compared with the traditional classification model based on a single image, the present invention uses the spatio-temporal sequence algorithm to model the spatio-temporal law of the changes of tobacco leaves during the baking process and estimate the change trend of tobacco leaves in the next period of time. This method not only considers the spatial characteristics of a single frame of image, but also fully explores the continuity in the time dimension.

[0110] Key point 3: Multi-source data modeling integrating environmental temperature and humidity information. Aiming at the influence of environmental temperature and humidity on the change rate of the future state of tobacco leaves, the present invention is based on the fully convolutional network architecture of the SimVP model. By introducing conditional normalization and FiLM layers, the image data is deeply fused with the environmental temperature and humidity information. Compared with a single data source, the fusion of multi-source data helps the model to make more accurate judgments.

[0111] Key point 4: Visualization combined with intelligent recognition comprehensively displays the tobacco leaf baking state at a future moment. Based on the tobacco leaf images at future moments output by the spatio-temporal sequence model, the tobacco leaf state recognition model is further called to recognize the yellowing state level, water loss state level, and browning state level of the tobacco leaf state at future moments, realizing the comprehensive display of "visualization-generated images + intelligent state recognition", and providing an intuitive and objective basis for baking process adjustment.

[0112] Key point 5: Support process parameter optimization decision-making. After the model is deployed, it supports front-end baking technicians to simulate and input different combinations of temperature and humidity sequences at future moments. According to the model output results, the optimal future temperature and humidity combination is comprehensively selected to guide the baking process parameter adjustment strategy. This function helps to improve the intelligent level of the baking process and reduce the baking losses caused by untimely parameter adjustment.

[0113] As Figure 6 shown, to achieve the above object, according to another aspect of the present application, a baking tobacco leaf state prediction system based on a spatio-temporal sequence algorithm is provided. The baking tobacco leaf state prediction system based on the spatio-temporal sequence algorithm includes:

[0114] A calibration module for color calibrating the tobacco leaf images during baking by using black field calibration and white balance calibration;

[0115] It can achieve good image color calibration effects, thus providing a basis and guarantee for subsequent accurate data judgment.

[0116] A segmentation and cleaning module for segmenting the dataset of the color-calibrated tobacco leaf images according to a preset ratio, and cleaning the segmented dataset to generate a baking image database;

[0117] It can achieve good dataset segmentation effects, and at the same time clean the dataset, thus ensuring the accuracy of the data, and further generating an accurate baking image database.

[0118] A construction module for constructing a baking image dataset with time series characteristics based on the optimized baking image database;

[0119] It can obtain the required baking image dataset, thus providing a data basis for subsequent spatio-temporal sequence prediction algorithms.

[0120] A prediction module for using the spatio-temporal sequence prediction algorithm and inputting the tobacco leaf baking image collected at the current moment to generate a baking state image at a future moment, so as to visually predict the tobacco leaf baking process.

[0121] It can achieve accurate tobacco leaf baking image prediction effects, thus ensuring good visualization effects.

[0122] As can be seen from the above description, the present application achieves the following technical effects:

[0123] In the embodiment of the present application, a method based on a spatio-temporal sequence algorithm is adopted. The color of the tobacco leaf image during baking is calibrated by using black field calibration and white balance calibration. The calibrated tobacco leaf image is segmented into a data set according to a preset ratio, and the segmented data set is cleaned to generate a baking image database. Based on the optimized baking image database, a baking image data set with time series characteristics is constructed. By using the spatio-temporal sequence prediction algorithm and inputting the tobacco leaf baking image collected at the current moment, a baking state image at a future moment is generated, so as to visually predict the tobacco leaf baking process, achieving the purpose of predicting the tobacco leaf baking state at a future moment. Thus, the technical effects of providing forward-looking guidance for process parameter adjustment and improving the quality and efficiency of tobacco leaf baking are realized. Furthermore, it solves the technical problem of the existing tobacco leaf baking process, which mainly automatically controls the temperature and humidity of the baking room collected by a controller in combination with a preset baking curve, and supplemented by baking technicians adjusting the baking process parameters according to the changes of tobacco leaves. The baking process parameters include parameters such as heating time, constant temperature time, heating rate, and constant temperature. During the baking process, technicians need to continuously observe the morphological changes such as yellowing and water loss of tobacco leaves through the observation window and dynamically control operations such as heating, constant temperature, and moisture discharge. This mode relies on the subjective experience of baking personnel's visual perception, has great subjectivity and limitations, and is difficult to ensure accuracy.

[0124] As Figure 7 shown, the electronic device 10 includes at least one processor 11 and a memory communicatively connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc. Among them, the memory stores a computer program executable by the at least one processor. The processor 11 can execute various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. The input / output (I / O) interface 15 is also connected to the bus 14.

[0125] Multiple components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disc, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0126] The processor 11 may be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the method for predicting the state of flue-cured tobacco based on the spatio-temporal sequence algorithm.

[0127] In some embodiments, the method for predicting the state of flue-cured tobacco based on the spatio-temporal sequence algorithm may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method for predicting the state of flue-cured tobacco based on the spatio-temporal sequence algorithm described above may be executed. Alternatively, in other embodiments, the processor 11 may be configured to execute the method for predicting the state of flue-cured tobacco based on the spatio-temporal sequence algorithm in any other suitable manner (e.g., by means of firmware).

[0128] Various embodiments of the systems and techniques described above in this document may be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, and may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0129] A computer program for implementing the method of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, a special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions / operations specified in the flowchart and / or block diagram to be implemented. The computer programs may be executed entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine, or entirely on the remote machine or server.

[0130] In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, the computer-readable storage medium may be a machine-readable signal medium. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0131] In order to provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0132] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), blockchain network, and the Internet.

[0133] The computing system can include a client and a server. The client and the server are generally remote from each other and typically interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system, solving the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services.

[0134] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in the present invention can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved, and this is not limited herein.

[0135] The above specific embodiments do not constitute a limitation on the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for predicting the state of baked tobacco leaves based on a spatio-temporal sequence algorithm, characterized in that, Including: Performing color calibration on the tobacco leaf images during baking by using black field calibration and white balance calibration; Splitting the dataset of the tobacco leaf images after color calibration according to a preset ratio, and cleaning the split dataset to generate a baking image database; Based on the optimized baking image database, constructing a baking image dataset with time series characteristics; Using a spatio-temporal sequence prediction algorithm and inputting the tobacco leaf baking images collected at the current moment to generate the baking state images at future moments, so as to visually predict the tobacco leaf baking process.

2. The method for predicting the state of baked tobacco leaves based on the spatio-temporal sequence algorithm according to claim 1, wherein, Performing color calibration on the tobacco leaf images during baking by using black field calibration and white balance calibration, including: By pre-calibrating the black field data of the sensor at different temperatures and integrating a temperature and humidity sensor, automatically linking the black field correction according to the temperature data to ensure the stability of color restoration at different temperatures; Using a Spydercolor24 standard 24-color colorimetric card for camera white balance calibration, automatically tracking the green light spectral response curve, and calling the corresponding white balance parameters in real time to ensure the color restoration degree of the images collected during the whole tobacco leaf baking process.

3. The method for predicting the state of baked tobacco leaves based on the spatio-temporal sequence algorithm according to claim 1, wherein Splitting the dataset of the tobacco leaf images after color calibration according to a preset ratio, and cleaning the split dataset to generate a baking image database, including: Randomly splitting the baking time series dataset into a training set, a validation set and a test set according to a ratio of 7:2:1, and scaling each image to a preset size; Checking the image monitoring data of each furnace, removing abnormal images affected by abnormal cameras, no tobacco leaves after the baking room door is opened, personnel operations after the baking room door is opened and light, and removing the tobacco leaf images at the end of the previous furnace baking included in the current furnace image due to inaccurate identification of the furnace starting point.

4. The method for predicting the state of baked tobacco leaves based on the spatio-temporal sequence algorithm according to claim 1, wherein, Based on the optimized baking image database, constructing a baking image dataset with time series characteristics, including: During the whole process of each baking furnace, the cameras installed in the baking room continuously collect flue-cured tobacco images at a preset frequency, synchronously collect temperature and humidity data, time information and baking room furnace information, and upload them to the database for storage to construct a baking image database; Extracting a complete set of baking data from the baking image database, and extracting 20 images in chronological order at a sampling frequency of extracting one image every 7 images. The time interval between the images is 7 * 20 minutes. The 20 images form 1 sample data of the spatio-temporal sequence dataset; among them, each sample data contains: 10 images as Input and 10 images as GroundTruth, and the time span of Input and Ground Truth is 7 * 20 minutes * 10; Performing the above sampling process in a sliding window manner with a sampling step of 1 and a sliding window width of 7 * 19, and discarding the samples with less than 20 images at the end. Organize the baking time-series dataset into the structure of the Moving MNIST dataset to construct a time-series image dataset; the structure of the Moving MNIST dataset includes: clips.npy, dims.npy, and input_raw_data.npy files; among them, clips.npy contains the picture position information of the Input frame and the ground truth frame, dims.npy contains the size information of a single image, and input_raw_data.npy contains the picture data of each frame, and the storage type is numpy.ndarray.

5. The method for predicting the state of baked tobacco leaves based on the spatio-temporal sequence algorithm according to claim 4, wherein After organizing the baking time-series dataset into the structure of the Moving MNIST dataset to construct a time-series image dataset, it also includes; Normalize and standardize the baking time-series dataset of the Moving MNIST dataset structure to accelerate the convergence of the model; Collect the accumulated temperature and accumulated humidity data corresponding to the images at 20 time points in each sample to form temperature and humidity sequence data, which correspond one-to-one with the image sequence.

6. The method for predicting the state of baked tobacco leaves based on the spatio-temporal sequence algorithm according to claim 1, characterized in that Use the spatio-temporal sequence prediction algorithm and input the tobacco leaf baking image collected at the current moment to generate the baking state image at the future moment to enable visual prediction of the tobacco leaf baking process, including: When extracting spatial features in the Encoder module, incorporate the current temperature and humidity, for each input image I t (t ∈ [1, 2,..., 10]), use the cumulative temperature and humidity data [∑T t , ∑H t at the time of image acquisition at the current time step t as auxiliary input, and use the information in the temperature and humidity data [∑T t , ∑H t extracted by the MLP as a condition to generate the normalization parameters at the current time step t, including the scaling parameter γ t and the translation parameter β t . The normalization parameters γ t and β t at the same time step t are shared across all Encoder layers; Before modeling the time evolution in the Translator module, fuse the trends of the historical temperature and humidity sequences, and explicitly introduce the temperature and humidity trend data by adding a FiLM layer; after extracting the feature map x, the dimension of x is (T, C, H, W), where T represents the time step, C represents the channel, and H and W represent the height and width of the feature map respectively. The Translator module convolves T×C channels on (H, W). The temperature and humidity sequence data are two scalar values at each time step, and the data dimension is (T, 2). Before the convolution operation of the Translator module, add a FiLM layer; Inject the predicted accumulated temperature / humidity conditions at future times when generating future frames in the Decoder module, and encode the accumulated temperature and humidity sequences at each future time step into a temporal condition vector c through MLP. t (t ∈ [11, 12,..., 20]), use c when generating the future frame at time step t. t Generate independent scaling parameter γt and translation parameter βt for each predicted time step t. The temporal condition vectors at the same predicted time step t are shared among all Decoder layers to ensure that the influence of temperature and humidity on the rate of change of the image state conforms to global consistency. Based on the SimVP algorithm, modify the algorithm by introducing environmental temperature and humidity information through conditional normalization and the FiLM layer, and construct a future frame prediction model based on temperature and humidity and spatio-temporal sequence image data.

7. The method for predicting the state of baked tobacco leaves based on the spatio-temporal sequence algorithm according to claim 6, characterized in that After modifying the algorithm by introducing environmental temperature and humidity information through conditional normalization and the FiLM layer based on the SimVP algorithm to construct a future frame prediction model based on temperature and humidity and spatio-temporal sequence image data, it also includes: Train an auxiliary time-series LSTM model, input the historical temperature and humidity sequences and the tobacco leaf state values, and predict the accumulated temperature ∑T and accumulated humidity ∑H at each future moment, and use them as the conditional information of the Decoder module.

8. The method for predicting the state of baked tobacco leaves based on the spatio-temporal sequence algorithm according to claim 6, wherein, After using the spatio-temporal sequence prediction algorithm and inputting the tobacco leaf baking image collected at the current moment to generate the baking state image at the future moment to enable visual prediction of the tobacco leaf baking process, it also includes: Input the baking image at the future moment predicted and generated into an image classification model based on Vision Transformer to accurately distinguish the key indicators of the yellowing degree, water loss degree, and browning degree of tobacco leaves.

9. The method for predicting the state of baked tobacco leaves based on the spatio-temporal sequence algorithm according to claim 8, characterized in that, Input the predicted future-time baking images into an image classification model based on Vision Transformer to accurately discriminate the key indicators of the yellowing degree, water loss degree, and browning degree of tobacco leaves. Including: In the model training step, the model generated a sequence of 10 frames of future t 11 -t 20 -moment sequence images. For each generated image, the image classification model for identifying the state of baked tobacco leaves was called to identify the yellowing state level, water loss state level, and browning state level of the tobacco leaves. The top 2 levels with the highest confidence probabilities were output for each state level, indicating that the state was between two levels; For each frame of real image, also call the image classification model for identifying the baking tobacco leaf state. Compare the tobacco leaf state identified from the real image with the tobacco leaf state identified from the generated image frame by frame, and calculate the state value similarity S. If the top2 levels or the top1 level are the same, S is recorded as 1; if there is 1 overlap in the top2 levels, S is recorded as 0.5; if the top2 levels are completely different, S is recorded as 0. Calculate the mean value of S for all test set samples at each time step respectively.

10. A baking tobacco leaf state prediction system based on a spatio-temporal sequence algorithm, characterized in that, Including: A calibration module for performing color calibration on the tobacco leaf images during baking using black field calibration and white balance calibration. A segmentation and cleaning module for segmenting the dataset of the color-calibrated tobacco leaf images according to a preset ratio and cleaning the segmented dataset to generate a baking image database. A construction module for constructing a baking image dataset with time series features based on the optimized baking image database. A prediction module for using a spatio-temporal sequence prediction algorithm and inputting the tobacco leaf baking images collected at the current moment to generate the baking state images at future moments to enable visual prediction of the tobacco leaf baking process.

Images