A tobacco primary processing and storage environment anomaly detection method, system and device

Abstract

The application discloses a tobacco primary processing and storage environment anomaly detection method, system and device, which can simultaneously process temperature, humidity, weight and visual information, comprehensively reflect the environment state of a storage cabinet, break through the limitation of traditional single sensor data isolation, and effectively reduce the false detection rate and the missed detection rate by constructing a unified multi-modal space-time tensor, extracting dynamic principal component features, introducing a cross-modal attention mechanism to fuse key features, performing anomaly scoring based on virtual dynamic features, and dynamically adjusting a determination threshold according to a storage stage and responding to different risk levels.

Description

Technical Field

[0001] This disclosure relates to the field of intelligent monitoring technology in warehousing, specifically to a method, system, and device for detecting abnormalities in the tobacco processing and storage environment. Background Technology

[0002] With the development of intelligent manufacturing and industrial Internet of Things (IoT) technologies, modern warehousing and storage facilities are increasingly demanding intelligent and refined environmental monitoring management. In scenarios where storage environments are sensitive, such as those for pharmaceuticals, food, and chemicals, real-time monitoring of key parameters such as temperature, humidity, weight, and spatial distribution, along with early identification and automatic response to potential risks, has become a crucial means of ensuring material quality and system safety.

[0003] Tobacco processing storage cabinets require strict monitoring of parameters such as temperature, humidity, and material distribution. Traditional error prevention methods rely on single sensor threshold judgments (such as temperature over-limit alarms), ignoring the spatiotemporal correlation of multimodal data (temperature, humidity, weight, images), resulting in high false positive and false negative rates. They cannot adapt to gradual environmental anomalies (such as local condensation or material mold), and manually set thresholds are difficult to cover complex operating conditions. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by proposing a method and device for detecting anomalies in the tobacco processing and storage environment. By constructing a unified multimodal spatiotemporal tensor, extracting dynamic principal component features, introducing a cross-modal attention mechanism to fuse key features, performing anomaly scoring based on virtual dynamic features, and dynamically adjusting the judgment threshold according to the storage stage and responding to different risk levels, this invention can simultaneously process temperature, humidity, weight, and visual information, comprehensively reflecting the environmental status of the storage cabinet. It overcomes the limitations of isolated data from traditional single sensors and effectively reduces the false detection rate and false negative rate.

[0005] The objective of this invention is achieved through the following technical solution: a method for detecting abnormalities in the tobacco processing and storage environment, comprising:

[0006] Collect storage index data of multiple modalities, and construct a multimodal spatiotemporal tensor based on the storage index data and the corresponding spatial location information;

[0007] Singular value decomposition is performed on the spatiotemporal tensor of each mode to extract principal components as low-dimensional dynamic features containing temporal evolution laws and spatial distribution patterns. The modal weights corresponding to the dynamic features of each mode are dynamically allocated through the mutual information entropy between the dynamic features of each mode and the historical anomaly storage index.

[0008] Ridge regression is used to train a linear combination of the dynamic features to construct a coupled feature that reflects the storage risk caused by the coupling of various modal storage indicators. The Mahalanobis distance between the coupled feature and the pre-trained Gaussian mixture model is determined as the anomaly probability score. The Gaussian mixture model is trained based on historical normal storage indicators.

[0009] Different anomaly detection thresholds are constructed based on the sensitivity differences at different stages of feeding, storage, and discharging.

[0010] The anomaly probability score is compared with the anomaly judgment threshold corresponding to different storage stages to determine the anomaly risk level and trigger the preset emergency handling process corresponding to the anomaly risk level.

[0011] Furthermore, the step of collecting multimodal storage index data and constructing a multimodal spatiotemporal tensor based on the storage index data and corresponding spatial location information specifically includes:

[0012] Temperature, humidity, weight, and material distribution image data are collected in real time by temperature and humidity sensors, weight sensors, and infrared vision sensors arranged in a three-dimensional grid within the storage cabinet, serving as the storage index data.

[0013] The stored index data is divided according to a preset time window, and each type of stored index data is associated with the three-dimensional spatial coordinates of the sensor that collected the stored index data to construct the spatiotemporal tensor of each mode.

[0014] Furthermore, the step of performing singular value decomposition on the spatiotemporal tensor of each modality, extracting principal components as dynamic features, and dynamically allocating modal weights corresponding to the dynamic features of each modality through attention weights specifically includes:

[0015] Singular value decomposition is performed on the spatiotemporal tensor for each mode. Based on the principle that the proportion of the sum of squared energy of multiple singular values ​​is greater than a preset threshold, multiple principal components are extracted as the dynamic features.

[0016] The mutual information entropy between the dynamic feature and the historical anomaly storage index for each modality is determined, and the corresponding modality weight is assigned to the dynamic feature based on the mutual information entropy, wherein the lower the mutual information entropy of the dynamic feature, the higher the assigned modality weight.

[0017] Furthermore, the linear combination of the dynamic features constructs a coupled feature that reflects the storage risk caused by the coupling of storage indicators of various modes, specifically including:

[0018] The linear combination coefficients corresponding to the linear combination are determined by ridge regression training based on historical abnormal storage indicators.

[0019] After fitting the dynamic features with the corresponding modal weights, a coupled feature vector is constructed by linearly combining the dynamic features of each modality based on the linear combination coefficients. :

[0020] in, For different modes Extracted dynamic features For the corresponding modal attention weights, These are the coefficients of the linear combination.

[0021] Furthermore, determining the Mahalanobis distance between the coupling feature and the pre-trained Gaussian mixture model as the anomaly probability score specifically includes:

[0022] The historical normal storage indicators are collected, and the corresponding dynamic features are extracted. Using modal weights and linear combination coefficients, the dynamic features of each modality are combined into coupled features, which serve as the training dataset. The expectation-maximization algorithm is used to fit multiple Gaussian distributions to form the Gaussian mixture model. For each sample in the training set, the expectation-maximization algorithm is used to fit the Gaussian mixture model, and the optimal Gaussian component is determined as the Gaussian component center using the Bayesian information criterion.

[0023]

[0024] in, The mixing coefficient represents the k-th Gaussian component; This represents the mean and covariance matrix of the component.

[0025] K represents the optimal number of components automatically determined by the Bayesian information criterion; That is, the k-th Gaussian distribution;

[0026] For each of the coupling features, determine the standardized statistical distance between the coupling feature and the center of the Gaussian component in each of the Gaussian mixture models: calculate the coupling feature vDF obtained in the current time window. new Substitute it into the model and compare it with the centers of all Gaussian distributions. Calculate Mahalanobis distance :

[0027]

[0028] The nearest Gaussian component is selected as the reference benchmark, and the corresponding standardized statistical distance is converted into anomaly probability scores: the nearest Gaussian component is selected. , corresponding distance value Anomaly probability scores mapped to standardized intervals:

[0029]

[0030] in, , The mean and standard deviation of the Mahalanobis distance of the training samples are respectively used. Finally, the score is compressed to the 0-1 interval using the Sigmoid function or a piecewise linear function, and used as a measure of the storage risk of the current time window.

[0031] Furthermore, the anomaly detection thresholds corresponding to different storage stages are divided based on the following steps:

[0032] For the material feeding stage, the anomaly detection threshold is the sum of a preset baseline value and three times the standard deviation of the training dataset.

[0033] For the storage phase, the anomaly detection threshold is the sum of the preset baseline value, twice the standard deviation, and the preset time decay term;

[0034] For the material discharge stage, the anomaly determination threshold is the sum of the preset baseline value and four times the standard deviation.

[0035] Furthermore, the determination of the abnormal risk level and the triggering of the preset emergency response procedure corresponding to the abnormal risk level specifically includes:

[0036] When the anomaly probability score is greater than the anomaly determination threshold of the corresponding storage stage, the anomaly probability score is mapped to a preset graded response score range to determine the corresponding graded response score.

[0037] When the graded response score is not less than a preset first score threshold and not greater than a preset second score threshold, the current storage state is determined to be at low anomaly risk, triggering local storage environment adjustment.

[0038] When the graded response score is greater than the preset second score threshold and not greater than the preset third score threshold, the current storage status is determined to be at medium risk of abnormality, the abnormality location information is determined and pushed to the user;

[0039] When the graded response score is greater than the preset third score threshold, the current storage status is determined to be at high risk of anomaly, the storage operation is stopped and the preset emergency handling procedure is initiated.

[0040] On the other hand, this invention also provides a system for detecting abnormalities in the tobacco processing and storage environment, comprising:

[0041] The spatiotemporal feature extraction module is used to collect stored index data of multiple modalities and construct a multimodal spatiotemporal tensor based on the stored index data and the corresponding spatial location information.

[0042] The feature decomposition module is used to perform singular value decomposition on the spatiotemporal tensor of each modality, extract principal components as dynamic features, and dynamically allocate modal weights corresponding to the dynamic features of each modality through attention weights.

[0043] The virtual dynamic feature construction module is used to linearly combine the dynamic features to construct coupled features that reflect the storage risk caused by the coupling of various modal storage indicators, and to determine the Mahalanobis distance between the coupled features and the pre-trained Gaussian mixture model as the anomaly probability score. The Gaussian mixture model is trained based on historical normal storage indicators.

[0044] The hierarchical decision-making module is used to compare the anomaly probability score with the anomaly judgment threshold corresponding to different storage stages, determine the anomaly risk level, and trigger the preset emergency handling process corresponding to the anomaly risk level.

[0045] On the other hand, this invention also provides a device for detecting abnormalities in the tobacco processing and storage environment, including a memory and one or more processors. The memory stores executable code, and when the processor executes the executable code, it implements the method for detecting abnormalities in the tobacco processing and storage environment.

[0046] On the other hand, this specification also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the method for detecting abnormalities in the tobacco processing and storage environment.

[0047] The beneficial effects of this invention are:

[0048] This invention provides a method, system, and apparatus for detecting anomalies in tobacco processing and storage environments. The method involves collecting storage index data across multiple modalities, constructing a multimodal spatiotemporal tensor based on the data and corresponding spatial location information, performing singular value decomposition on the spatiotemporal tensor for each modality, extracting principal components as dynamic features, and dynamically allocating modal weights corresponding to these dynamic features using attention weights. The dynamic features are then linearly combined to construct coupling features reflecting the storage risks caused by the coupling of storage indices across different modalities. The Mahalanobis distance between the coupling features and a pre-trained Gaussian mixture model (GMM) is determined as the anomaly probability score. The GMM is trained based on historical normal storage indices. The anomaly probability score is compared with anomaly judgment thresholds corresponding to different storage stages to determine the anomaly risk level and trigger a preset emergency response procedure corresponding to that risk level. By constructing a unified multimodal spatiotemporal tensor, extracting dynamic principal component features, introducing a cross-modal attention mechanism to fuse key features, performing anomaly scoring based on virtual dynamic features, and dynamically adjusting the judgment threshold according to the storage stage and responding to different risk levels, it can simultaneously process temperature, humidity, weight and visual information, comprehensively reflect the environmental status of the storage cabinet, break through the limitation of isolated data from traditional single sensors, and effectively reduce the false detection rate and false negative rate. Attached Figure Description

[0049] Figure 1 A flowchart of a method for detecting abnormalities in the tobacco processing and storage environment provided in an embodiment of this disclosure is shown;

[0050] Figure 2 A flowchart is shown below illustrating a method for determining the probability of anomalies in tobacco processing and storage environments, as provided in an embodiment of this disclosure.

[0051] Figure 3 A schematic diagram of an abnormal tobacco processing and storage environment detection system provided in an embodiment of this disclosure is shown.

[0052] Figure 4 A schematic diagram of an abnormal tobacco processing and storage environment detection device provided in an embodiment of this disclosure is shown. Detailed Implementation

[0053] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0054] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. The components of the embodiments of this disclosure described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0055] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0056] In this document, the term "and / or" merely describes a relationship, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0057] Research has revealed that tobacco processing storage cabinets require strict monitoring of parameters such as temperature, humidity, and material distribution. Traditional error prevention methods rely on single sensor threshold judgments (e.g., temperature over-limit alarms), ignoring the spatiotemporal correlation of multimodal data (temperature, humidity, weight, images), resulting in high false positive and false negative rates. They are also unable to adapt to gradual environmental anomalies (e.g., localized condensation, material mold growth), and manually set thresholds are insufficient to cover complex operating conditions.

[0058] Based on the above research, this disclosure provides a method, device, and electronic device for detecting anomalies in tobacco processing and storage environments. The method involves collecting storage index data across multiple modalities, constructing a multimodal spatiotemporal tensor based on the storage index data and corresponding spatial location information, performing singular value decomposition on the spatiotemporal tensor of each modality to extract principal components as dynamic features, and dynamically allocating modal weights corresponding to the dynamic features of each modality through attention weights, linearly combining the dynamic features to construct coupling features reflecting the storage risks caused by the coupling of storage indices across different modalities, and determining the Mahalanobis distance between the coupling features and a pre-trained Gaussian mixture model as an anomaly probability score. The Gaussian mixture model is trained based on historical normal storage indices. The anomaly probability score is compared with anomaly judgment thresholds corresponding to different storage stages to determine the anomaly risk level and trigger a preset emergency handling procedure corresponding to the anomaly risk level. By constructing a unified multimodal spatiotemporal tensor, extracting dynamic principal component features, introducing a cross-modal attention mechanism to fuse key features, performing anomaly scoring based on virtual dynamic features, and dynamically adjusting the judgment threshold according to the storage stage and responding to different risk levels, it can simultaneously process temperature, humidity, weight and visual information, comprehensively reflect the environmental status of the storage cabinet, break through the limitation of isolated data from traditional single sensors, and effectively reduce the false detection rate and false negative rate.

[0059] To facilitate understanding of this embodiment, a detailed description of the method for detecting abnormalities in the tobacco processing and storage environment disclosed in this disclosure is provided first. The execution entity of the method for detecting abnormalities in the tobacco processing and storage environment provided in this disclosure is generally a computer device with a certain computing capability. This computer device may include, for example, a terminal device, a server, or other processing equipment. The terminal device may be a user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, vehicle-mounted device, wearable device, etc. In some possible implementations, this method for detecting abnormalities in the tobacco processing and storage environment can be implemented by a processor calling computer-readable instructions stored in memory.

[0060] See Figure 1 The diagram shows a flowchart of a method for detecting abnormalities in the tobacco processing and storage environment according to an embodiment of this disclosure. The method includes steps S101 to S104, wherein:

[0061] S101. Collect storage index data of multiple modes, and construct a multimodal spatiotemporal tensor based on the storage index data and the corresponding spatial location information.

[0062] In practical implementation, the following types of sensors are deployed in a three-dimensional grid inside the tobacco processing storage cabinet:

[0063] Temperature sensors are used to collect temperature values ​​at each spatial node;

[0064] Humidity sensors are used to collect local air humidity.

[0065] Weight sensors are installed under shelves or pallets to measure changes in the load of tobacco products within a unit area;

[0066] The infrared vision module periodically collects images of material distribution or movement to help determine structural changes.

[0067] Here, each sensor has unique spatial coordinates and collects data at a uniform sampling frequency, such as once every 10 minutes. The collected raw data is divided into fixed time windows (e.g., each time step is 10 minutes) to form a discrete time series. The data from all sensors are aligned by timestamps to ensure that each modal data corresponds to a unified time index within each time window.

[0068] Here, for each modal data, a tensor is constructed based on its time series T and spatial location (x, y, z) to form a multimodal tensor set. Visual image modalities can be incorporated after feature processing (such as extracting spatial grayscale statistics, texture features, etc., to avoid introducing redundant high-dimensional pixel data).

[0069] Among them, the multimodal spatiotemporal tensor refers to the unified modeling of multidimensional data (such as temperature, humidity, weight, images, etc.) collected from different types (modal) sensors, combined with the spatial layout information of the sensors and synchronous acquisition in the temporal dimension, thereby preserving the modal characteristics, spatial relationships and temporal evolution features of the data, and providing a foundation for subsequent feature extraction and anomaly detection.

[0070] S102. Perform singular value decomposition on the spatiotemporal tensor of each modality, extract principal components as dynamic features, and dynamically allocate modal weights corresponding to the dynamic features of each modality through attention weights.

[0071] In practice, principal component analysis is performed on the spatiotemporal tensors corresponding to each mode for the constructed multimodal spatiotemporal tensor to extract feature vectors that characterize the dynamic change trend of the storage environment, and the importance of modes is dynamically weighted by combining the attention mechanism.

[0072] Specifically, the spatiotemporal tensor of each mode is first expanded. Taking the temperature mode as an example, its original tensor data dimension consists of sampling time windows and a three-dimensional grid of storage space. The tensor is expanded into a two-dimensional matrix along the time dimension, that is, the three-dimensional spatial data of each time window is flattened into a one-dimensional vector as a row of the matrix.

[0073] Here, singular value decomposition (SVD) is used to perform eigenvalue decomposition on the spatiotemporal matrix of each mode. Principal components are represented in the temporal direction, a diagonal matrix of singular values ​​reflecting the importance of each principal component, and a principal component representation in the spatial direction. The number of principal components to be retained is determined based on the cumulative energy ratio of the singular values; for example, the top k singular values ​​are selected such that their sum of squares accounts for more than 95% of the total energy. The extracted top k principal components are used as the dynamic features (DF) of the mode to represent the changing trend of the mode across the spatiotemporal scale.

[0074] In this embodiment, in order to effectively extract representative spatiotemporal features of each modality of sensing data in the tobacco storage environment, singular value decomposition (SVD) is first performed on the spatiotemporal tensor constructed for each modality to obtain principal component feature vectors that reflect its temporal evolution and spatial structure. Furthermore, modality weights are adaptively adjusted based on information theory methods to achieve dynamic feature enhancement under multimodal data fusion.

[0075] Specifically, for each modality, the spatiotemporal tensor is vectorized along the time dimension to construct a two-dimensional matrix. Then, singular value decomposition is applied to this matrix, which is represented by the product of the eigenbase in the time dimension, the singular value diagonal matrix (singular values ​​are arranged in descending order), and the eigenbase in the spatial dimension.

[0076] In practical applications, to reduce redundant dimensions and improve computational efficiency, the energy proportion criterion is used to select principal components. The first k singular values ​​are selected sequentially from left to right, such that the cumulative energy proportion of the sum of the squares of these singular values ​​reaches or exceeds a preset threshold (e.g., 95%). The extracted first k principal components constitute the dynamic feature vector of this mode, which is used to characterize the main changing trends of the storage environment in the time and space dimensions under this mode.

[0077] The dynamic principal components extracted by singular value decomposition (SVD) are not arbitrarily ordered mathematical components, but rather orthogonal basis vectors arranged in descending order of their contribution to the energy of the original spacetime tensor (i.e., the magnitude of the corresponding singular values). The spacetime pattern represented by each principal component can be interpreted through reconstruction error analysis and physical field verification.

[0078] For example, after performing SVD on the humidity modal spatiotemporal tensor of a tobacco storage cabinet, the principal component with the highest energy proportion (corresponding to the largest singular value) usually reflects the average humidity level of the overall environment and its diurnal periodic fluctuations; while the principal component with the second highest energy but a significant spatial gradient structure (such as its spatial load vector showing a monotonically increasing or decreasing trend in the Z-axis direction) may correspond to a non-uniform humidity evolution process in the vertical direction. In an actual operation, the temporal coefficient of this gradient principal component continuously increased over 48 hours, while the overall average humidity did not exceed the limit. Combining infrared thermal imaging and dew point calculation, it was confirmed that the temperature at the top of the cabinet was lower than the dew point due to a defect in the insulation layer, causing moisture to accumulate and diffuse downwards, forming a typical "precursor to condensation from top to bottom". At this time, the traditional threshold alarm based on the average value had not yet been triggered, while this method achieved early warning through the abnormal activation of this principal component.

[0079] Similarly, in the weight mode, if the spatial load of a certain principal component is concentrated in the central area of ​​the cabinet and its time coefficient shows a slow decay trend, it may indicate the mass loss of the central material due to mold or volatilization; in the temperature mode, if a certain principal component exhibits phase lag characteristics in the area near the vent (i.e., the temperature response is delayed compared to the set period), it may reflect local blockage of the air duct.

[0080] The identification of the aforementioned principal components relies on the decoupling capability of SVD for spatiotemporal coupling structures—it projects high-dimensional sensor data into a low-dimensional subspace dominated by historical normal behavior, allowing potential anomalous patterns deviating from this subspace (such as local gradients, phase shifts, asymmetric decay, etc.) to be explicitly characterized. Therefore, the dynamic principal components extracted in this invention are not only a dimensionality reduction tool, but also an interpretable bridge connecting multimodal sensor data with the unique physical anomaly mechanisms of tobacco storage.

[0081] Next, to achieve effective fusion among multiple modalities, the mutual information entropy between the dynamic features of each modality and the historical anomaly storage indicators is calculated. The mutual information entropy is used to measure the correlation between the current modality and historical anomaly scenarios, and is expressed through the dynamic feature vector of each modality, the historically labeled anomaly storage samples, the joint probability of feature values ​​and anomaly states, and the marginal probability distribution.

[0082] Based on the calculation results of mutual information entropy, attention weights are assigned to the dynamic features of each modality. Specifically, modality weights are assigned according to the reciprocal of mutual information entropy; the smaller the entropy value, the higher the contribution of that modality to abnormal pattern recognition, and the greater the attention weight assigned to it.

[0083] For example, in the actual process of identifying tobacco mold risk, it was found that the mutual information entropy of the humidity mode was 0.23, which is much smaller than that of other modes such as temperature (0.56) and weight (0.61). Therefore, the attention weight of the dynamic features of the humidity mode was automatically increased to 0.74, which significantly enhanced the contribution of this mode to the comprehensive coupled features, thereby improving the sensitivity and accuracy of such abnormal states.

[0084] S103. Linearly combine the dynamic features to construct a coupling feature that reflects the storage risk caused by the coupling of various modal storage indicators, and determine the Mahalanobis distance between the coupling feature and the pre-trained Gaussian mixture model as the anomaly probability score. The Gaussian mixture model is trained based on historical normal storage indicators.

[0085] In specific implementation, in order to further reveal the potential synergistic change relationship among multimodal storage indicators and improve the discrimination ability of anomaly detection, based on the dynamic features of each modality obtained by weighting with singular value decomposition (SVD) and mutual information entropy, a linear combination method is adopted to construct coupled features that can reflect storage risk, and Mahalanobis distance is introduced in combination with a pre-trained Gaussian mixture model (GMM) to achieve anomaly scoring.

[0086] Specifically, after fitting the dynamic features with the corresponding modal weights, the dynamic features of each modality are combined linearly to construct coupled features. During the linear combination process, ridge regression training is performed based on historical anomaly storage indicators to determine the linear combination coefficients corresponding to the linear combination.

[0087] Here, for different modalities Extracted dynamic features and its corresponding modal attention weights Define the coupled feature vector vDF as:

[0088]

[0089] in, These are linear combination coefficients, used to further fine-tune the coupling relationship between modes; The weights are obtained by training on historical outlier data using ridge regression with a regularization term to avoid overfitting. This represents the overall influence factor of the mode in coupled modeling.

[0090] In practice, historically known anomalous samples (e.g., mold, deliquescence) are used to construct the objective function. By minimizing the mean squared error between the predicted coupling features and the known risk labels, and by adding an L2 regularization term to control the model complexity, the optimal set of linear coefficients is determined. .

[0091] For example, in mold risk analysis, the combination coefficients of temperature dynamic features and humidity dynamic features obtained through ridge regression training are 0.7 and 0.3, respectively. This indicates that humidity is highly correlated with mold risk, but temperature also has a significant impact. The final coupled feature vector is expressed as follows:

[0092]

[0093] For further details on how to determine the anomaly probability score, please refer to [link / reference needed]. Figure 2 The diagram shows a flowchart of a method for determining the probability of anomalies in tobacco processing and storage environments according to an embodiment of this disclosure. The method includes steps S1031 to S1033, wherein:

[0094] S1031. Collect the historical normal storage indicators, extract the corresponding dynamic features as training datasets, and use the expectation-maximization algorithm to fit multiple Gaussian distributions to form the Gaussian mixture model.

[0095] S1032. Determine the optimal Gaussian component as the Gaussian component center using the Bayesian information criterion, and for each of the coupling features, determine the standardized statistical distance between the coupling feature and each of the Gaussian component centers.

[0096] S1033. Select the nearest Gaussian component as a reference benchmark, and convert the corresponding standardized statistical distance into the anomaly probability score in the interval of 0 to 1.

[0097] In this embodiment, in order to achieve unsupervised modeling and real-time assessment of the abnormal risks of tobacco processing and storage environment, a Gaussian mixture model (GMM) is used to model the coupling features extracted from historical normal storage data, and an anomaly probability scoring function is constructed by combining Mahalanobis distance.

[0098] In practical implementation, during the early operation phase, data stored without anomalies for 30 consecutive days or more is selected as the training sample source. The multimodal sensor data in this phase is processed to form the training dataset through the following steps: the collected data is synchronously divided into 10-minute time windows; singular value decomposition (SVD) is performed on the spatiotemporal tensor of each mode within each time window, retaining principal components with a cumulative singular value energy greater than 95% as dynamic features; this method can effectively extract low-dimensional dynamic principal component features that simultaneously contain temporal evolution patterns and spatial distribution patterns; using the modal weights and ridge regression coefficients obtained from the aforementioned training, the dynamic features of each mode are linearly combined into coupled features (vDF).

[0099] Here, a Gaussian mixture model is fitted to each sample in the training set using the expectation-maximization algorithm. The Gaussian mixture model is expressed by the weights of the Gaussian distribution, the centers of the Gaussian components, the covariance matrix of the Gaussian components, and the number of Gaussian distributions. The optimal value of the number of Gaussian distributions is determined by the Bayesian information criterion. The finally trained GMM model is used to characterize the distribution features of coupled features under normal storage conditions.

[0100] In the real-time operation phase, for the coupled features extracted in the current time window, the Mahalanobis distance between the current feature and its center is calculated for each component in the GMM. The component with the smallest distance is selected for standardization, based on the statistical characteristics of historical Mahalanobis distances. The standardized score is mapped to the [0,1] interval using the Sigmoid function. The closer the score is to 1, the more the current storage environment deviates from the historical normal distribution, and the higher the risk of anomalies.

[0101] In this way, the abnormal score is used as a quantitative indicator of storage risk. It is further combined with the dynamic thresholds set in the storage stage (such as different multiples for the feeding, storage and discharging stages) to determine the risk level and trigger the corresponding alarm strategy.

[0102] Specifically, first, extract all coupling features of the corresponding time window from the normal operating period (e.g., 30 days) of the storage cabinet to construct a training dataset, and then use the expectation-maximization algorithm to fit a GMM model to this dataset:

[0103]

[0104] in, The mixing coefficient represents the k-th Gaussian component; The mean and covariance matrix of the component are represented; K represents the optimal number of components automatically determined by the Bayesian information criterion. This is the kth Gaussian distribution.

[0105] During the detection phase, the coupled feature vDF calculated in the current time window is used. new Substitute it into the model and compare it with the mean of all Gaussian distributions. Calculate Mahalanobis distance :

[0106]

[0107] Select the nearest Gaussian component , corresponding distance value Anomaly probability scores mapped to standardized intervals:

[0108]

[0109] in, , The mean and standard deviation of the Mahalanobis distance of the training samples are respectively used. Finally, the score is compressed to the 0-1 interval using the Sigmoid function or a piecewise linear function, and used as a measure of the storage risk of the current time window.

[0110] S104. The abnormal probability score is compared with the abnormal judgment threshold corresponding to different storage stages to determine the abnormal risk level and trigger the preset emergency handling process corresponding to the abnormal risk level.

[0111] In practical implementation, in order to further improve the accuracy and response efficiency of anomaly detection during tobacco processing and storage, this embodiment, after completing the anomaly probability score of the current time window storage state, realizes risk level classification and graded emergency response control by setting adaptive anomaly judgment thresholds and multi-level response mechanisms for different storage stages.

[0112] Here, since the storage process can be divided into a feeding stage, a storage stage, and a discharging stage, and the environmental stability and anomaly sensitivity of each stage differ significantly, this embodiment sets different anomaly judgment thresholds according to the storage stage to achieve a balance between sensitivity and false alarm rate control. For the feeding stage, the anomaly judgment threshold is the sum of the preset baseline value and the standard deviation corresponding to three times the training dataset; for the storage stage, the anomaly judgment threshold is the sum of the preset baseline value and two times the standard deviation plus a preset time decay term; for the discharging stage, the anomaly judgment threshold is the sum of the preset baseline value and four times the standard deviation.

[0113] It should be noted that the preset baseline value can be set according to actual needs, and no specific restrictions are imposed here.

[0114] To enhance the system's emergency response capabilities for abnormal situations, this embodiment further refines the anomaly score into multiple risk levels and sets corresponding response measures. When the anomaly probability score is greater than the anomaly judgment threshold for the corresponding storage stage, the anomaly probability score is mapped to a preset graded response score range to determine the corresponding graded response score. When the graded response score is not less than a preset first score threshold and not greater than a preset second score threshold, the current storage state is determined to be at low anomaly risk, triggering local storage environment adjustment. When the graded response score is greater than a preset second score threshold and not greater than a preset third score threshold, the current storage state is determined to be at medium anomaly risk, anomaly location information is determined and pushed to the user. When the graded response score is greater than a preset third score threshold, the current storage state is determined to be at high anomaly risk, storage operations are stopped and a preset emergency handling procedure is initiated.

[0115] It should be noted that the preset first scoring threshold, preset second scoring threshold, and preset third scoring threshold can be set according to actual needs, and no specific restrictions are imposed here.

[0116] Here, once the detection score exceeds the preset first score threshold, the environmental controller will be activated to start relevant modules, such as turning on the fan, adjusting the humidity, and local heating. When the score exceeds the preset second score threshold, an abnormal alarm log with location information will be generated and pushed to the operator through the human-machine interface or mobile terminal. When the score exceeds the preset third score threshold, the storage operation will be stopped immediately, the power will be cut off to the safe area, and the preset emergency response module will be invoked, such as recording abnormal data, calling backup sensors, and self-testing logic.

[0117] This disclosure provides a method for detecting anomalies in the tobacco processing and storage environment. The method involves collecting storage index data across multiple modalities, constructing a multimodal spatiotemporal tensor based on the storage index data and corresponding spatial location information, performing singular value decomposition on the spatiotemporal tensor of each modality to extract principal components as dynamic features, and dynamically allocating modal weights corresponding to the dynamic features of each modality through attention weights, linearly combining the dynamic features to construct coupling features reflecting the storage risks caused by the coupling of storage indices across different modalities, and determining the Mahalanobis distance between the coupling features and a pre-trained Gaussian mixture model as an anomaly probability score. The Gaussian mixture model is trained based on historical normal storage indices. The anomaly probability score is compared with anomaly judgment thresholds corresponding to different storage stages to determine the anomaly risk level and trigger a preset emergency handling procedure corresponding to the anomaly risk level. By constructing a unified multimodal spatiotemporal tensor, extracting dynamic principal component features, introducing a cross-modal attention mechanism to fuse key features, performing anomaly scoring based on virtual dynamic features, and dynamically adjusting the judgment threshold according to the storage stage and responding to different risk levels, it can simultaneously process temperature, humidity, weight and visual information, comprehensively reflect the environmental status of the storage cabinet, break through the limitation of isolated data from traditional single sensors, and effectively reduce the false detection rate and false negative rate.

[0118] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0119] Based on the same inventive concept, this disclosure also provides a tobacco processing and storage environment anomaly detection device corresponding to the tobacco processing and storage environment anomaly detection method. Since the principle of the device in this disclosure for solving the problem is similar to the tobacco processing and storage environment anomaly detection method described above, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0120] Please see Figure 3 , Figure 3 This is a schematic diagram of an abnormal tobacco processing and storage environment detection device provided in an embodiment of this disclosure. Figure 3 As shown in the figure, the tobacco processing and storage environment abnormality detection device 300 provided in this embodiment includes:

[0121] The spatiotemporal feature extraction module 310 is used to collect stored index data of multiple modalities and construct a multimodal spatiotemporal tensor based on the stored index data and the corresponding spatial location information.

[0122] The feature decomposition module 320 is used to perform singular value decomposition on the spatiotemporal tensor of each modality, extract principal components as dynamic features, and dynamically allocate modal weights corresponding to the dynamic features of each modality through attention weights.

[0123] The virtual dynamic feature construction module 330 is used to linearly combine the dynamic features to construct coupled features that reflect the storage risk caused by the coupling of various modal storage indicators, and to determine the Mahalanobis distance between the coupled features and the pre-trained Gaussian mixture model as the anomaly probability score. The Gaussian mixture model is trained based on historical normal storage indicators.

[0124] The hierarchical decision module 340 is used to compare the anomaly probability score with the anomaly judgment threshold corresponding to different storage stages, determine the anomaly risk level, and trigger the preset emergency handling process corresponding to the anomaly risk level.

[0125] The processing flow of each module in the device and the interaction flow between each module can be referred to the relevant descriptions in the above method embodiments, and will not be detailed here.

[0126] This disclosure provides an abnormal tobacco processing and storage environment detection device. It collects storage index data across multiple modalities, constructs a multimodal spatiotemporal tensor based on the storage index data and corresponding spatial location information, performs singular value decomposition on the spatiotemporal tensor of each modality, extracts principal components as dynamic features, and dynamically allocates modal weights corresponding to the dynamic features of each modality through attention weights. It linearly combines the dynamic features to construct a coupling feature reflecting the storage risk caused by the coupling of storage indices across different modalities, and determines the Mahalanobis distance between the coupling feature and a pre-trained Gaussian mixture model as an anomaly probability score. The Gaussian mixture model is trained based on historical normal storage indices. The anomaly probability score is compared with anomaly judgment thresholds corresponding to different storage stages to determine the anomaly risk level and triggers a preset emergency handling procedure corresponding to the anomaly risk level. By constructing a unified multimodal spatiotemporal tensor, extracting dynamic principal component features, introducing a cross-modal attention mechanism to fuse key features, performing anomaly scoring based on virtual dynamic features, and dynamically adjusting the judgment threshold according to the storage stage and responding to different risk levels, it can simultaneously process temperature, humidity, weight and visual information, comprehensively reflect the environmental status of the storage cabinet, break through the limitation of isolated data from traditional single sensors, and effectively reduce the false detection rate and false negative rate.

[0127] Corresponding to the aforementioned embodiment of a method for detecting abnormalities in the tobacco processing and storage environment, the present invention also provides an embodiment of a device for detecting abnormalities in the tobacco processing and storage environment.

[0128] See Figure 4 The present invention provides a tobacco processing and storage environment anomaly detection device, which includes a memory and one or more processors. The memory stores executable code, and when the processor executes the executable code, it is used to implement a tobacco processing and storage environment anomaly detection method in the above embodiment.

[0129] An embodiment of the tobacco processing and storage environment anomaly detection device provided by this invention can be applied to any device with data processing capabilities, such as a computer. The device embodiment can be implemented through software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by the processor of any data processing device reading the corresponding computer program instructions from non-volatile memory into memory and executing them. From a hardware perspective, such as... Figure 4 The diagram shown is a hardware structure diagram of any device with data processing capabilities, which is part of the tobacco processing and storage environment anomaly detection device provided by the present invention. (Except for...) Figure 4 In addition to the processor, memory, network interface, and non-volatile memory shown, any data processing device in the embodiment may also include other hardware depending on the actual function of the data processing device, which will not be described in detail here.

[0130] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0131] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the present invention according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0132] This invention also provides a computer-readable storage medium storing a program thereon, which, when executed by a processor, implements a method for detecting abnormalities in the tobacco processing and storage environment as described in the above embodiments.

[0133] The computer-readable storage medium can be an internal storage unit of any data processing device described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device of any data processing device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units and external storage devices of any data processing device. The computer-readable storage medium is used to store the computer program and other programs and data required by the data processing device, and can also be used to temporarily store data that has been output or will be output.

[0134] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements the aforementioned method for detecting abnormalities in the tobacco processing and storage environment.

[0135] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this application are indicated by the claims.

[0136] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. This application is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for detecting abnormalities in the tobacco processing and storage environment, characterized in that, include: Collect storage index data of multiple modalities, and construct a multimodal spatiotemporal tensor based on the storage index data and the corresponding spatial location information; Singular value decomposition is performed on the spatiotemporal tensor of each modality to extract principal components as low-dimensional dynamic features containing temporal evolution laws and spatial distribution patterns. Modal attention weights corresponding to the dynamic features of each modality are dynamically allocated through the mutual information entropy between the dynamic features of each modality and the historical anomaly storage index. Ridge regression is used to train a linear combination of the dynamic features to construct a coupled feature that reflects the storage risk caused by the coupling of various modal storage indicators. The Mahalanobis distance between the coupled feature and the pre-trained Gaussian mixture model is determined as the anomaly probability score. The Gaussian mixture model is trained based on historical normal storage indicators. Determining the Mahalanobis distance between the coupled features and the pre-trained Gaussian mixture model as the anomaly probability score specifically includes: collecting the historical normal storage indicators, extracting the corresponding dynamic features, combining the dynamic features of each modality into coupled features using modal attention weights and linear combination coefficients, using this as the training dataset, fitting multiple Gaussian distributions using the expectation-maximization algorithm to form the Gaussian mixture model; fitting the Gaussian mixture model using the expectation-maximization algorithm for each sample in the training set, and determining the optimal Gaussian component as the Gaussian component center using the Bayesian information criterion. For each of the coupling features, determine the standardized statistical distance between the coupling feature and the center of the Gaussian component in each of the Gaussian mixture models: calculate the coupling feature vDF obtained in the current time window. new Substitute it into the model and compare it with the centers of all Gaussian distributions. Calculate Mahalanobis distance ; The nearest Gaussian component is selected as the reference benchmark, and the corresponding standardized statistical distance is converted into anomaly probability scores: the nearest Gaussian component is selected. , corresponding distance value Anomaly probability scores mapped to standardized intervals; Finally, the score is compressed to the 0-1 interval using a Sigmoid function or a piecewise linear function, and used as a measure of the storage risk in the current time window. Different anomaly detection thresholds are constructed based on the sensitivity differences at different stages of feeding, storage, and discharging. The anomaly probability score is compared with the anomaly judgment threshold corresponding to different storage stages to determine the anomaly risk level and trigger the preset emergency handling process corresponding to the anomaly risk level.

2. The method according to claim 1, characterized in that, Collect storage index data of multiple modalities, and construct a multimodal spatiotemporal tensor based on the storage index data and corresponding spatial location information, specifically including: Temperature, humidity, weight, and material distribution image data are collected in real time by temperature and humidity sensors, weight sensors, and infrared vision sensors arranged in a three-dimensional grid within the storage cabinet, serving as the storage index data. The stored index data is divided according to a preset time window, and each type of stored index data is associated with the three-dimensional spatial coordinates of the sensor that collected the stored index data to construct the spatiotemporal tensor of each mode.

3. The method according to claim 1, characterized in that, Singular value decomposition is performed on the spatiotemporal tensors of each modality to extract principal components as low-dimensional dynamic features containing temporal evolution patterns and spatial distribution patterns. Modal attention weights corresponding to the dynamic features of each modality are dynamically allocated based on the mutual information entropy between the dynamic features of each modality and historical anomaly storage indicators. Specifically, this includes: Singular value decomposition is performed on the spatiotemporal tensor for each mode. Based on the principle that the proportion of the sum of squared energy of multiple singular values ​​is greater than a preset threshold, multiple principal components are extracted as the dynamic features. The mutual information entropy between the dynamic feature and the historical anomaly storage index for each modality is determined, and the corresponding modal attention weight is assigned to the dynamic feature based on the mutual information entropy, wherein the lower the mutual information entropy of the dynamic feature, the higher the assigned modal attention weight.

4. The method according to claim 1, characterized in that, By linearly combining the aforementioned dynamic features, a coupled feature is constructed that reflects the storage risk caused by the coupling of storage indicators across different modes, specifically including: The linear combination coefficients corresponding to the linear combination are determined by ridge regression training based on historical abnormal storage indicators. After fitting the dynamic features with the corresponding modal attention weights, a coupled feature vector is constructed by combining the dynamic features of each modality in a linear combination manner based on the linear combination coefficients.

5. The method according to claim 1, characterized in that, The construction of different anomaly judgment thresholds based on the sensitivity differences at different stages of feeding, storage, and discharging includes: For the material feeding stage, the anomaly detection threshold is the sum of a preset baseline value and three times the standard deviation of the training dataset. For the storage phase, the anomaly detection threshold is the sum of the preset baseline value, twice the standard deviation, and the preset time decay term; For the material discharge stage, the anomaly determination threshold is the sum of the preset baseline value and four times the standard deviation.

6. The method according to claim 1, characterized in that, Determine the level of abnormal risk and trigger the preset emergency response procedure corresponding to the level of abnormal risk, specifically including: When the anomaly probability score is greater than the anomaly determination threshold of the corresponding storage stage, the anomaly probability score is mapped to a preset graded response score range to determine the corresponding graded response score. When the graded response score is not less than a preset first score threshold and not greater than a preset second score threshold, the current storage state is determined to be at low anomaly risk, triggering local storage environment adjustment. When the graded response score is greater than the preset second score threshold and not greater than the preset third score threshold, the current storage status is determined to be at medium risk of abnormality, the abnormality location information is determined and pushed to the user; When the graded response score is greater than the preset third score threshold, the current storage status is determined to be at high risk of anomaly, the storage operation is stopped and the preset emergency handling procedure is initiated.

7. A system for detecting abnormalities in the tobacco processing and storage environment, characterized in that, include: The spatiotemporal feature extraction module is used to collect stored index data of multiple modalities and construct a multimodal spatiotemporal tensor based on the stored index data and the corresponding spatial location information. The feature decomposition module is used to perform singular value decomposition on the spatiotemporal tensor of each modality, extract principal components as low-dimensional dynamic features containing temporal evolution laws and spatial distribution patterns, and dynamically allocate modal attention weights corresponding to the dynamic features of each modality through the mutual information entropy between the dynamic features of each modality and the historical anomaly storage index. A virtual dynamic feature construction module is used to train a linear combination of the dynamic features using ridge regression to construct coupled features reflecting the storage risk caused by the coupling of various modal storage indicators, and to determine the Mahalanobis distance between the coupled features and the pre-trained Gaussian mixture model as the anomaly probability score. The Gaussian mixture model is trained based on historical normal storage indicators. Different anomaly judgment thresholds are constructed according to the sensitivity differences at different stages of feeding, storage, and discharging. Specifically, determining the Mahalanobis distance between the coupled features and the pre-trained Gaussian mixture model as the anomaly probability score includes: collecting the historical normal storage indicators, extracting the corresponding dynamic features, combining the dynamic features of each modality into coupled features using modal attention weights and linear combination coefficients as training datasets, fitting multiple Gaussian distributions using the expectation-maximization algorithm to form the Gaussian mixture model; fitting the Gaussian mixture model using the expectation-maximization algorithm for each sample in the training set, and determining the optimal Gaussian component as the Gaussian component center using the Bayesian information criterion. For each of the coupling features, determine the standardized statistical distance between the coupling feature and the center of the Gaussian component in each of the Gaussian mixture models: calculate the coupling feature vDF obtained in the current time window. new Substitute it into the model and compare it with the centers of all Gaussian distributions. Calculate Mahalanobis distance ; The nearest Gaussian component is selected as the reference benchmark, and the corresponding standardized statistical distance is converted into anomaly probability scores: the nearest Gaussian component is selected. , corresponding distance value Anomaly probability scores mapped to standardized intervals; Finally, the score is compressed to the 0-1 interval using a Sigmoid function or a piecewise linear function, and used as a measure of the storage risk in the current time window. The hierarchical decision-making module is used to compare the anomaly probability score with the anomaly judgment threshold corresponding to different storage stages, determine the anomaly risk level, and trigger the preset emergency handling process corresponding to the anomaly risk level.

8. A device for detecting abnormalities in tobacco processing and storage environment, comprising a memory and one or more processors, wherein the memory stores executable code, characterized in that... When the processor executes the executable code, it implements a method for detecting abnormalities in the tobacco processing and storage environment as described in any one of claims 1-6.

9. A computer-readable storage medium having a program stored thereon, characterized in that, When the program is executed by the processor, it implements a method for detecting abnormalities in the tobacco processing and storage environment as described in any one of claims 1-6.

Images