Intelligent monitoring system for white spirit brewing process based on multi-modal sensing and digital twinning

Abstract

The invention discloses a baijiu brewing process intelligent monitoring system based on multi-mode sensing and digital twinning, and belongs to the technical field of baijiu brewing intelligent monitoring. The method solves the problems that in the prior art, fusion perception and a virtual experiment environment for multi-dimensional data are lacked, and predictive control and self-adaptive optimization are difficult to achieve, the causal knowledge graph and the incremental updating algorithm are used for rapidly recognizing data changes and reasons thereof, only the changed data are updated, and the data change speed is greatly improved. Frequent refreshing of the whole digital twinborn model is avoided, so that the response speed and the calculation efficiency of the system are improved; when the data is identified to be abnormal, prediction is carried out through a neural network model, the influence of deviation on the subsequent brewing process is simulated in advance, an optimization control instruction is generated, and conversion from post-event correction to beforehand prevention is realized; high-quality technical support is provided for intelligent monitoring of the white spirit brewing process, and the product quality and the production efficiency can be improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent monitoring technology for baijiu brewing, specifically to an intelligent monitoring system for the baijiu brewing process based on multimodal perception and digital twins. Background Technology

[0002] Traditional baijiu brewing relies heavily on the experience of craftsmen, resulting in problems such as low transparency in the production process, rough control of process parameters, large quality fluctuations, and difficulty in traceability.

[0003] Many existing monitoring systems use single sensors for local monitoring, lacking global, real-time, and in-depth fusion perception of key information such as mash status, microbial activity, and aroma components. In addition, existing process optimization lacks a high-fidelity virtual experimental environment, making it difficult to achieve predictive control and adaptive optimization.

[0004] Therefore, to meet existing needs, a smart monitoring system for the Baijiu brewing process based on multimodal perception and digital twins is proposed. Summary of the Invention

[0005] The purpose of this invention is to provide an intelligent monitoring system for the Baijiu brewing process based on multimodal perception and digital twins. By utilizing causal knowledge graphs and incremental update algorithms, it can quickly identify data changes and their causes, updating only the changed data and avoiding frequent refreshes of the entire digital twin model, thereby improving the system's response speed and computational efficiency. When data anomalies are identified, a neural network model is used for prediction, simulating the impact of deviations on the subsequent brewing process in advance, and generating optimized control commands, realizing the shift from post-event correction to pre-event prevention. This provides high-quality technical support for the intelligent monitoring of the Baijiu brewing process, helps improve product quality and production efficiency, and solves the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: A smart monitoring system for the Baijiu brewing process based on multimodal perception and digital twins includes: The multimodal sensing unit is configured to collect raw material physical parameters, material state image parameters, wine indicators, environmental parameters and process parameters in real time during the brewing process based on a passive sensing radio frequency chip and a multimodal sensor, and then integrate and process them to form multimodal data. The twin modeling unit is configured to construct a digital twin model of the liquor brewing process based on virtual reality technology, and receive multimodal data in real time. It uses an incremental update algorithm to automatically adjust the parameters that have changed in the model, so that the digital twin model is synchronized with the actual brewing process in real time. A neural network model is constructed, multimodal data is used as input, and historical brewing data is combined to analyze the brewing process and predict the fermentation endpoint and wine quality indicators; based on the prediction results, optimization instructions are generated to dynamically adjust the brewing process parameters. The Internet of Things (IoT) control unit is configured to drive the actuators for precise control based on optimized instructions.

[0007] Furthermore, the twin modeling unit includes: The dynamic feedback module is configured to introduce a dynamic feedback mechanism and transmit real-time multimodal data to the digital twin model through a redundant communication mechanism. Based on historical brewing data, a causal knowledge graph is constructed in a digital twin model; When the digital twin model receives real-time multimodal data, it uses an incremental update algorithm to reason in the causal knowledge graph, identify whether there are changes in the real-time multimodal data, and further reason about the reasons for the changes in the data. Based on the cause of the change, identify the root cause parameters that need to be adjusted in the causal knowledge graph; Based on the root cause corresponding parameters, parameters are updated in the digital twin model, and parameters that have changed in the model are automatically adjusted.

[0008] Furthermore, the twin modeling unit also includes: The feedforward early warning module is configured to mark the changed data as abnormal when the incremental update algorithm identifies changes in real-time multimodal data. The data from this batch is generated into a data package and fed into a neural network model for prediction. The neural network model generates the expected trend for the subsequent brewing process based on historical data and the current state. When the real-time multimodal data deviates from the expected results, the feedforward prediction process is immediately triggered, and the digital twin model is used to simulate the impact of the current deviation on the subsequent liquor brewing process. Based on the feedforward prediction results, optimized control commands are automatically generated to respond to potential changes.

[0009] Furthermore, the twin modeling unit also includes: The updated evaluation module is configured to calculate the updated state confidence by comparing the deviation between the neural network model prediction and the real-time multimodal data after the digital twin model is updated, and measure the consistency between the digital twin model state and the actual environment state based on the preset maximum deviation threshold. Based on historical data and real-time feedback information, dynamically adjust the deviation threshold in the confidence calculation; Based on the state confidence value, the state of the local twin is divided into different confidence ratings. When the deviation between the digital twin model and the actual environmental state still cannot be converged after the digital twin model is updated, the confidence rating of the local twin state is automatically reduced and a low confidence warning is issued. Examine the actual environmental state corresponding to the low confidence level and compare it with the virtual state in the digital twin model; feed the confirmed results back into the digital twin model to quickly correct the model parameters.

[0010] Furthermore, the updated evaluation module includes: The monitoring submodule is used to continuously monitor the deviation between the neural network model prediction value and the real-time multimodal data, calculate the updated state confidence, and determine the multimodal data combination and process context corresponding to the local twin whose state confidence is continuously lower than the preset threshold based on the continuous monitoring results. The analysis submodule is used for: Based on the configuration of the low confidence event statistics window and frequency threshold on the management terminal, and based on the configuration results, the multimodal data combination and process context corresponding to the local twin are analyzed. When the number of times the same type of data combination and process context triggers a low confidence state in the low confidence event statistics window exceeds the frequency threshold, a causal discovery trigger signal is generated. Based on the causal discovery trigger signal, multimodal data time series associated with low confidence states are extracted from historical databases and real-time multimodal data, and corresponding complete process parameter records and environmental parameter records are extracted simultaneously to obtain the dataset to be analyzed; The dataset to be analyzed is time-series aligned and feature-processed to construct a feature-state matrix. Based on a preset association rule mining strategy, the co-occurrence patterns between each feature item in the feature-state matrix and low-confidence states are scanned to determine the corresponding confidence and lift index. Based on a preset association threshold, association rules with lift index greater than the first threshold and confidence greater than the second threshold are selected, and the feature items corresponding to the association rules are identified as a potential unrecorded causal factor candidate set. The potential unrecorded causal factor candidate set is compared with the causal nodes recorded in the causal knowledge graph. Duplicate factors are eliminated to obtain the effective unrecorded causal factor set. Based on the business agreement, the action path of the brewing process is constructed for each factor in the effective unrecorded causal factor set in the digital twin model. Based on the action path, perform controlled variable simulation in the digital twin model, and determine whether the low-confidence state is reproduced based on the controlled variable simulation results; When the probability of low-confidence states changes after the existence of a moderating factor, the current factor is determined to be a new causal factor that has been verified. At the same time, the quantitative relationship between the new causal factor and the low-confidence state is determined based on the simulation results of the control variables. Based on the determination results, the new causal factor, the action path, and the quantitative relationship between the new causal factor and the low-confidence state are fused and stored in the causal knowledge graph. The fusion and storage results are synchronized to the training dataset of the neural network model, and the prediction mechanism of the neural network model is optimized based on the synchronization results.

[0011] Furthermore, the IoT control unit includes: The closed-loop feedback module is configured to connect to the actuator. After executing the optimized control command, it monitors the control result in real time and feeds the result back to the digital twin model. Based on the feedback result, it further adjusts the optimized command to form a closed-loop control. The fault detection module is configured to monitor the operating status of the actuator in real time, detect equipment faults in a timely manner and issue early warning signals; and feed the fault information back to the digital twin model to automatically adjust and optimize strategies based on the fault situation.

[0012] Furthermore, the multimodal sensing unit includes: The parameter acquisition module is configured to embed a passive sensor radio frequency chip in the brewing raw materials to monitor the physical parameters of the raw materials in real time, as well as the information on metabolites during the microbial fermentation process; it also combines multi-mode sensors to collect brewing environment parameters and process parameters. The image acquisition module is configured to install high-definition cameras at key locations in the brewing workshop and use visual inspection technology to monitor and analyze the material status in real time during the brewing process; and through image recognition algorithms, it automatically detects the maturity of raw materials and the clarity of the wine.

[0013] Furthermore, the multimodal sensing unit also includes: The data fusion module is configured to merge and organize the collected data to form multimodal data. A data fusion algorithm is used to assign different weights to data from different sources based on their importance and reliability, and in combination with actual needs. A weighted fusion algorithm is then used to sum or average the data with different weights to obtain comprehensive data. Data from different sources and of different types are calibrated, and the correlation between different data is analyzed to identify characteristic information that affects the quality indicators of baijiu brewing.

[0014] Furthermore, it also includes: The visualization interaction unit is configured to display the operating status and control results of the actuator through a visual interface. Operators can manually adjust brewing process parameters or intervene in the operating status of the equipment through the interface. When an anomaly is detected, an alarm is immediately issued, and detailed anomaly information and suggested measures are displayed on the visual interface to gain a deeper understanding of the various states and changes in the brewing process; It also provides historical data query and analysis functions based on a visual interface, allowing operators to query parameter changes and equipment operating status during the historical brewing process.

[0015] Furthermore, the fermentation product concentration increment assessment unit is configured as follows: The real-time concentration increment calculation module is used to collect the measured values ​​of the target product concentration at two adjacent data acquisition times during the fermentation process of baijiu based on the multimodal sensing unit; and to calculate the real-time increment of the target product concentration. The concentration theoretical increment calculation module is used to obtain the maximum specific growth rate of microorganisms, the substrate concentration and cell concentration of the reaction raw materials during microbial metabolism, and to calculate the theoretical increment of the target product concentration based on the maximum growth rate of microorganisms, substrate concentration and cell concentration. The concentration increment deviation rate calculation module is used to calculate the concentration increment deviation rate of the target product based on the real-time concentration increment of the target product and the theoretical concentration increment of the target product. The calibration judgment module is used for: The concentration increment deviation rate of the target product is compared with the preset deviation threshold to determine whether it is necessary to initiate the calibration process for microbial metabolism-related parameters in the digital twin model. When the concentration increment deviation rate of the target product is less than the preset deviation threshold, it is determined that the calibration process for the microbial metabolism-related parameters in the digital twin model does not need to be initiated. Otherwise, it is determined that a calibration process for the microbial metabolism-related parameters in the digital twin model needs to be initiated; The calibration module is used to obtain the target deviation value between the theoretical increase in the concentration of the target product and the theoretical increase in the concentration of the target product when the calibration process for microbial metabolism-related parameters in the digital twin model needs to be initiated. The module then adjusts the generation coefficients of growth-related products and non-growth-related products based on the target deviation value until the concentration increment deviation rate of the target product is less than a preset deviation threshold.

[0016] Compared with the prior art, the beneficial effects of the present invention are: This invention utilizes causal knowledge graphs and incremental update algorithms to quickly identify data changes and their causes, automatically adjust model parameters, and update only the changed data, avoiding frequent refreshes of the entire model. This ensures real-time synchronization between the digital twin model and the actual brewing process while improving the system's response speed and computational efficiency. When data anomalies are identified, a neural network model is used for prediction, simulating the impact of deviations on the subsequent brewing process in advance and generating optimized control commands, shifting from post-event correction to pre-event prevention. By calculating state confidence levels and dynamically adjusting deviation thresholds, the updated model state is evaluated and warned. In cases of low confidence, manual intervention is guided to quickly correct the model. This significantly improves the system's intelligence and robustness, providing high-quality technical support for intelligent monitoring of the liquor brewing process and contributing to improved product quality and production efficiency. Attached Figure Description

[0017] Figure 1 This is a flowchart of the intelligent monitoring system for the Baijiu brewing process based on multimodal perception and digital twins, as described in this invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] To address the shortcomings of existing monitoring systems, which often rely on single sensors for localized monitoring and lack comprehensive, real-time, and in-depth fusion sensing of multi-dimensional key information such as mash status, microbial activity, and aroma components; and to further address the technical challenges of current process optimization lacking a high-fidelity virtual experimental environment that hinders predictive control and adaptive optimization, please refer to [link to relevant documentation]. Figure 1 This embodiment provides the following technical solution: A smart monitoring system for the Baijiu brewing process based on multimodal perception and digital twins includes: The multimodal sensing unit is configured based on a passive sensing radio frequency chip and multimodal sensors to collect real-time data on raw material physical parameters, material state image parameters, liquor indicators, environmental parameters, and process parameters during the brewing process. This data is then fused and processed to form multimodal data. The multimodal sensing unit includes: The parameter acquisition module is configured to embed passive sensor radio frequency chips in brewing raw materials such as sorghum and wheat to monitor the physical parameters of the raw materials in real time, such as moisture, temperature, and humidity, as well as information on metabolic products during microbial fermentation. It also combines multi-mode sensors such as temperature sensors, humidity sensors, pressure sensors, and gas sensors to collect brewing environmental parameters and process parameters, such as temperature, humidity, and carbon dioxide concentration in the fermentation tank.

[0020] The image acquisition module is configured to install high-definition cameras at key locations in the brewing workshop, such as distillation pots and fermentation tanks, and to use visual inspection technology to monitor and analyze the material status during the brewing process in real time, such as the stacking shape of raw materials and the color change of the liquor. Through image recognition algorithms, it automatically detects the maturity of raw materials and the clarity of the liquor, providing rich data support for subsequent monitoring and decision-making.

[0021] The data fusion module is configured to integrate and organize the collected data into multimodal data. For example, it uses filtering algorithms to remove outliers from moisture data collected by passive sensor RF chips, ensuring data accuracy; it performs grayscale and binarization processing on image data collected by visual inspection technology to facilitate subsequent feature extraction and analysis; it converts image data into numerical feature vectors using feature extraction algorithms and fuses them with data from RF chip sensors; it employs data fusion algorithms to assign different weights to data from different sources based on data importance and reliability, combined with practical needs such as different brewing stages, different raw material batches, and different environmental conditions; and it uses weighted fusion algorithms to perform weighted summation or weighted averaging of data with different weights to obtain comprehensive data, thus more accurately reflecting the fermentation state of the raw materials. For example, when judging the fermentation state of raw materials, moisture data may have a greater impact, so it is assigned a higher weight; while raw material accumulation morphology image data, although it can also reflect the fermentation state, has a relatively smaller impact and is assigned a lower weight. Weighting: In the early stages of fermentation, temperature and humidity data are more important, so their weight can be increased. In the later stages of fermentation, microbial metabolite data and liquor index data are more critical, and the weighting should be adjusted accordingly. Calibration of data from different sources and of different types is crucial, and the correlation between different data points is analyzed to identify characteristic information affecting the quality indicators of baijiu brewing, thereby improving the accuracy and reliability of the data. For example, by comparing data collected by multiple sensors at the same time, the deviation value is calculated, and the data is adjusted to ensure consistency and accuracy. Combining raw material moisture data monitored by radio frequency chip sensors with raw material accumulation morphology image data identified by visual inspection devices reveals that excessively high raw material moisture may cause changes in the raw material accumulation morphology, such as clumping or uneven accumulation. Through correlation analysis, a comprehensive judgment can be made on whether the fermentation state of the raw materials is normal. If excessively high raw material moisture and abnormal accumulation morphology are detected, early warnings can be issued, reminding operators to take measures such as adjusting the amount of raw materials added and improving ventilation conditions to ensure the smooth progress of the fermentation process.

[0022] The twin modeling unit is configured to construct a digital twin model of the Baijiu brewing process based on virtual reality technology. It receives multimodal data in real time and uses an incremental update algorithm to automatically adjust changing parameters in the model, ensuring real-time synchronization between the digital twin model and the actual brewing process. This accurately reflects various physical, chemical, and biological changes during brewing. A neural network model is constructed, using multimodal data as input and combining it with historical brewing data to analyze the brewing process and predict the fermentation endpoint and liquor quality indicators. Based on the prediction results, optimization instructions are generated, such as adjusting the insulation layer of a specific fermentation pit or regulating the flow rate and temperature of a section of pipeline, dynamically adjusting brewing process parameters to ensure product quality. The twin modeling unit includes: The dynamic feedback module is configured to introduce a dynamic feedback mechanism, transmitting real-time multimodal data to the digital twin model through redundant communication. For example, using dual-link communication technology, when the main link fails, it automatically switches to the backup link to ensure continuous data transmission. Based on historical brewing data, a causal knowledge graph is constructed in the digital twin model. When the digital twin model receives real-time multimodal data, it uses an incremental update algorithm to perform reasoning in the causal knowledge graph, identifying whether there are changes in the real-time multimodal data, and further reasoning about the reasons for the data changes, such as: an abnormal rise in temperature in a certain fermentation pit, a sudden change in the color of the mash image in a certain area, analyzing whether it is caused by environmental disturbances (such as cooling system failure), process execution deviations (such as excessively high fermentation temperature), or potential root causes such as abnormal microbial activity. The system analyzes current data changes and identifies the root cause parameters that need adjustment in the causal knowledge graph based on the reasons for these changes. For example, it adjusts the heat transfer boundary conditions of the fermentation pit in the digital twin model and corrects the local activation parameters of the microbial growth model. For instance, if the inference results indicate that the abnormal temperature rise is due to a cooling system malfunction, the system will identify the heat transfer boundary condition parameters that need adjustment. Based on the root cause parameters, the system updates the parameters in the digital twin model, automatically adjusting the parameters that have changed. For example, if the temperature in the fermentation tank rises from 30℃ to 32℃, only the temperature parameter in the digital twin model is updated from 30℃ to 32℃, ensuring that the digital twin model remains consistent with the actual brewing process while avoiding frequent updates to the entire model, thus improving the system's response speed and computational efficiency.

[0023] The feedforward early warning module is configured to mark the changed data as anomalies when the incremental update algorithm identifies changes in real-time multimodal data; generate a data packet for this batch of data and feed it back to the neural network model for prediction; the neural network model generates the expected trend for the subsequent brewing process based on historical data and the current state; for example, the model predicts that the temperature in the fermentation tank will slowly rise by 2°C in the next hour; when the real-time multimodal data deviates from the expected result, the feedforward prediction process is immediately triggered, using a digital twin model to simulate the impact of the current deviation on the subsequent baijiu brewing process; for example, if the temperature rise trend at several key points is detected to be slightly faster than the model's expectation, such as the actual temperature rising by 3°C in one hour. This deviation is then used as a feedforward signal. A digital twin model predicts that if the temperature continues to rise at the current rate, fermentation may be too rapid, affecting the flavor and quality of the wine. The impact of this change on key parameters such as subsequent fermentation time and wine indicators can be estimated in advance. Based on the feedforward prediction results, optimized control commands are automatically generated to address potential changes. Strategies include adjusting the operating frequency of ventilation equipment, changing the stirring speed, and starting the cooling system in advance. For example, if the feedforward prediction indicates that a rapid temperature rise may lead to excessive fermentation, the optimized control strategy is to start the local ventilation equipment in advance to reduce the rate of temperature rise in the fermentation tank. This is immediately sent to the IoT control system to achieve real-time optimized control of the brewing equipment.

[0024] The updated evaluation module is configured to calculate the updated state confidence score by comparing the deviation between the neural network model's predicted values ​​and real-time multimodal data after the digital twin model is updated. Based on a preset maximum deviation threshold, it measures the consistency between the digital twin model's state and the actual environmental state. For example, if the neural network model predicts the temperature in the fermentation tank to be 30℃, while the actual perceived temperature is 32℃, and the preset maximum deviation threshold is 5℃, then the confidence score is calculated as 0.6. The deviation threshold in the confidence score calculation is dynamically adjusted based on historical data and real-time feedback information. For example, the deviation thresholds for certain parameters may change at different brewing stages or under different environmental conditions. By dynamically adjusting the thresholds, it can better adapt to various changes during the brewing process and improve the confidence score. The system optimizes the adaptability and accuracy of the calculation; based on the state confidence value, the local twin's state is divided into different confidence ratings, such as: high (>0.8), medium (0.5-0.8), and low (<0.5); when the deviation between the digital twin model and the actual environment state still cannot be converged after the digital twin model is updated, the confidence rating of the local twin state is automatically reduced, and a low confidence warning is issued; for example, different colored icons or text are used to indicate the current state confidence rating; low confidence states are highlighted with red icons or text, along with detailed analysis of the causes of the anomalies and suggested measures; the system allows users to view the actual environment state to which the low confidence belongs and compare it with the virtual state in the digital twin model; the confirmed results are fed back into the digital twin model to quickly correct the model parameters.

[0025] The IoT control unit is configured to precisely regulate actuators such as valves, fans, and heating devices based on optimized commands. For example, during fermentation, when the liquid flow rate in the fermentation tank needs adjustment, a command is sent to the valve actuator to precisely control the valve opening, ensuring the liquid flow rate meets process requirements. When the digital twin model predicts that the temperature in the fermentation tank is too high, the fan is activated in advance to increase ventilation and lower the temperature. When the temperature in the fermentation tank is lower than the set value, the heating device is automatically activated to ensure the fermentation temperature meets process requirements. The IoT control unit includes: The closed-loop feedback module is configured to connect to the actuator. After executing the optimized control command, it monitors the control results in real time and feeds the results back to the digital twin model. Based on the feedback results, it further adjusts the optimized command to form a closed-loop control. Through real-time monitoring and feedback, it ensures that the digital twin model is consistent with the actual brewing process. It can not only respond quickly to current changes, but also intervene in advance through feedforward prediction, transforming the system's response from post-event correction to in-process intervention and pre-event prevention, significantly improving the quality and robustness of control.

[0026] The fault detection module is configured to monitor the operating status of the actuators in real time, promptly detect equipment faults, and issue early warning signals. For example, when a blower malfunctions, an alarm is immediately issued, and operators are reminded to inspect and repair the equipment through a visual interface. The fault information is also fed back to the digital twin model, which automatically adjusts and optimizes strategies based on the fault situation to ensure the stability of the brewing process. For example, when a blower malfunction leads to insufficient ventilation, the power of the heating device is automatically adjusted to reduce the fermentation temperature and prevent the fermentation process from being affected.

[0027] The visualization and interaction unit is configured to intuitively display the operating status and control results of the actuators through a visual interface. Operators can manually adjust brewing process parameters or intervene in equipment operation through the interface; for example, operators can manually adjust the temperature setpoint in the fermentation tank or start / stop the operation of a certain device through the interface; and the operation commands are immediately transmitted to the IoT control unit to achieve real-time intervention in the brewing process; when an abnormality is detected, an alarm is immediately issued, and detailed abnormal information and suggested measures are displayed on the visualization interface to gain a deeper understanding of various states and changes in the brewing process; for example, when the temperature in the fermentation tank rises abnormally, the system will issue an alarm and... The interface displays an abnormally high temperature in the fermentation tank; possible cause: cooling system malfunction; recommended measures: check the cooling system's operating status, and manually start the backup cooling equipment if necessary. This not only improves operators' intuitive understanding of the brewing process but also facilitates timely problem detection and intervention, enhancing the system's usability and user experience. Furthermore, the visual interface provides historical data query and analysis functions, allowing operators to query parameter changes and equipment operating status throughout the brewing process. For example, operators can query the temperature change curve in the fermentation tank over the past week, analyzing the frequency and causes of temperature anomalies, providing strong support for optimizing and improving the brewing process.

[0028] The beneficial effects achieved by the above are as follows: By utilizing causal knowledge graphs and incremental update algorithms, data changes and their causes can be quickly identified, model parameters can be automatically adjusted, and updates can be performed only on data that has changed, avoiding frequent refreshes of the entire model. This ensures that the digital twin model is synchronized with the actual brewing process in real time, while improving the system's response speed and computational efficiency. When data anomalies are identified, predictions are made through neural network models to simulate the impact of deviations on the subsequent brewing process in advance and generate optimized control instructions, realizing the shift from post-event correction to pre-event prevention. Furthermore, by calculating state confidence and dynamically adjusting deviation thresholds, the updated state of the model can be evaluated and warned. In cases of low confidence, manual intervention can be guided to quickly correct the model. This significantly improves the system's intelligence level and robustness, providing high-quality technical support for intelligent monitoring of the liquor brewing process, and contributing to improved product quality and production efficiency.

[0029] Working Principle: Based on virtual reality technology, a digital twin model is constructed to receive multimodal data in real time. A dynamic feedback mechanism is introduced, employing an incremental update algorithm combined with a causal knowledge graph for reasoning. This identifies data changes and their causes, adjusting the parameters in the model to maintain synchronization with the actual brewing process. When data anomalies are identified, the data is fed back to the neural network model for prediction, generating the expected trend for the subsequent brewing process. When deviations occur between real-time data and expected results, a feedforward prediction process is triggered to simulate the impact of the deviation on subsequent brewing and generate optimized control commands. By comparing the deviation between the neural network model's predicted values ​​and real-time data, the state confidence level is calculated, and the deviation threshold is dynamically adjusted. When the confidence level is low, manual intervention is guided to correct the model. The optimized commands drive the actuators for precise control, and the closed-loop feedback module monitors the control results and feeds them back to the digital twin model, forming a closed-loop control. A visual interface displays the equipment's operating status and control results, providing anomaly alarms, suggested measures, and historical data query and analysis functions. This allows operators to monitor and intervene in the brewing process in real time, ensuring the system's reliability and intelligence under complex operating conditions, and improving the quality and efficiency of baijiu brewing.

[0030] In one embodiment, an intelligent monitoring system for the Baijiu (Chinese liquor) brewing process based on multimodal perception and digital twins is provided, characterized by an update evaluation module, including: The monitoring submodule is used to continuously monitor the deviation between the neural network model prediction value and the real-time multimodal data, calculate the updated state confidence, and determine the multimodal data combination and process context corresponding to the local twin whose state confidence is continuously lower than the preset threshold based on the continuous monitoring results. The analysis submodule is used for: Based on the configuration of the low confidence event statistics window and frequency threshold on the management terminal, and based on the configuration results, the multimodal data combination and process context corresponding to the local twin are analyzed. When the number of times the same type of data combination and process context triggers a low confidence state in the low confidence event statistics window exceeds the frequency threshold, a causal discovery trigger signal is generated. Based on the causal discovery trigger signal, multimodal data time series associated with low confidence states are extracted from historical databases and real-time multimodal data, and corresponding complete process parameter records and environmental parameter records are extracted simultaneously to obtain the dataset to be analyzed; The dataset to be analyzed is time-series aligned and feature-processed to construct a feature-state matrix. Based on a preset association rule mining strategy, the co-occurrence patterns between each feature item in the feature-state matrix and low-confidence states are scanned to determine the corresponding confidence and lift index. Based on a preset association threshold, association rules with lift index greater than the first threshold and confidence greater than the second threshold are selected, and the feature items corresponding to the association rules are identified as a potential unrecorded causal factor candidate set. The potential unrecorded causal factor candidate set is compared with the causal nodes recorded in the causal knowledge graph. Duplicate factors are eliminated to obtain the effective unrecorded causal factor set. Based on the business agreement, the action path of the brewing process is constructed for each factor in the effective unrecorded causal factor set in the digital twin model. Based on the action path, perform controlled variable simulation in the digital twin model, and determine whether the low-confidence state is reproduced based on the controlled variable simulation results; When the probability of low-confidence states changes after the existence of a moderating factor, the current factor is determined to be a new causal factor that has been verified. At the same time, the quantitative relationship between the new causal factor and the low-confidence state is determined based on the simulation results of the control variables. Based on the determination results, the new causal factor, the action path, and the quantitative relationship between the new causal factor and the low-confidence state are fused and stored in the causal knowledge graph. The fusion and storage results are synchronized to the training dataset of the neural network model, and the prediction mechanism of the neural network model is optimized based on the synchronization results.

[0031] In this embodiment, feature processing includes converting continuous data in the dataset to be analyzed into state intervals, extracting image parameters into feature vectors, and encoding the process context into structured labels.

[0032] In this embodiment, the simulation of control variables refers to the simulation of the state of the independent adjustment factor while keeping other recorded factors unchanged, and observing and recording whether the low-confidence state is reproduced in the simulation.

[0033] In this embodiment, the low confidence event statistics window refers to a time range or quantity range that is pre-set by the system for aggregating and statistically analyzing similar abnormal events. It serves as a time benchmark for determining whether an anomaly "recurs".

[0034] In this embodiment, the frequency threshold refers to the minimum number of abnormal events required to trigger subsequent in-depth causal analysis within the statistical window, serving as a threshold value to distinguish between occasional noise and systemic problems.

[0035] In this embodiment, the causal discovery trigger signal refers to an internal instruction automatically generated by the system when the cumulative frequency of abnormal events exceeds the frequency threshold, which is used to initiate the complete causal discovery process from data mining to simulation verification.

[0036] In this embodiment, the dataset to be analyzed refers to the collection of all raw data, derived data, and metadata associated with the target low-confidence event, extracted from historical storage and real-time streams in response to a causal discovery trigger signal.

[0037] In this embodiment, the feature-state matrix refers to a structured two-dimensional data table composed of time-series data, image features, process parameters, and other multi-dimensional information in the dataset to be analyzed, after cleaning, transformation, and encoding, together with "low confidence" or "normal" state labels. It is the direct input to the association rule mining algorithm.

[0038] In this embodiment, the potential unrecorded causal factor candidate set refers to the set of data features or parameters that have been initially screened by association rule mining strategy, have significant statistical association with low confidence states (such as high lift, high confidence), but have not yet been modeled as causal relationships in the existing knowledge base of the system.

[0039] In this embodiment, the effective unrecorded causal factor set refers to the set of truly novel and unverified unknown causes remaining after removing factors that have been covered or explained by existing causal knowledge graphs from the potential candidate set.

[0040] In this embodiment, the action path refers to the specific physical, chemical, or biodynamic influence mechanism assumed for each factor to be verified in the virtual environment of the digital twin model, describing how the factor gradually leads to the final observable state deviation.

[0041] In this embodiment, the verified new causal factors refer to factors that have been proven to have a clear, stable and repeatable impact on the system state through controlled variable simulation. These factors represent new knowledge discovered by the system through its own analysis and will be formally incorporated into the system's cognitive framework.

[0042] The working principle and beneficial effects of the above technical solution are as follows: by continuously monitoring prediction deviations and automatically triggering the causal discovery process, it can proactively identify the unknown causes behind recurring abnormal patterns, extract potential causal relationships from multi-source data, and verify them using digital twin simulation. The confirmed new knowledge is integrated into the core model of the system, which can not only correct current deviations, but also continuously accumulate and transform experience, so that the prediction mechanism in the neural network model can continuously evolve with the increase of brewing batches, thereby significantly improving the advance warning of the state of complex brewing processes and the accuracy of control strategies.

[0043] In one embodiment, a smart monitoring system for the Baijiu brewing process based on multimodal perception and digital twins is provided, further comprising: a fermentation product concentration increment assessment unit, configured as follows: The real-time concentration increment calculation module is used to collect the measured values ​​of the target product concentration at two adjacent data acquisition times during the fermentation process of baijiu based on the multimodal sensing unit; and to calculate the real-time increment of the target product concentration. ; in, This indicates the real-time increase in the concentration of the target product; , These represent the measured values ​​of the target product concentration at two consecutive data acquisition times; The concentration theoretical increment calculation module is used to obtain the maximum specific growth rate of microorganisms, the substrate concentration and cell concentration of the reaction raw materials during microbial metabolism, and to calculate the theoretical increment of the target product concentration based on the maximum growth rate of microorganisms, substrate concentration and cell concentration. ; in, This represents the theoretical increase in the concentration of the target product; Indicates the coefficient of growth-related product formation; Indicates the generation coefficient of non-growth-related products; Indicates the maximum specific growth rate of microorganisms; This indicates the substrate concentration corresponding to the reactants in the microbial metabolic process; Indicates cell concentration; Represents the saturation constant; Indicates the time interval between two consecutive data collections; The concentration increment deviation rate calculation module is used to calculate the concentration increment deviation rate of the target product based on the real-time concentration increment of the target product and the theoretical concentration increment of the target product. ; in, This indicates the deviation rate of the concentration increment of the target product; The calibration judgment module is used for: The concentration increment deviation rate of the target product is compared with the preset deviation threshold to determine whether it is necessary to initiate the calibration process for microbial metabolism-related parameters in the digital twin model. When the concentration increment deviation rate of the target product is less than the preset deviation threshold, it is determined that the calibration process for the microbial metabolism-related parameters in the digital twin model does not need to be initiated. Otherwise, it is determined that a calibration process for the microbial metabolism-related parameters in the digital twin model needs to be initiated; The calibration module is used to obtain the target deviation value between the theoretical increase in the concentration of the target product and the theoretical increase in the concentration of the target product when the calibration process for microbial metabolism-related parameters in the digital twin model needs to be initiated. The module then adjusts the generation coefficients of growth-related products and non-growth-related products based on the target deviation value until the concentration increment deviation rate of the target product is less than a preset deviation threshold.

[0044] In this embodiment, adjusting the growth-related product generation coefficient and the non-growth-related product generation coefficient according to the target deviation value until the concentration increment deviation rate of the target product is less than a preset deviation threshold includes: reading the target deviation value; when the target deviation value is positive, increasing the initial growth-related product generation coefficient and the non-growth-related product generation coefficient according to a preset ratio; when the target deviation value is negative, decreasing the initial growth-related product generation coefficient and the non-growth-related product generation coefficient according to a preset ratio; and updating the adjusted coefficients to the fermentation metabolism parameter library of the digital twin model. The preset ratio is determined based on the correlation analysis of the product concentration increment deviation with the growth-related product generation coefficient and the non-growth-related product generation coefficient in the historical fermentation data of Baijiu brewing, and the range of 0.01 to 0.1 is defined in combination with the microbial metabolic characteristics of different fermentation stages, and is dynamically adapted by the system according to the real-time fermentation conditions.

[0045] In this embodiment, the preset deviation threshold is a configurable process parameter. It is set based on a comprehensive evaluation of measurement error, production process stability requirements, and model reliability. Its function is to act as a switch to trigger adaptive calibration of the digital twin model parameters. When setting it, it is typically based on experience and quality standards in the liquor brewing process to determine the acceptable range of product concentration increment fluctuations without affecting the final liquor flavor and quality, the measurement accuracy and noise level of the multimodal sensing unit (such as a sensor), and other conditions. The threshold needs to be greater than normal measurement fluctuations to avoid frequent calibration triggers due to minor errors. For example, it can be set to... In this embodiment, Used to calculate the specific growth rate of organisms during the fermentation of baijiu. The relative growth rate of microbial cell concentration per unit time is the core kinetic indicator for quantifying microbial metabolic activity. When the substrate concentration is sufficient ( ): Denominator The formula simplifies to That is, microorganisms grow at their maximum specific growth rate. Growth (corresponding to the early stage of baijiu fermentation, when the raw materials are saccharified to produce a large amount of fermentable sugars and yeast proliferates rapidly). When the substrate concentration is low ( ): Denominator The formula simplifies to That is, the specific growth rate of microorganisms With substrate concentration It shows a linear positive correlation (corresponding to the middle and late stages of baijiu fermentation, when the consumption of fermentable sugars decreases and the growth rate of microorganisms decreases as the sugar content decreases). When the substrate concentration is appropriate ( ): This is when the growth rate reaches half of its maximum value, marking the transition from rapid growth to a slower pace.

[0046] The working principle and beneficial effects of the above technical solution are as follows: By collecting and calculating the changes in product concentration during the fermentation process in real time and comparing them with the theoretical increment based on the microbial growth and metabolism model, it is possible to dynamically assess whether the fermentation state deviates from expectations; when the deviation exceeds the set threshold, the parameter calibration mechanism can be automatically triggered to adaptively adjust the key biological metabolic coefficients in the model, thereby continuously improving the prediction accuracy of the digital twin model; it helps to achieve refined and automated monitoring of the brewing process, improve product quality stability, reduce human intervention, and provide reliable data support for process optimization.

[0047] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or high-voltage switchgear that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or high-voltage switchgear.

[0048] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A smart monitoring system for the Baijiu brewing process based on multimodal perception and digital twin, characterized in that: include: The multimodal sensing unit is configured to collect raw material physical parameters, material state image parameters, wine indicators, environmental parameters and process parameters in real time during the brewing process based on a passive sensing radio frequency chip and a multimodal sensor, and then integrate and process them to form multimodal data. The twin modeling unit is configured to construct a digital twin model of the liquor brewing process based on virtual reality technology, and receive multimodal data in real time. It uses an incremental update algorithm to automatically adjust the parameters that have changed in the model, so that the digital twin model is synchronized with the actual brewing process in real time. A neural network model is constructed, multimodal data is used as input, and historical brewing data is combined to analyze the brewing process and predict the fermentation endpoint and wine quality indicators; based on the prediction results, optimization instructions are generated to dynamically adjust the brewing process parameters. The Internet of Things (IoT) control unit is configured to drive the actuators for precise control based on optimized instructions.

2. The intelligent monitoring system for the Baijiu brewing process based on multimodal perception and digital twin as described in claim 1, characterized in that, The twin modeling unit includes: The dynamic feedback module is configured to introduce a dynamic feedback mechanism and transmit real-time multimodal data to the digital twin model through a redundant communication mechanism. Based on historical brewing data, a causal knowledge graph is constructed in a digital twin model; When the digital twin model receives real-time multimodal data, it uses an incremental update algorithm to reason in the causal knowledge graph, identify whether there are changes in the real-time multimodal data, and further reason about the reasons for the changes in the data. Based on the cause of the change, identify the root cause parameters that need to be adjusted in the causal knowledge graph; Based on the root cause corresponding parameters, parameters are updated in the digital twin model, and parameters that have changed in the model are automatically adjusted.

3. The intelligent monitoring system for the Baijiu brewing process based on multimodal perception and digital twin as described in claim 2, characterized in that, The twin modeling unit also includes: The feedforward early warning module is configured to mark the changed data as abnormal when the incremental update algorithm identifies changes in real-time multimodal data. The data from this batch is generated into a data package and fed into a neural network model for prediction. The neural network model generates the expected trend for the subsequent brewing process based on historical data and the current state. When the real-time multimodal data deviates from the expected results, the feedforward prediction process is immediately triggered, and the digital twin model is used to simulate the impact of the current deviation on the subsequent liquor brewing process. Based on the feedforward prediction results, optimized control commands are automatically generated to respond to potential changes.

4. The intelligent monitoring system for the Baijiu brewing process based on multimodal perception and digital twin as described in claim 2, characterized in that, The twin modeling unit also includes: The updated evaluation module is configured to calculate the updated state confidence by comparing the deviation between the neural network model prediction and the real-time multimodal data after the digital twin model is updated, and measure the consistency between the digital twin model state and the actual environment state based on the preset maximum deviation threshold. Based on historical data and real-time feedback information, dynamically adjust the deviation threshold in the confidence calculation; Based on the state confidence value, the state of the local twin is divided into different confidence ratings. When the deviation between the digital twin model and the actual environmental state still cannot be converged after the digital twin model is updated, the confidence rating of the local twin state is automatically reduced and a low confidence warning is issued. Examine the actual environmental state corresponding to the low confidence level and compare it with the virtual state in the digital twin model; feed the confirmed results back into the digital twin model to quickly correct the model parameters.

5. The intelligent monitoring system for the Baijiu brewing process based on multimodal perception and digital twin as described in claim 4, characterized in that, Update the evaluation module, including: The monitoring submodule is used to continuously monitor the deviation between the neural network model prediction value and the real-time multimodal data, calculate the updated state confidence, and determine the multimodal data combination and process context corresponding to the local twin whose state confidence is continuously lower than the preset threshold based on the continuous monitoring results. The analysis submodule is used for: Based on the configuration of the low confidence event statistics window and frequency threshold on the management terminal, and based on the configuration results, the multimodal data combination and process context corresponding to the local twin are analyzed. When the number of times the same type of data combination and process context triggers a low confidence state in the low confidence event statistics window exceeds the frequency threshold, a causal discovery trigger signal is generated. Based on the causal discovery trigger signal, multimodal data time series associated with low confidence states are extracted from historical databases and real-time multimodal data, and corresponding complete process parameter records and environmental parameter records are extracted simultaneously to obtain the dataset to be analyzed; The dataset to be analyzed is time-series aligned and feature-processed to construct a feature-state matrix. Based on a preset association rule mining strategy, the co-occurrence patterns between each feature item in the feature-state matrix and low-confidence states are scanned to determine the corresponding confidence and lift index. Based on a preset association threshold, association rules with lift index greater than the first threshold and confidence greater than the second threshold are selected, and the feature items corresponding to the association rules are identified as a potential unrecorded causal factor candidate set. The potential unrecorded causal factor candidate set is compared with the causal nodes recorded in the causal knowledge graph. Duplicate factors are eliminated to obtain the effective unrecorded causal factor set. Based on the business agreement, the action path of the brewing process is constructed for each factor in the effective unrecorded causal factor set in the digital twin model. Based on the action path, perform controlled variable simulation in the digital twin model, and determine whether the low-confidence state is reproduced based on the controlled variable simulation results; When the probability of low-confidence states changes after the existence of a moderating factor, the current factor is determined to be a new causal factor that has been verified. At the same time, the quantitative relationship between the new causal factor and the low-confidence state is determined based on the simulation results of the control variables. Based on the determination results, the new causal factor, the action path, and the quantitative relationship between the new causal factor and the low-confidence state are fused and stored in the causal knowledge graph. The fusion and storage results are synchronized to the training dataset of the neural network model, and the prediction mechanism of the neural network model is optimized based on the synchronization results.

6. The intelligent monitoring system for the Baijiu brewing process based on multimodal perception and digital twin as described in claim 3, characterized in that, The IoT control unit includes: The closed-loop feedback module is configured to connect to the actuator. After executing the optimized control command, it monitors the control result in real time and feeds the result back to the digital twin model. Based on the feedback result, it further adjusts the optimized command to form a closed-loop control. The fault detection module is configured to monitor the operating status of the actuator in real time, detect equipment faults in a timely manner and issue early warning signals; and feed the fault information back to the digital twin model to automatically adjust and optimize strategies based on the fault situation.

7. The intelligent monitoring system for the Baijiu brewing process based on multimodal perception and digital twin as described in claim 1, characterized in that, The multimodal sensing unit includes: The parameter acquisition module is configured to embed a passive sensor radio frequency chip in the brewing raw materials to monitor the physical parameters of the raw materials in real time, as well as the information on metabolites during the microbial fermentation process; it also combines multi-mode sensors to collect brewing environment parameters and process parameters. The image acquisition module is configured to install high-definition cameras at key locations in the brewing workshop and use visual inspection technology to monitor and analyze the material status in real time during the brewing process; and through image recognition algorithms, it automatically detects the maturity of raw materials and the clarity of the wine.

8. The intelligent monitoring system for the Baijiu brewing process based on multimodal perception and digital twin as described in claim 7, characterized in that, The multimodal sensing unit further includes: The data fusion module is configured to merge and organize the collected data to form multimodal data. A data fusion algorithm is used to assign different weights to data from different sources based on their importance and reliability, and in combination with actual needs. A weighted fusion algorithm is then used to sum or average the data with different weights to obtain comprehensive data. Data from different sources and of different types are calibrated, and the correlation between different data is analyzed to identify characteristic information that affects the quality indicators of baijiu brewing.

9. The intelligent monitoring system for the Baijiu brewing process based on multimodal perception and digital twin as described in claim 1, characterized in that, Also includes: The visualization interaction unit is configured to display the operating status and control results of the actuator through a visual interface. Operators can manually adjust brewing process parameters or intervene in the operating status of the equipment through the interface. When an anomaly is detected, an alarm is immediately issued, and detailed anomaly information and suggested measures are displayed on the visual interface to gain a deeper understanding of the various states and changes in the brewing process; It also provides historical data query and analysis functions based on a visual interface, allowing operators to query parameter changes and equipment operating status during the historical brewing process.

10. The intelligent monitoring system for the Baijiu brewing process based on multimodal perception and digital twin as described in claim 1, characterized in that, Also includes: The fermentation product concentration increment assessment unit is configured as follows: The real-time concentration increment calculation module is used to collect the measured values ​​of the target product concentration at two adjacent data acquisition times during the fermentation process of baijiu based on the multimodal sensing unit. And based on the real-time increment of the target product concentration; The concentration theoretical increment calculation module is used to obtain the maximum specific growth rate of microorganisms, the substrate concentration and cell concentration of the reaction raw materials during microbial metabolism, and to calculate the theoretical increment of the target product concentration based on the maximum growth rate of microorganisms, substrate concentration and cell concentration. The concentration increment deviation rate calculation module is used to calculate the concentration increment deviation rate of the target product based on the real-time concentration increment of the target product and the theoretical concentration increment of the target product. The calibration judgment module is used for: The concentration increment deviation rate of the target product is compared with the preset deviation threshold to determine whether it is necessary to initiate the calibration process for microbial metabolism-related parameters in the digital twin model. When the concentration increment deviation rate of the target product is less than the preset deviation threshold, it is determined that the calibration process for the microbial metabolism-related parameters in the digital twin model does not need to be initiated. Otherwise, it is determined that a calibration process for the microbial metabolism-related parameters in the digital twin model needs to be initiated; The calibration module is used to obtain the target deviation value between the theoretical increase in the concentration of the target product and the theoretical increase in the concentration of the target product when the calibration process for microbial metabolism-related parameters in the digital twin model needs to be initiated. The module then adjusts the generation coefficients of growth-related products and non-growth-related products based on the target deviation value until the concentration increment deviation rate of the target product is less than a preset deviation threshold.

Images