Intelligent quantification method and system for automated tea beverage production

Abstract

The application discloses an intelligent quantitative method and system for automatic tea production, and relates to the technical field of intelligent quantification, which comprises inputting a tea production data set into a cross-modal contrast learning model to extract a multi-modal feature vector, calculating a distribution consistency score of the multi-modal feature vector, and predicting a real-time anomaly score through a residual regression method; when the real-time anomaly score exceeds a dynamic deviation threshold, an abnormal event signal is triggered and an abnormal area is marked in an AR tea production scene, a spatial interaction context is integrated and generated, and user sensing data is collected according to the spatial interaction context; a digital twin model is called to verify the feasibility of a virtual control parameter set, and a dynamic correction is made through a predictive control algorithm to generate a quantitative control instruction. The application realizes intelligent conversion from natural interaction to production control by predicting a real-time anomaly score and synchronously constructing an intention recognition model, and improves operation accuracy and production efficiency.

Description

Technical Field

[0001] This invention relates to the field of intelligent quantitative technology, and in particular to an intelligent quantitative method and system for automated tea beverage production. Background Technology

[0002] With the deep integration of Industrial Internet of Things (IIoT) and Artificial Intelligence (AI) technologies, the food and beverage production sector is accelerating its evolution towards intelligence and automation. Tea beverage production, as a continuous process involving multiple steps and parameters, faces the core challenge of achieving precise quantitative control of key process parameters. In recent years, data-driven control technologies have gradually replaced traditional PID control. The widespread adoption of industrial edge computing devices enables real-time processing of high-frequency sensor data, while multimodal learning frameworks are beginning to be applied to the joint representation of heterogeneous data. Furthermore, augmented reality (AR) human-computer interaction technology is being increasingly adopted in production sites, using a visual interface that overlays virtual and real elements to assist operators in quickly locating anomalies and adjusting parameters.

[0003] The core problem of current automated tea beverage production systems lies in the lack of a coordination mechanism between digital twin models and real-time control commands. Traditional digital twin technology focuses on offline parameter simulation. When parameters exceed limits, the traditional process control systems and digital twin systems currently deployed in industrial settings only trigger static alarms or execute fixed rules, without considering the effects of multi-objective coupling. Furthermore, due to the lack of cross-modal feature alignment, the virtual verification of digital twins struggles to accurately map the nonlinear relationships of real-world operating conditions, leading to simulation results of corrective commands deviating from reality. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides an intelligent quantitative method for automated tea beverage production to solve the problem of the lack of a collaborative optimization mechanism between digital twin models and real-time multi-objective control.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] In a first aspect, the present invention provides an intelligent quantitative method for automated tea beverage production, comprising,

[0008] Collect tea beverage production data, preprocess it, and obtain a tea beverage production dataset with synchronized timestamps;

[0009] The tea production dataset is input into a cross-modal contrastive learning model to extract multimodal feature vectors, the distribution consistency score of the multimodal feature vectors is calculated, and the real-time anomaly score is predicted by the residual regression method.

[0010] When the real-time anomaly score exceeds the dynamic deviation threshold, an anomaly event signal is triggered and the anomaly area is marked in the AR tea beverage production scene. The spatial interaction context is integrated and generated, and user sensor data is collected based on the spatial interaction context.

[0011] User sensor data is input into the intent recognition model. The multimodal feature encoding layer performs feature extraction and semantic encoding, the cross-modal attention layer performs dynamic weight allocation and feature alignment, and the decision output layer performs fully connected transformation to generate a set of virtual control parameters.

[0012] The feasibility of the virtual control parameter set is verified by calling the digital twin model, and dynamic correction is performed through predictive control algorithm to generate quantitative control commands;

[0013] The quantitative control instructions are executed, and the quantitative effect is evaluated using the process index deviation evaluation method. The quantitative control instructions are then optimized based on the evaluation results.

[0014] As a preferred embodiment of the intelligent quantitative method for automated tea beverage production according to the present invention, the tea beverage production data includes temperature data, flow rate data, stirring status data, image data summary features, and user interaction data.

[0015] The preprocessing includes anomaly detection and removal, sampling frequency unification and window construction, timestamp standardization, interpolation correction of missing or offset data, soft synchronization processing, and full-channel alignment synchronization.

[0016] An interpolation method based on the principle of temporal proximity and a soft synchronization strategy are used to align the timestamps and correct for missing data in the preprocessed tea production data, generating a timestamp-synchronized tea production dataset.

[0017] As a preferred embodiment of the intelligent quantitative method for automated tea beverage production described in this invention, the steps of inputting the tea beverage production dataset into a cross-modal contrastive learning model to extract multimodal feature vectors, calculating the distribution consistency score of the multimodal feature vectors, and predicting real-time anomaly scores using a residual regression method are as follows:

[0018] Interpolation alignment and sample construction are performed on the timestamp-synchronized tea beverage production dataset to generate a timestamp-synchronized cross-modal dataset;

[0019] A cross-modal contrastive learning model is used to extract feature vectors of sensor, image, and text parameters from the cross-modal dataset after timestamp synchronization, and a gating mechanism is used to fuse them to generate multimodal feature vectors.

[0020] The distribution consistency score of the multimodal feature vectors is obtained by calculating the Wasserstein distance of the multimodal feature vectors, and a real-time anomaly score is generated by the residual regression method.

[0021] As a preferred embodiment of the intelligent quantitative method for automated tea beverage production described in this invention, the steps are as follows: when the real-time anomaly score exceeds the dynamic deviation threshold, an anomaly event signal is triggered, and the anomaly area is marked in the AR tea beverage production scene; a spatial interaction context is generated; and user sensor data is collected based on the spatial interaction context.

[0022] When the abnormal score exceeds the dynamic deviation threshold, an abnormal event signal is triggered and the abnormal area is dynamically marked in the AR tea production scene according to the spatial calibration result. The abnormal event signal and the spatial calibration result are integrated, and the spatial interaction context is dynamically generated through AR annotation.

[0023] Based on the spatial interaction context, users are guided to interact with AR devices in real time, and user sensor data for abnormal areas are collected simultaneously.

[0024] As a preferred embodiment of the intelligent quantitative method for automated tea beverage production described in this invention, the user sensing data includes user hand image sequences, IMU data, and eye-tracking coordinates.

[0025] As a preferred embodiment of the intelligent quantitative method for automated tea beverage production according to the present invention, the intent recognition model is constructed using the following steps.

[0026] Configure a multi-branch neural network architecture and build a multimodal feature encoding layer, a cross-modal attention layer, and a decision output layer;

[0027] A cross-entropy loss function is used to perform backpropagation and gradient updates on the multimodal feature encoding layer, cross-modal attention layer, and decision output layer to construct an intent recognition model.

[0028] As a preferred embodiment of the intelligent quantitative method for automated tea beverage production according to the present invention, the specific steps for generating the virtual control parameter set are as follows:

[0029] Perform 3D convolution and pooling on the user's hand image sequence to generate spatiotemporal dynamic features;

[0030] One-dimensional convolution and temporal state encoding are performed on IMU data to extract motion pattern features;

[0031] Fully connected encoding and spatial region mapping are performed on eye-tracking coordinates to generate semantic feature vectors;

[0032] Spatiotemporal dynamic features, motion pattern features, and semantic feature vectors are input into a cross-modal attention layer for linear transformation and attention fusion to generate context-aware feature vectors.

[0033] The context-aware feature vectors are input into the decision output layer for feature abstraction and nonlinear transformation to generate a set of virtual control parameters.

[0034] As a preferred embodiment of the intelligent quantitative method for automated tea beverage production described in this invention, the steps of verifying the feasibility of the virtual control parameter set by calling a digital twin model and dynamically correcting it through a predictive control algorithm to generate quantitative control commands are as follows.

[0035] The mapping function, trained and continuously fine-tuned based on historical tea beverage data, accurately maps virtual control parameters to the set of actual process control variables required for the current tea beverage production task.

[0036] The actual process control variables are input into a high-fidelity digital twin model to predict the changing trends of key process physical variables in multiple future control cycles. The feasibility of the actual process control variable set is evaluated based on the physical coupling characteristics, and the corrected process control variable set is obtained.

[0037] Predictive control algorithms are used to dynamically correct initial process variables, optimize process performance over multiple cycles, and transform the corrected set of process control variables into quantitative control commands.

[0038] As a preferred embodiment of the intelligent quantitative method for automated tea beverage production described in this invention, the steps of executing quantitative control commands, evaluating the quantitative effect using a process index deviation evaluation method, and optimizing the quantitative control commands based on the evaluation results are as follows.

[0039] Interpolation methods are used to align control commands and sensor data in time, and a process index matrix is ​​constructed and normalized through standardized modeling of process indices.

[0040] Based on the time variation trend in the normalized process index matrix, the gradient of the index change rate within the time window is calculated. Based on the product of the index change rate gradient and the Hessian matrix, a time-varying weight vector is generated by the Softmax function. The time-varying weight vector is used as the driving force for multi-parameter perturbation simulation, and the pre-optimized instruction sequence is generated by the alternating direction multiplier method.

[0041] Based on the process index deviation and current deviation in the normalized process index matrix, a fuzzy rule model is used to infer the control correction amount. The correction is superimposed to generate new instructions and verified in a digital twin to obtain a verified optimized instruction set.

[0042] Secondly, this invention provides an intelligent quantitative system for automated tea beverage production, comprising:

[0043] The data acquisition module collects tea beverage production data, performs preprocessing, and obtains a timestamp-synchronized tea beverage production dataset.

[0044] The feature extraction module inputs the tea production dataset into the cross-modal contrastive learning model to extract multimodal feature vectors, calculates the distribution consistency score of the multimodal feature vectors, and predicts real-time anomaly scores using the residual regression method.

[0045] The anomaly response module triggers an anomaly event signal and marks the abnormal area in the AR tea beverage production scene when the real-time anomaly score exceeds the dynamic deviation threshold. It also integrates and generates a spatial interaction context and collects user sensor data based on the spatial interaction context.

[0046] The intent recognition module inputs user sensor data into the intent recognition model. The multimodal feature encoding layer performs feature extraction and semantic encoding, the cross-modal attention layer performs dynamic weight allocation and feature alignment, and the decision output layer performs fully connected transformation to generate a set of virtual control parameters.

[0047] The parameter verification module calls the digital twin model to verify the feasibility of the virtual control parameter set, and performs dynamic correction through predictive control algorithms to generate quantitative control commands.

[0048] The control execution module executes quantitative control commands, evaluates the quantitative effect using a process index deviation assessment method, and optimizes the quantitative control commands based on the assessment results.

[0049] The beneficial effects of this invention are as follows: by inputting the tea production dataset into a cross-modal contrastive learning model to extract multimodal feature vectors, calculating the distribution consistency score of the multimodal feature vectors, and predicting real-time anomaly scores through residual regression, early warning and localization of production anomalies are achieved, improving the detection capability for complex anomalies; simultaneously, an intent recognition model is constructed to transform the user's natural interaction behavior into precise control commands that can be executed by the machine, realizing the intelligent transformation from natural interaction to production control, and improving operational accuracy and production efficiency. Attached Figure Description

[0050] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 A flowchart for an intelligent quantitative method for automated tea beverage production.

[0052] Figure 2 A schematic diagram of an intelligent quantitative system for automated tea beverage production.

[0053] Figure 3A flowchart for generating a virtual control parameter set for an intelligent quantitative method for automated tea beverage production.

[0054] Figure 4 A flowchart for generating quantitative control instructions for an intelligent quantitative method in automated tea beverage production. Detailed Implementation

[0055] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0056] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0057] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0058] Reference Figures 1-4 As one embodiment of the present invention, this embodiment provides an intelligent quantitative method for automated tea beverage production, comprising the following steps:

[0059] S1: Collect tea beverage production data, preprocess it, and obtain a tea beverage production dataset with timestamp synchronization.

[0060] Specifically, the equipment that needs to collect data in the tea production line is selected from the automated tea production equipment. The equipment includes a temperature controller for collecting the temperature of the tea liquid, a volumetric flow meter for recording the liquid flow rate, a motor control device for monitoring the stirring status, a machine vision device for acquiring image data, and a human-computer interaction interface recording device for recording user interaction behavior. Each data acquisition node is assigned a unique acquisition channel number, and the data acquisition channels are combined to form a complete set of data acquisition channels, which constitutes the data source for tea production.

[0061] Data sources for tea beverage production include, but are not limited to, the following types: tea temperature data, flow rate data, stirring status information, visual image data summaries, and user operation event data, etc.

[0062] A unified sampling period is set, the sampling frequency of the complete set of data acquisition channels is synchronized, and within each sampling period, continuous observations of the current time point and several time periods before it are extracted to construct a data window of fixed length. For data acquisition channels whose original sampling frequency is lower than the unified period, the historical value preservation filling method is used to complete the data, resulting in an observation data set organized based on the sliding window method.

[0063] Anomaly detection is performed on the raw data of each channel to remove extreme values ​​that deviate from the mean. Based on the acquisition scheduling time recorded in the synchronization clock, the start time of each sliding window in the data set organized by the sliding window method is extracted as a standardized data timestamp. Based on the standardized data timestamp, a standardized timestamp index sequence is constructed. An interpolation method based on the principle of time proximity is used to correct the error of the sliding window data set with timestamp offset or missing data. A soft synchronization strategy is applied for independent asynchronous devices. After alignment and completion, the data set organized by the sliding window method is synchronized across all channels on a unified timestamp sequence to generate a timestamp-synchronized tea beverage production dataset.

[0064] S2: Input the tea production dataset into the cross-modal contrastive learning model to extract multimodal feature vectors, calculate the distribution consistency score of the multimodal feature vectors, and predict real-time anomaly scores using the residual regression method.

[0065] Specifically, the tea production dataset synchronized with timestamps is further aligned by using cubic spline interpolation to resample the original sensor data in a time series. The original sensor data is then divided into sampling windows with fixed time intervals (e.g., each sampling window covers 5 seconds) using a sliding window method. Each sampling window contains tea temperature data, flow rate data, stirring status information, visual image data summary, and user operation event data, etc.

[0066] To improve the ability to identify anomalies in modal data processing, text process parameters are perturbed in a portion of the sampling window (e.g., randomly replacing some parameter values ​​with manually set anomalies) to construct labeled negative sample data pairs. The original samples and negative samples are then combined to form a timestamp-synchronized cross-modal dataset with anomaly annotations.

[0067] In the feature extraction stage, a cross-modal contrastive learning model is constructed, which includes three types of modal feature encoders. The sensor modal feature encoder includes a time series encoder that integrates a bidirectional long short-term memory network (Bi-LSTM) and a self-attention mechanism. The multivariate time series matrix is ​​input into the Bi-LSTM network to obtain the bidirectional hidden state sequence, and self-attention weighted pooling is used to generate a fixed-length sensor feature vector. The image modal feature encoder uses an EfficientNet-B3 convolutional neural network as the backbone network and combines spatial pyramid pooling (SPP) to extract scale-robust image feature vectors. The text process modal feature encoder uses a pre-trained BERT language model (a transformer language model represented by a bidirectional encoder) to encode process parameters and connects to a fully connected layer to generate text process parameter embedding vectors.

[0068] The sensor modal feature encoder, image modal feature encoder, and text process modal feature encoder are trained. Specifically, positive sample pairs are formed by taking samples from the same time window (fixed-length sensor feature vector, image feature vector, and text process parameter embedding vector), and negative sample pairs are formed by taking samples from different time windows or combinations containing text interference. A cross-modal contrastive loss function is used to effectively guide the three modal encoders to gradually align their semantic representations in the feature space by minimizing the feature distance of positive sample pairs and maximizing the distance of negative sample pairs. The Adam optimizer is used to jointly train the three modal encoders and the contrastive loss function on a labeled sample set, thus completing the training of the sensor modal feature encoder, image modal feature encoder, and text process modal feature encoder.

[0069] Based on this, the Wasserstein distances of the sensor feature vector, image feature vector, and text process parameter feature vector within each time window are calculated sequentially. For example, the Wasserstein distance between the sensor feature vector and the image feature vector, the Wasserstein distance between the sensor feature vector and the text process parameter feature vector, and the Wasserstein distance between the image feature vector and the text process parameter feature vector are calculated.

[0070] The formula for calculating the Wasserstein distance is as follows:

[0071] ;

[0072] in, Represents multimodal feature vectors With multimodal feature vectors Wasserstein distance between them This represents the total number of multimodal feature vectors within a time window. Multimodal feature vectors The first in From individual feature vectors to multimodal feature vectors The first in The distance between each sub-feature vector Represents multimodal feature vectors The first in Sub-feature vectors and multimodal feature vectors The first in Transmission weight coefficients between sub-feature vectors The transmission weighting coefficient is defined based on the statistical distribution of the distance between each pair of multimodal feature vectors, and its value range is [value range missing]. .

[0073] The Wasserstein distance is normalized and reverse-mapped to generate distribution consistency scores for three sets of multimodal feature vectors. The distribution consistency scores are then weighted and summed to generate real-time anomaly scores within a time window.

[0074] S3: When the real-time anomaly score exceeds the dynamic deviation threshold, an anomaly event signal is triggered and the anomaly area is marked in the AR tea beverage production scene. The anomaly event signal and the spatial calibration result are integrated, and the spatial interaction context is dynamically generated through AR annotation.

[0075] Specifically, an anomaly event signal is triggered when the real-time anomaly score exceeds the dynamic deviation threshold. This signal includes the dynamic deviation threshold trigger status, the spatial location information of the anomaly event, dynamic visualization attributes of the anomaly score, associated process parameters, and a timestamp and equipment identifier. The dynamic deviation threshold is dynamically calculated using the moving average and standard deviation of historical anomaly scores, as shown in the following formula.

[0076] ;

[0077] in, Indicates time The moving average of historical anomaly scores, Indicates time The standard deviation of historical outlier scores This is the sensitivity adjustment coefficient. Indicates time The dynamic deviation threshold obtained from internal dynamic calculation. Indicates the current time step.

[0078] Given the known spatial layout of the process equipment, calibration objects (such as standard checkerboard or laser targets) are used in conjunction with sensors to synchronously acquire data from multiple perspectives. By calling open-source visual calibration tools (such as the ROS camera calibration package), the joint optimization of multiple calibration parameters, such as the camera intrinsic matrix, extrinsic matrix, lens distortion parameters, and time synchronization offset, is completed. Coordinate alignment and error convergence verification are then performed in the constructed 3D model of the tea production site, forming a unified spatial calibration result dataset that describes the spatial correspondence between the sensor, camera, and field equipment.

[0079] After an abnormal event signal is triggered, the location of the abnormal event signal is mapped to the three-dimensional coordinate system used in the AR (Augmented Reality) tea beverage production scene based on the spatial calibration result dataset, forming a list of abnormal event signal coordinates.

[0080] The spatial calibration results include, but are not limited to, the following: the intrinsic parameter matrix of the camera, such as focal length and principal point position; the extrinsic parameter matrix of the camera or depth sensor; distortion coefficients (such as radial distortion and tangential distortion); time synchronization parameters (used for data alignment between the camera and the sensor); and calibration error indicators (such as reprojection error).

[0081] In the AR tea beverage production scenario, each abnormal coordinate point in the abnormal event signal coordinate list is dynamically labeled with an abnormal area (dynamic label box and color). The color of the label box is dynamically adjusted according to the real-time abnormal score. For example, the red component is enhanced and the blue component is weakened when the real-time abnormal score is higher, and the transparency increases linearly with the score. The width of the label box remains fixed, and the height is adjusted proportionally according to the score.

[0082] By integrating anomalous event signals, spatial calibration result datasets, anomalous event signal coordinate list, and dynamic AR annotations (visual elements whose color and size are dynamically adjusted according to the score), a spatial interaction context is generated.

[0083] Based on the established spatial interaction context, the system renders abnormal areas in real time using AR devices and guides users to utilize the spatial location information marked by AR annotations. The abnormal areas are highlighted in the user's field of vision, and a data acquisition protocol is initiated simultaneously. The system uses the AR device's built-in sensors to capture real-time image sequences of the user's hands (focusing on gesture operations, such as pointing or touching the abnormal annotation box), IMU (Head Inertial Measurement Unit) data, and eye-tracking focus coordinates (to confirm whether the user's gaze point is aligned with the coordinates of the abnormal area). All collected data is automatically labeled with spatial context tags (such as abnormal device identifiers, AR coordinates, and timestamps) to ensure that the data is directly associated with the specific abnormal area.

[0084] S4: Input user sensor data into the intent recognition model, perform feature extraction and semantic encoding in the multimodal feature encoding layer, perform dynamic weight allocation and feature alignment in the cross-modal attention layer, and perform fully connected transformation in the decision output layer to generate a virtual control parameter set.

[0085] Specifically, a multi-branch neural network architecture is configured to build a multimodal feature encoding layer, a cross-modal attention layer, and a decision output layer. The steps are as follows: Independent feature encoding branches are used for user sensor data, with each feature encoding branch employing a modality-specific network structure (e.g., using CNN to extract spatial features for image data, using Transformer or LSTM to capture sequence dependencies for text data, and using 1D-CNN to process time series for sensor data). A normalization layer and ReLU activation function are added to the output of each feature encoding branch to generate high-dimensional feature vectors, thus completing the construction of the multimodal feature encoding layer. Multi-head attention is used to obtain the correlation weights between different high-dimensional feature vectors, achieving dynamic feature fusion and alignment, thus building a cross-modal attention layer. The fused multimodal features are input into a fully connected network for dimensionality reduction and nonlinear transformation, and an output layer is added for multi-class classification and linear regression, constructing the decision output layer.

[0086] The cross-entropy loss function is used to perform forward propagation on the multimodal feature encoding layer, cross-modal attention layer, and decision output layer to obtain the prediction error matrix. The prediction error matrix is ​​then backpropagated to generate the gradient tensor of each layer. The Adam optimizer is used to dynamically adjust the learning rate and update the gradient tensor of each layer to obtain the updated network weights. Based on the updated network weights, batch normalization and dropout layers are used to perform hyperparameter tuning on the multimodal feature encoding layer and cross-modal attention layer. Finally, cross-layer feature fusion is performed through residual connections to output the constructed intent recognition model.

[0087] Furthermore, the user sensor data is divided into training, validation, and test sets according to its characteristics (e.g., a 7:2:1 ratio). Linear interpolation is used to augment the features of the training set, and batch normalization is applied to standardize the data, creating enhanced training samples. During training, the Adam optimizer is used to dynamically schedule the learning rate of the enhanced training samples, and early stopping is employed for training monitoring to obtain intermediate model parameters. The cross-entropy loss function is used to quantize the loss of these intermediate parameters, yielding the prediction error on the validation set. When the prediction error on the validation set exceeds the convergence threshold, training terminates, and the trained intent recognition model is output synchronously.

[0088] It should be noted that the convergence threshold is defined based on the rate of change of the prediction error on the validation set, and its value ranges from 0.0001 to 0.01.

[0089] User sensor data is input into the intent recognition model. A multimodal feature encoding layer performs feature extraction and semantic encoding. Specifically, a 3D convolutional kernel is applied to the user's hand image sequence to extract spatiotemporal feature vectors of hand movements, including gesture shape and dynamic trajectory, through sliding operations in the spatial and temporal dimensions. Max pooling is then performed to reduce feature dimensionality, complexity, and feature robustness, generating spatiotemporal dynamic features that encode the dynamic changes of the gesture. For IMU data, a one-dimensional convolution is used to capture local temporal patterns (such as short-term motion abrupt changes) of sensor data such as acceleration and angular velocity. This is combined with a Long Short-Term Memory (LSTM) network for temporal state encoding to model the long-term dependencies of motion sequences (such as the complete action flow from walking to stopping), extracting motion pattern features that characterize the user's action patterns. For eye-tracking coordinate data, high-dimensional encoding is performed through a fully connected layer to map the original gaze coordinates to the semantic space. Spatial region mapping is then performed to associate the original gaze with specific objects (such as device buttons or abnormal areas), generating semantic feature vectors that reflect the user's attention focus.

[0090] Spatiotemporal dynamic features, motion pattern features, and semantic feature vectors are input into the cross-modal attention layer and linearly transformed to generate Q (query) matrix, K (key) matrix, and V (value) matrix. The Q matrix captures the "query requirement" of the current context (e.g., "which type of abnormal operation is the user's current gesture intent related to"), the K matrix encodes the "key information" of multimodal features (e.g., "the spatial association between the historical pattern of hand movements and the gaze point"), and the V matrix stores the "value content" of multimodal features (e.g., "the semantic label of the abnormal region and motion state details"). The dot product of the Q and K matrices is used as the association score, which is then normalized using the softmax function to generate the attention weight matrix. The attention weight matrix and the V matrix are then element-wise weighted and summed to generate the context-aware feature vector.

[0091] The context-aware feature vector is input into the decision output layer and abstracted through a multi-layer fully connected network. The high-order feature representation is gradually compressed and extracted. After the feature abstraction is completed, the abstracted context-aware feature vector is mapped to the actual dimension of the process parameters (such as temperature and flow rate) through a linear fully connected layer to generate a virtual control parameter set. The Sigmoid activation function is used to output a probability value between 0 and 1, which represents the reliability of the current virtual control parameter set (the closer the value is to 1, the more reliable the virtual control parameter set).

[0092] S5: Verify the feasibility of the virtual control parameter set by calling the digital twin model, and dynamically correct it through predictive control algorithm to generate quantitative control commands.

[0093] Specifically, a nonlinear mapping function, trained by the mapping relationship between historical tea production data and user control behavior, is used to convert each virtual control parameter in the virtual control parameter set into an actual process control variable in the tea production process. Furthermore, each virtual control parameter is mapped to a corresponding physical control variable. After the variable mapping process, a set of actual process control variables corresponding to the current gesture operation is obtained.

[0094] The nonlinear mapping function in the above steps is trained based on the correspondence between historical tea production data and user control behavior. In actual operation, it is continuously fine-tuned periodically by accepting new operation samples to improve the generalization ability and adaptability of variable mapping, so as to ensure that the mapping accuracy can be maintained under different tea recipes, equipment parameters or production environments.

[0095] By substituting the actual set of process control variables into a high-fidelity digital twin model, multi-step simulation predictions are performed on the current tea production process. During the simulation prediction process, based on the coupling relationship between the physical characteristics of tea production and control behavior (e.g., temperature control response time lag, liquid flow velocity inertia, sugar dissolution and diffusion behavior, stirring nonlinearity, and other physical coupling mechanisms), the key process change trends under multiple control cycles in the future are output within a set prediction time range. The prediction time range can be set to extend several control cycles forward from the current moment, obtaining a sequence of predicted values ​​for multi-dimensional physical quantities changing over time. For example, extending from the current moment t to t+5 control cycles, the predicted values ​​of key process variables such as temperature, sugar concentration, and tea liquid volume are output sequentially in each cycle.

[0096] The high-fidelity digital twin model in the above steps is based on the heat conduction equation, the Navier-Stokes fluid dynamics equation, and Fick's diffusion law. It constructs partial differential equations for the temperature field, flow velocity field, and concentration field to describe the core physical mechanism of tea production. It also uses an LSTM network to learn dynamic characteristics not covered by the physical model (such as the reduction in heating efficiency caused by equipment aging) for dynamic correction. Combined with dynamic parameter calibration and multi-timescale co-simulation, a high-fidelity digital twin model of multi-parameter coupled evolution trend is constructed.

[0097] Using a high-fidelity digital twin model, under the current set of process control variables and production conditions, the predicted value sequence of key process physical quantities (e.g., temperature, sugar concentration, and tea liquid volume) over time is predicted within the control cycle. The predicted value sequence is then compared with the target value sequence (e.g., the target process trajectory preset according to production tasks or quality standards) time-by-time. The predicted value sequence of multi-dimensional physical quantities over time is the dynamic evolution trajectory of key process physical variables (e.g., temperature, sugar concentration, and tea liquid volume) predicted by the high-fidelity digital twin model over several future control cycles, based on the current set of process control variables and production conditions. The selected predictive variables are determined based on the process criticality and equipment dynamic response characteristics. The prediction time window N is typically set to extend 3 to 10 control cycles forward from the current moment. The control cycle refers to the basic scheduling cycle of the control terminal, determined based on the equipment's response characteristics and control frequency, typically 1-2 seconds, and is configurable. The predicted value range for each physical quantity is set based on equipment constraints, process quality requirements, and historical statistical data. In the comparison process, each control cycle in the predicted value sequence of multidimensional physical quantities changing over time is used as the time unit to calculate the deviation between the predicted value and the target value. The deviation results of all control cycles and key variables are weighted and summarized to obtain the comprehensive process deviation score caused by the current set of input control variables.

[0098] Based on the current tea beverage production status information and the initial set of actual process control variables, and using the dynamic evolution trajectory of key process physical quantities as the core basis of the optimization model, a dual objective function is constructed, which includes minimizing the fitting error and minimizing the disturbance of control variables. Through constrained optimization modeling, the current tea beverage production status information, the actual set of process control variables, and the dual objective function are constructed into a constrained multivariate predictive control model. The multivariate predictive control model is combined with time series prediction methods to predict the changes in production status in future control cycles. In each future control cycle, a rolling optimization strategy is executed, and the control variables are optimized in real time based on the continuous update of the production status. Through the iterative solution process, a corrected set of process control variables that meets the current process requirements is obtained.

[0099] The modified set of process control variables is substituted into the specific tea beverage production equipment type and control interface protocol for adaptation mapping (for example, tea beverage production equipment types include electric heaters, water pumps, and multi-channel liquid distribution valves, and control interface protocols include Modbus RTU, CAN Bus, and OPC UA, etc.). Each modified process control variable is converted into a specific quantitative control command, and the quantitative control command is sent to the actuator or driver to control the actual tea beverage production equipment.

[0100] S6: Execute quantitative control instructions, evaluate the quantitative effect using process index deviation evaluation methods, and optimize quantitative control instructions based on evaluation results.

[0101] Specifically, sliding window statistics are performed on the data at each time point in the sensor data to remove outliers exceeding the standard deviation of the mean. For example, when a temperature sensor shows a sudden change of 200℃ (assuming the normal range is 0-100℃), it is marked as invalid. The sensor data includes real-time physical and chemical measurements related to the tea production process, such as temperature, pressure, flow rate, liquid level, conductivity, concentration, and color. Lagrange interpolation is used to align the timestamps of the control commands and the sensor sampling data. Based on the timestamp alignment and outlier-handled sensor data, a process index matrix is ​​constructed with time steps as rows and process standards as columns. The process index matrix is ​​then subjected to Min-Max normalization to eliminate the influence of index dimensions, resulting in a normalized process index matrix.

[0102] Based on the time change trend in the normalized process index matrix, the gradient of the rate of change of the index within the time window is calculated. For example, if the pressure index rises from 0.2 to 0.5 in the last 5 seconds, its rate of change gradient is (0.5-0.2) / 5=0.06 seconds. The Hessian matrix describing the coupling relationship between the process indexes is obtained through the open interface of the high-fidelity digital twin model. Based on the product of the rate of change gradient of the index and the Hessian matrix, a time-varying weight vector is generated after transformation by the Softmax function.

[0103] In a high-fidelity digital twin model, a predicted scenario of future time window content is established as the simulation environment support. Regular perturbations are applied to four types of parameters in the control instruction set: temperature, pressure, flow rate, and composition. Candidate control instruction sets for these four types of parameters are generated. The candidate instruction sets are executed one by one, and the deviations of future multi-step process indicators predicted by the candidate control instruction sets in the high-fidelity digital twin model are weighted and summed term by term with time-varying weight vectors. A penalty term for the change amplitude of the control quantity is added to construct the optimization objective function for rolling time domain optimization. The time-varying weight vector is the weighting coefficient of the optimization objective function. The ADMM algorithm (alternating direction multiplier method) is used to solve the constrained optimization problem and generate and pre-optimize the instruction sequence.

[0104] It should be noted that a control instruction set refers to a set of specific parameters that can be issued to the execution equipment within a certain control cycle to adjust the process, such as temperature control values ​​(set temperature), pressure set values ​​(pump or valve pressure), flow control values, and proportions such as tea / syrup / water ratio.

[0105] Based on the pre-optimized instruction sequence, and combined with the current process index deviation and the current deviation in the normalized process index matrix, basic feature components are formed. The rate of change of the basic feature components per unit time is calculated (e.g., if the temperature deviation decreases from 0.06 at the previous moment to 0.05 at the current moment, the rate of change is -0.01 / second), which is used as a dynamic trend component. The deviation values ​​of the control instruction set (static state, such as temperature, pressure, and tea volume, etc.) and the rate of change (dynamic trend, such as temperature change rate, nighttime change rate, and concentration decrease rate, etc.) are concatenated in order of index category to generate a composite feature vector containing static state and dynamic trend. The TSK type (Takagi–Sugeno–Kang type) fuzzy rule base is called for logical reasoning, and the fuzzy correction amount is obtained by defuzzification calculation.

[0106] The formula for defuzzification is as follows.

[0107] ;

[0108] in, Indicates the first The correction amount of the control parameters output by the fuzzy rule. Indicates the first The linear consequent parameter vector corresponding to the fuzzy rule. Representing vectors The transpose form, This represents the set of input feature vectors consisting of the deviations of current process parameters and their rates of change. Indicates the first The bias term of a fuzzy rule.

[0109] The correction amount is added to the initial instruction set according to the correction step size factor to generate new instructions. The stability and physical feasibility of the new instructions are verified in a high-fidelity digital twin model (e.g., limiting the temperature fluctuation range and energy consumption threshold). When the optimization fails to reach the convergence condition, random perturbations are injected into the control instruction set and the iteration process is restarted to generate a verified optimized instruction set.

[0110] It should be noted that the correction step size factor is set based on the dynamic characteristics of the control terminal and the correction confidence level. For example, the maximum value range is 0.1 to 0.5, which is used to suppress mutations.

[0111] This embodiment also provides an intelligent quantitative system for automated tea beverage production, including:

[0112] The data acquisition module collects tea beverage production data, performs preprocessing, and obtains a timestamp-synchronized tea beverage production dataset.

[0113] The feature extraction module inputs the tea production dataset into the cross-modal contrastive learning model to extract multimodal feature vectors, calculates the distribution consistency score of the multimodal feature vectors, and predicts real-time anomaly scores using the residual regression method.

[0114] The anomaly response module triggers an anomaly event signal and marks the abnormal area in the AR tea beverage production scene when the real-time anomaly score exceeds the dynamic deviation threshold. It also integrates and generates a spatial interaction context and collects user sensor data based on the spatial interaction context.

[0115] The parameter verification module calls the digital twin model to verify the feasibility of the virtual control parameter set, and performs dynamic correction through predictive control algorithms to generate quantitative control commands.

[0116] The control execution module executes quantitative control commands, evaluates the quantitative effect using a process index deviation assessment method, and optimizes the quantitative control commands based on the assessment results.

[0117] This embodiment also provides a computer device applicable to the intelligent quantitative method for automated tea beverage production, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the intelligent quantitative method for automated tea beverage production as proposed in the above embodiment.

[0118] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0119] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the intelligent quantitative method for automated tea production as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0120] In summary, this invention achieves early warning and localization of production anomalies by: inputting a tea production dataset into a cross-modal contrastive learning model to extract multimodal feature vectors, calculating the distribution consistency score of the multimodal feature vectors, and predicting real-time anomaly scores using residual regression methods, thereby improving the detection capability for complex anomalies; simultaneously, it constructs an intent recognition model to transform users' natural interactive behaviors into precise control commands that can be executed by machines, realizing the intelligent transformation from natural interaction to production control, and improving operational accuracy and production efficiency.

[0121] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. An intelligent quantitative method for automated tea beverage production, characterized in that: include, Collect tea beverage production data, preprocess it, and obtain a tea beverage production dataset with synchronized timestamps; The tea production dataset is input into a cross-modal contrastive learning model to extract multimodal feature vectors, the distribution consistency score of the multimodal feature vectors is calculated, and the real-time anomaly score is predicted by the residual regression method. When the real-time anomaly score exceeds the dynamic deviation threshold, an anomaly event signal is triggered and the anomaly area is marked in the AR tea beverage production scene. The spatial interaction context is integrated and generated, and user sensor data is collected based on the spatial interaction context. User sensor data is input into the intent recognition model. The multimodal feature encoding layer performs feature extraction and semantic encoding, the cross-modal attention layer performs dynamic weight allocation and feature alignment, and the decision output layer performs fully connected transformation to generate a set of virtual control parameters. The feasibility of the virtual control parameter set is verified by calling the digital twin model, and dynamic correction is performed through predictive control algorithm to generate quantitative control commands; The quantitative control instructions are executed, and the quantitative effect is evaluated using the process index deviation evaluation method. The quantitative control instructions are then optimized based on the evaluation results.

2. The intelligent quantitative method for automated tea beverage production as described in claim 1, characterized in that: The tea beverage production data includes temperature data, flow rate data, stirring status data, image data summary features, and user interaction data; The preprocessing includes anomaly detection and removal, sampling frequency unification and window construction, timestamp standardization, interpolation correction of missing or offset data, soft synchronization processing, and full-channel alignment synchronization. An interpolation method based on the principle of temporal proximity and a soft synchronization strategy are used to align the timestamps and correct for missing data in the preprocessed tea production data, generating a timestamp-synchronized tea production dataset.

3. The intelligent quantitative method for automated tea beverage production as described in claim 1, characterized in that: The steps are as follows: inputting the tea production dataset into a cross-modal contrastive learning model to extract multimodal feature vectors, calculating the distribution consistency score of the multimodal feature vectors, and predicting real-time anomaly scores using residual regression. Interpolation alignment and sample construction are performed on the timestamp-synchronized tea beverage production dataset to generate a timestamp-synchronized cross-modal dataset; A cross-modal contrastive learning model is used to extract feature vectors of sensor, image, and text parameters from the cross-modal dataset after timestamp synchronization, and a gating mechanism is used to fuse them to generate multimodal feature vectors. The distribution consistency score of the multimodal feature vectors is obtained by calculating the Wasserstein distance of the multimodal feature vectors, and a real-time anomaly score is generated by the residual regression method.

4. The intelligent quantitative method for automated tea beverage production as described in claim 1, characterized in that: When the real-time anomaly score exceeds the dynamic deviation threshold, an anomaly event signal is triggered, and the anomaly area is marked in the AR tea beverage production scene. A spatial interaction context is then generated, and user sensor data is collected based on the spatial interaction context. The steps are as follows: When the abnormal score exceeds the dynamic deviation threshold, an abnormal event signal is triggered, and the abnormal area is dynamically marked in the AR tea production scene according to the spatial calibration results. The abnormal event signal and the spatial calibration results are integrated, and the spatial interaction context is dynamically generated through AR annotation. Based on the spatial interaction context, users are guided to interact with AR devices in real time, and user sensor data for abnormal areas are collected simultaneously.

5. The intelligent quantitative method for automated tea beverage production as described in claim 4, characterized in that: The user sensor data includes user hand image sequences, IMU data, and eye-tracking coordinates.

6. The intelligent quantitative method for automated tea beverage production as described in claim 1, characterized in that: The intent recognition model is constructed using the following specific steps. Configure a multi-branch neural network architecture and build a multimodal feature encoding layer, a cross-modal attention layer, and a decision output layer; A cross-entropy loss function is used to perform backpropagation and gradient updates on the multimodal feature encoding layer, cross-modal attention layer, and decision output layer to construct an intent recognition model.

7. The intelligent quantitative method for automated tea beverage production as described in claim 6, characterized in that: The specific steps for generating the virtual control parameter set are as follows: Perform 3D convolution and pooling on the user's hand image sequence to generate spatiotemporal dynamic features; One-dimensional convolution and temporal state encoding are performed on IMU data to extract motion pattern features; Fully connected encoding and spatial region mapping are performed on eye-tracking coordinates to generate semantic feature vectors; Spatiotemporal dynamic features, motion pattern features, and semantic feature vectors are input into a cross-modal attention layer for linear transformation and attention fusion to generate context-aware feature vectors. The context-aware feature vectors are input into the decision output layer for feature abstraction and nonlinear transformation to generate a set of virtual control parameters.

8. The intelligent quantitative method for automated tea beverage production as described in claim 1, characterized in that: The steps for verifying the feasibility of the virtual control parameter set by calling the digital twin model and dynamically correcting it using a predictive control algorithm to generate quantitative control commands are as follows. Based on historical tea drinking data, a mapping function model is trained and obtained. The mapping function model is continuously fine-tuned, and virtual control parameters are substituted into the mapping function model to generate a set of actual process control variables. The actual process control variables are input into a high-fidelity digital twin model to predict the changing trends of key process physical variables in multiple future control cycles. The feasibility of the actual process control variable set is evaluated based on the physical coupling characteristics, and the corrected process control variable set is obtained. Predictive control algorithms are used to dynamically correct the actual set of process control variables, optimize the process performance over multiple cycles, and transform the corrected set of process control variables into quantitative control commands.

9. The intelligent quantitative method for automated tea beverage production as described in claim 1, characterized in that: The execution of quantitative control commands involves evaluating the quantitative effect using a process index deviation assessment method, and optimizing the quantitative control commands based on the evaluation results. The steps are as follows. Interpolation methods are used to align control commands and sensor data in time. A process index matrix is ​​generated by constructing and normalizing the matrix based on a standardized modeling method. Based on the time variation trend in the normalized process index matrix, the gradient of the index change rate within the time window is calculated. Based on the product of the index change rate gradient and the Hessian matrix, a time-varying weight vector is generated by the Softmax function. The time-varying weight vector is used as the driving force for multi-parameter perturbation simulation, and the pre-optimized instruction sequence is generated by the alternating direction multiplier method. Based on the pre-optimized instruction sequence, and combined with the process index deviation and the current deviation in the normalized process index matrix, the control correction amount is inferred using a fuzzy rule model. The correction is superimposed to generate new instructions, which are then verified in a digital twin to obtain a verified optimized instruction set.

10. An intelligent quantitative system for automated tea beverage production, based on the intelligent quantitative method for automated tea beverage production according to any one of claims 1 to 9, characterized in that: include, The data acquisition module collects tea beverage production data, performs preprocessing, and obtains a timestamp-synchronized tea beverage production dataset. The feature extraction module inputs the tea production dataset into the cross-modal contrastive learning model to extract multimodal feature vectors, calculates the distribution consistency score of the multimodal feature vectors, and predicts real-time anomaly scores using the residual regression method. The anomaly response module triggers an anomaly event signal and marks the abnormal area in the AR tea beverage production scene when the real-time anomaly score exceeds the dynamic deviation threshold. It also integrates and generates a spatial interaction context and collects user sensor data based on the spatial interaction context. The intent recognition module inputs user sensor data into the intent recognition model. The multimodal feature encoding layer performs feature extraction and semantic encoding, the cross-modal attention layer performs dynamic weight allocation and feature alignment, and the decision output layer performs fully connected transformation to generate a set of virtual control parameters. The parameter verification module calls the digital twin model to verify the feasibility of the virtual control parameter set, and performs dynamic correction through predictive control algorithms to generate quantitative control commands. The control execution module executes quantitative control commands, evaluates the quantitative effect using a process index deviation assessment method, and optimizes the quantitative control commands based on the assessment results.

Images