State monitoring method for multi-sensor data fusion in fresh stewed bird's nest production process

Abstract

The present application belongs to the technical field of processing quality monitoring and control of bird's nest, and relates to a state monitoring method for multi-sensor data fusion in the production process of fresh stewed bird's nest. The method solves the problems of single monitoring dimension, physical and data fragmentation, unreliable uncertainty, inaccurate quality prediction, and lagging early warning decision. The present application forms a multi-dimensional observation vector through multi-sensor deployment and space-time alignment, calculates a low-dimensional physical state in combination with the current process, extracts data-driven features, and obtains a comprehensive state representation vector by subtracting the physical residual. The vector is input into a time series evidence deep learning network to output concentration parameters, reliability quality, and overall uncertainty. The comprehensive state representation vector and the concentration parameters are input into a deep evidence regression network to output the predicted mean and predicted variance of the key quality indicators. According to the reliability quality, overall uncertainty, and predicted variance, state early warning and decision are realized through multi-level rules. The present application realizes automatic and standardized monitoring of the whole process of fresh stewed bird's nest.

Description

Technical Field

[0001] This invention belongs to the field of bird's nest processing quality monitoring and control technology, specifically involving a state monitoring method for multi-sensor data fusion during the production of fresh stewed bird's nest. Background Technology

[0002] The production of freshly stewed bird's nest involves core processes such as soaking, stewing, sterilization, cooling, and packaging. These processes are interconnected, and fluctuations in process parameters are sensitive, directly affecting key quality indicators such as sialic acid content, solid content, and taste stability. Traditional production monitoring relies heavily on manual periodic inspections and single-point sensor data collection of basic parameters such as temperature or time. This approach results in limited monitoring dimensions and isolated data, failing to simultaneously capture multimodal information such as moisture, texture, flavor, and microstructure, making it difficult to comprehensively, in real-time, and accurately depict the entire production process.

[0003] While existing deep learning monitoring solutions incorporate data-driven models, they suffer from significant drawbacks: First, they fail to deeply integrate data-driven features with physical mechanisms, resulting in models that deviate from the physicochemical laws governing the stewing process of bird's nest (such as the thermal degradation kinetics of sialic acid), leading to poor generalization and low prediction accuracy under extreme conditions. Second, the loss function design of time-series evidence networks is unreasonable, exhibiting issues such as overconfidence, abrupt changes in time-series states, and unreliable uncertainties, making it difficult to quantify the true monitoring risks. Third, quality regression prediction lacks physical constraints and statistical calibration mechanisms, resulting in large prediction biases for key indicators and low interval coverage. Fourth, the early warning rules are simple and the thresholds are fixed, failing to link uncertainty with quality predictions, easily leading to false alarms, missed alarms, and delayed decision-making.

[0004] In summary, existing technologies cannot meet the core requirements of full-dimensional perception, physical-data collaborative modeling, quantifiable uncertainty, accurate quality prediction, and multi-level reliable early warning in the production of fresh stewed bird's nest. There is an urgent need for a state monitoring method that integrates multi-sensor data fusion, physical mechanism embedding, time-series evidence-driven approach, and accurate early warning decision-making. Summary of the Invention

[0005] To address the aforementioned problems in existing technologies, namely, the single monitoring dimension, the separation of physical data and physical data, unreliable uncertainty, inaccurate quality prediction, and delayed early warning decisions, this invention proposes a multi-sensor data fusion-based state monitoring method for the production process of freshly stewed bird's nest, comprising the following steps: Multiple types of sensors are deployed on the fresh stewed bird's nest production line to collect multimodal data, which are then spatiotemporally aligned to form a multidimensional observation vector; Based on the multidimensional observation vector and the current process stage, a low-dimensional physical state vector is calculated using a pre-established process physical equation, and a temporal convolutional network is used to extract data-driven features. The data-driven features are linearly projected and the difference between them and the low-dimensional physical state vector is used to obtain physical residuals. Then, a modulation vector is generated through a learnable network. The modulation vector is scaled element-wise on the data-driven features and fused with the low-dimensional physical state vector to obtain a comprehensive state representation vector. The time series of the comprehensive state representation vector is input into a pre-trained temporal evidence deep learning network, which outputs the final Dirichlet distribution concentration parameter, and then calculates the confidence quality and overall uncertainty. The loss function of the temporal evidence deep learning network includes a squared loss term based on projection probability after removing the variance minimization term and the global KL regularization term, and a KL divergence penalty term with threshold truncation between the final Dirichlet distribution concentration parameters at adjacent time points. The comprehensive state representation vector and the final Dirichlet distribution concentration parameters are input into the pre-trained deep evidence regression network, which outputs the predicted mean and predicted variance of key quality indicators. The loss function of the pre-trained deep evidence regression network introduces a physical gradient constraint term and a differentiable quantile calibration loss term. Based on the reliability quality, the overall uncertainty, and the prediction variance, state warning information and decision instructions are output through preset multi-level rules.

[0006] The beneficial effects of this invention are: 1) Multi-source sensor fusion solves the problem of single monitoring dimension: By deploying multiple types of sensors such as temperature, moisture, image, and odor and achieving spatiotemporal alignment, a multi-dimensional observation vector covering process, materials, and environment is formed to comprehensively depict the state of the entire process of soaking, stewing, and cooling. 2) Deep integration of physics and data to solve the problem of separation between physics and data: Introduce linear projection alignment between low-dimensional physical state vectors and data-driven features, generate modulation vectors through physical residuals to dynamically correct features, and apply physical gradient constraints to force the predicted mean to conform to quasi-first-order reaction dynamics, so as to achieve collaborative modeling of physical laws and data-driven features. 3) Decoupling evidence and repair to solve the problem of uncertainty and unreliability: By decoupling the evidence head through dual channels, the sources of accidental uncertainty and cognitive uncertainty are separated. Combined with super-opinion projection, information bottleneck regularization and Re-EDL simplified loss, the overconfidence, gradient vanishing and "uncertainty illusion" of traditional EDL are fundamentally eliminated, and endogenous reliable out-of-distribution detection and confidence assessment are achieved. 4) Density-weighted calibration and physical constraint regression to solve the problem of inaccurate quality prediction: physical gradient constraints are applied to the predicted mean, and density-weighted differentiable quantile calibration loss is introduced to the predicted variance, so that the deviation between the actual coverage of the prediction interval and the nominal value is reduced. At the same time, the prediction interval in low-density areas is automatically widened, realizing the joint optimization of accuracy, physical consistency and statistical coverage. 5) Multi-level linkage decision-making to solve the problem of lagging early warning decisions: Linking reliability quality, overall uncertainty, prediction variance and density estimation, establishing multi-level rules such as out-of-distribution early warning, state attention early warning, abnormal shutdown, and prediction reliability prompts, and supplementing them with interpretability traceability such as gating factors and density estimation, to achieve closed-loop intelligent decision-making from data reliability to state discrimination and then to quality prediction. Attached Figure Description

[0007] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a flowchart of the steps of the state monitoring method for multi-sensor data fusion in the production process of fresh stewed bird's nest according to the present invention. Detailed Implementation

[0008] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0009] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0010] To more clearly explain the multi-sensor data fusion state monitoring method for the production process of freshly stewed bird's nest according to the present invention, the following is a detailed explanation. Figure 1 The steps in the embodiments of the present invention will be described in detail below.

[0011] The first embodiment of the present invention provides a status monitoring method for multi-sensor data fusion during the production of freshly stewed bird's nest. (See also...) Figure 1 This includes the following steps: S1, deploy multiple types of sensors in the fresh stewed bird's nest production line to collect multimodal data, and form a multidimensional observation vector through spatiotemporal alignment; In this embodiment, to address the problems of traditional monitoring relying on single-point parameters, limited information dimensions, asynchronous temporal and spatial data from multiple sources, and difficulty in comprehensively depicting the entire process of soaking, stewing, and cooling, a solution is adopted: multiple sensors for temperature, moisture, pressure, image, odor, and vibration are deployed in key processes such as the soaking tank, stewing pot, and cooling tunnel of the fresh-stewed bird's nest production line. These sensors simultaneously collect multimodal data on process parameters, material states, and environmental information. This data is then synchronized with a unified clock and calibrated with spatial coordinates to achieve spatiotemporal alignment, forming a standardized multidimensional observation vector. Specifically: Deploy a three-tiered sensor network along the production line: Four PT100 temperature sensors are evenly arranged around the periphery of the foaming tank, with an accuracy of ±0.1℃ and a sampling frequency of 1Hz. Three dielectric moisture sensors are arranged in the upper, middle, and lower layers inside the tank, with a sampling frequency of 0.5Hz, outputting a raw voltage signal of 0–3.3V. The volumetric moisture content is converted according to the factory-calibrated cubic polynomial curve. The calibration formula is as follows: ; In the formula, Representing the upper, middle, and lower levels. For the first Volumetric water content of layer sensor (unit: m³) 3 / m 3 ), For the first The raw voltage signal (unit: V) output by the layer sensor. a , b , c , d This is the factory calibration factor, taken as... ; One pressure sensor is installed on the tank wall; a 5-megapixel industrial camera with an LED ring diffused light source is installed at the tank opening, capturing one frame of top-view image every 30 seconds, from which CIELAB space is extracted. L , a , b The mean value, and texture features based on the gray-level co-occurrence matrix (contrast, energy, correlation, homogeneity, calculation window 32×32 pixels, step size 16 pixels, orientation 0°, 45°, 90°, 135° combined and averaged); one set of 8 array metal oxide semiconductor electronic nose modules (Figaro TGS series) is arranged inside the tank, with a sampling frequency of 1Hz, and the steady-state response value of each sensor is the resistance ratio of the adsorption equilibrium period (5-10 seconds after sampling). Rs. / R 0, constituting 8-dimensional features; The stewing pot is divided into five layers vertically, with four temperature sensors arranged in each layer to form a three-dimensional temperature monitoring array; two pressure sensors are arranged inside the pot; one vibration sensor is arranged on the pot wall; one industrial camera is arranged at the pot opening; and one set of electronic noses is arranged inside the pot. Four temperature sensors are installed at the entrance, middle, and exit sections of the cooling tunnel; an industrial camera is installed above the conveyor belt to dynamically capture changes in the surface morphology of the bird's nest. All sensors use the PLC of the stewing pot as a unified clock source and are synchronized via the SNTP protocol, with timing deviations controlled within 100ms. Spatially, the center of the stewing pot's bottom is used as the origin, and a laser rangefinder is used to calibrate the three-dimensional coordinates of each sensor point-by-point, eliminating spatial misalignment. Raw data on temperature, moisture, pressure, and vibration, along with Lab color mean and grayscale co-occurrence matrix texture features extracted from the industrial camera, and the 8-channel resistivity response of the electronic nose, are fused and stitched together to form a standardized multidimensional observation vector. ,in, D The total dimension of all sensor channel features is represented by t, which is the time index. Each frame corresponds to a time t, fully covering three types of information: process, material, and environment, and realizing unified representation of multimodal data. S2, based on the multidimensional observation vector and the current process stage, a low-dimensional physical state vector is calculated using a pre-established process physical equation, and a temporal convolutional network is used to extract data-driven features; the data-driven features are linearly projected and the difference between them and the low-dimensional physical state vector is used to obtain physical residuals, which are then used to generate modulation vectors through a learnable network; the modulation vectors are scaled element-wise on the data-driven features and fused with the low-dimensional physical state vectors to obtain a comprehensive state representation vector; In this embodiment, to address issues such as mixed evidence parameterization, overconfidence in out-of-distribution inputs, poor physical consistency, and vanishing gradients in traditional evidence deep learning (EDL), a scheme is adopted: establishing process physical equations in stages to calculate low-dimensional physical states; using temporal convolutional networks to extract temporal data features; aligning the physical space with linear projection and calculating residuals; generating modulation vectors from the residuals to dynamically correct data features; and deeply fusing features with physical vectors. Specifically: Based on the multidimensional observation vector and the current process stage, a low-dimensional physical state vector is calculated by calling a pre-established process physical equation. : During the soaking stage, from Extract the raw voltage signals from the three moisture sensors and convert them into volumetric water content according to the calibration curve described in S1. ; Calculate the average moisture content at the edge The average of the five sampling points in the first 30 seconds is taken as the saturated moisture content. initial moisture content The moisture content was determined by the 105℃ drying method; based on the analytical solution of Fick's second law for a semi-infinite flat plate, the average moisture content was calculated, specifically at time [time missing]. t The average moisture content of bird's nest as a whole , In the formula, L The characteristic thickness of bird's nest Let be the effective moisture diffusion coefficient, and take . ; The water absorption rate was obtained by numerically differentiating the soaking time (in seconds) using cubic splines. And take values ​​at discrete time t. Output a 2D physical state vector ,in, ; Take temperature data for each layer during the stewing stage. Calculate the average temperature Temperature uniformity ( ,in, (These are the highest and lowest temperatures, respectively), and the cumulative sterilization value is calculated using the trapezoidal rule. ( ,in, The continuous time during stewing The temperature (121.1 is the standard sterilization reference temperature) is used to output a 3D physical state vector. ; During the cooling phase, cross-sectional temperature data is collected to calculate the instantaneous temperature. With cooling rate (Using a 5-second sliding window to fit Newton's law of cooling, solve for the cooling constant k and calculate the cooling rate) ,in, (Ambient temperature), output a 2D physical state vector. ; The temporal convolutional network uses a three-layer dilated temporal convolutional network (TCN) to extract temporal features, with the input being the observation vectors from the past 16 frames. Network parameters: First layer dilation=1, filters=32; second layer dilation=2, filters=64; third layer dilation=4, filters=128; each layer contains causal convolution, batch normalization, and ReLU activation; the output of the last frame of the third layer is used to obtain 128-dimensional data-driven features. ; The fixed and mixed evidence parameterization method in traditional EDL is replaced by a dual-channel decoupled evidence head, which separates accidental uncertainty and cognitive uncertainty from the source of parameterization. The dual-channel decoupled evidence head includes a category prediction path and a density estimation path. The category prediction pathway takes the data-driven features as input and outputs a conditional category distribution via a multilayer perceptron; the density estimation pathway independently estimates the density scalar of the data-driven features in the training distribution; specifically, the dual-channel decoupled evidence head is divided into two independent computational pathways that execute operations synchronously, wherein the category prediction pathway takes the data-driven features (128 dimensions) as input. As input, a multilayer perceptron is used to perform feature transformation, combined with a softmax activation function, i.e., MLP (128→64→3, softmax activation), to output the conditional class distribution corresponding to each class. They do not directly participate in determining the amplitude of evidence; The density estimation pathway (density head) independently performs feature distribution discrimination. It also uses a multilayer perceptron for feature processing, combined with a sigmoid activation function, i.e., MLP (128→32→1, sigmoid activation), and outputs a density scalar with values ​​between 0 and 1. It accurately determines the confidence level that the current input feature falls within the distribution range of the model training data; Training is supervised by the true density labels, which are pre-computed offline in the feature space of the training data using Gaussian kernel density estimation (KDE, bandwidth determined by the Silverman rule). The density loss is: ; In the formula, Estimate the loss for density; The density estimate output by the density head network (parameter is...) i ); The true density label is obtained by offline calculation of kernel density estimation; The decoupled Dirichlet concentration parameter is obtained by fusing the two pathways. This parameter is then multiplied by the preset total number of equivalent samples, and then by the conditional class distribution. The resulting product is added element-wise to the all-1 vector to obtain the decoupled concentration parameter, expressed as: ; In the formula, This is the Dirichlet distribution concentration parameter vector obtained after decoupling (dimension equal to the number of categories, 3); 1 represents an all-1 vector (dimension 3). r (x) is the density scalar (dimensionless, value 0 to 1) output by the density estimation pathway. N The equivalent total number of samples (a preset constant, ranging from 100 to 500) represents the scaling factor for converting density into pseudo-counts; The conditional class distribution (3 dimensions, sum of 1) is used to predict the output of the class prediction path; when the input falls into a low-density region of the training distribution ( rWhen (x)→0), the concentration parameters of all categories automatically shrink to close to 1 (uniform Dirichlet), and the uncertainty naturally increases, realizing the endogenous capability of out-of-distribution (OOD) detection; To ensure that the physical residual directly affects the evidence amplitude, the physical residual is obtained by subtracting the data-driven features from the low-dimensional physical state vector through linear projection, including the following steps: The data-driven features are mapped to the physical state space using a learnable linear projection matrix to obtain projected physical quantities. Subtract the projected physical quantity from the low-dimensional physical state vector to obtain the physical residual; The linear projection matrix is ​​initialized as a PCA matrix during training, so that the principal component variance structure of the data-driven features in the physical state space is aligned with the low-dimensional physical state vector. Specifically, construct a learnable linear projection matrix. ,in, To minimize the physical state vector dimension, during the training phase, the linear projection matrix is ​​initialized as a PCA matrix to ensure that the principal component variance structure remains consistent with the low-dimensional physical state vector after the data-driven features are mapped to the physical space. During training, the core modules, such as the temporal convolutional network, are frozen first, and the linear projection matrix is ​​optimized separately over 60 training epochs. The optimization objective is to minimize the mean square error between the physical state vector and the projected physical quantities. After individual training is completed, the linear projection matrix is ​​unfrozen and participates in end-to-end fine-tuning of the entire network to further improve mapping accuracy. During the inference phase, data-driven features are implemented. Input a linear projection matrix, map it to the physical state space, and obtain the projected physical quantities. Then use low-dimensional physical state vectors Subtracting the projected physical quantities yields the physical residual: Physical residuals directly characterize the degree of deviation between data-driven features and the physical laws of the process, providing a basis for subsequent feature correction and evidence modulation. The decoupled concentration parameter is modulated by a density-sensing physical modulation gate to obtain a modulated concentration parameter. Specifically, the decoupled concentration parameter output by the dual-channel decoupled evidence header is further modulated by a density-sensing physical modulation gate to generate a gating factor. The method is as follows: The density scalar output by the density estimation path The density influence index is increased to the power of the preset value. ,get ; The physical residual Take the 2-norm Then divide by the preset temperature parameter The negative value is then mapped to a value between 0 and 1 using the Sigmoid function to obtain the physical consistency factor. ; Multiplying the density scalar after raising it to the power of 1 by the physical consistency factor yields the gating factor. : ; In the formula, , which is the gate signal / factor (dimensionless, value 0 to 1); c >0 represents the density influence index (default value is 1.0), which controls the nonlinear effect of density estimation on gating; s ( () is the Sigmoid function. ; It is the Euclidean norm (L2 norm) of the physical residual vector. The temperature parameter (default value 0.5) controls the steepness of the Sigmoid function. This gated signal has dual attenuation logic: in low-density regions or when the physical residual is large. Approaching 0, smoothing out evidence of suppression; The decoupling concentration parameter Subtracting the all-1 vector, multiplying by the gate factor, and then adding back the all-1 vector yields the modulated decoupling concentration parameter. : ; When the data-driven feature falls into a low-density region of the training distribution, the density scalar approaches 0, the gating factor approaches 0, and the modulated decoupling concentration parameter shrinks to an all-1 vector; when the physical residual is large, the physical consistency factor approaches 0, the gating factor approaches 0, and the modulated decoupling concentration parameter shrinks to an all-1 vector. The physical residual is input into a learnable network to generate a modulation vector; the modulation vector is then used to scale the data-driven features element-wise to obtain the modulated data features; specifically, a two-layer learnable network is constructed, with the following network structure: Input physical residual After ReLU activation and Sigmoid output, a 128-dimensional modulation vector with values ​​in the interval [0,1] is generated. , , ,in, s ( ) is the Sigmoid activation function; ReLU( )=max(0, () is the linear rectification activation function; The first layer weights and biases; For the second layer weights and biases; To obtain the modulated data features, the absolute values ​​of each component of the physical residual vector are taken; the modulation vector is then multiplied element-wise with the data-driven features to achieve dynamic feature correction. ; The modulated data features are fused with the low-dimensional physical state vector to obtain a comprehensive state representation vector. Modulated data features With low-dimensional physical state vector splicing, through a fully connected layer With ReLU activation, feature fusion is completed, outputting a 128-dimensional comprehensive state representation vector: ; In the formula, These are the weights for the fully connected layer; Bias for fully connected layers; S3, input the time series of the comprehensive state representation vector into the pre-trained temporal evidence deep learning network, output the final Dirichlet distribution concentration parameter, and then calculate the confidence quality and overall uncertainty; the loss function of the temporal evidence deep learning network includes a squared loss term based on projection probability after removing the variance minimization term and the global KL regularization term, and a KL divergence penalty term with threshold truncation between the final Dirichlet distribution concentration parameters at adjacent time points; In this embodiment, to address the problems of overconfidence, vanishing gradients, spurious correlation interference, and temporal state jumps inherent in traditional EDL, four targeted remediation schemes are adopted: simplified loss function, hyper-opinion projection, information bottleneck regularization, and temporal smoothing penalty. Specifically, a temporal evidence deep learning network is constructed, comprising a temporal network layer, an information bottleneck layer, and a hyper-opinion projection module. Systematic optimizations are performed from four dimensions: temporal modeling, information filtering, evidence expansion, and uncertainty decoupling. A detailed description follows: The temporal network layer described in S3.1 is configured to extract temporal dependencies from the time sequence of the synthesized state representation vector and output the hidden state; specifically, the input is a sequence of 32 consecutive frames of synthesized state representation vectors. The temporal network layer employs a two-layer bidirectional gated recurrent unit (GRU). The first bidirectional GRU layer sets the unidirectional hidden layer dimension to 128, simultaneously extracting features from the 32-frame integrated state representation vector sequence along both the forward and reverse temporal directions, outputting forward and backward hidden states respectively. The forward and backward hidden states are concatenated along the feature dimension to form a 256-dimensional first-layer bidirectional output feature. A dropout rate of 0.2 is set between layers to randomly suppress the activation of some neurons and prevent overfitting of the temporal features. The second bidirectional GRU layer takes the first-layer bidirectional output feature as input, also employing a bidirectional structure with a unidirectional hidden layer dimension of 128, to further extract temporal dependencies. It outputs forward and backward hidden states, which are then concatenated again to form a 256-dimensional feature, resulting in a 256-dimensional temporal hidden state. Similarly, Dropout=0.2 is configured for regularization. Thus, through the progressive temporal modeling of two layers of bidirectional GRU, the network can fully capture short-term associations and long-term dependencies, and fully learn the dynamic change patterns of the soaking, stewing and cooling stages in the production process of fresh stewed bird's nest. The information bottleneck layer described in S3.2 is configured to compress the hidden state into a low-dimensional embedding. During training, by maximizing the mutual information between the low-dimensional embedding and the true label while minimizing the upper bound of the mutual information between the hidden state and the low-dimensional embedding, the information bottleneck layer is forced to retain only information effective for class prediction. Specifically, evidence generated by existing evidence deep learning often contains spurious relevance information, and the model is prone to generating high evidence based on background features unrelated to the class label, resulting in poor prediction calibration. Therefore, an information bottleneck layer is inserted between the GRU output layer and the hyper-opinion mapping module to construct a compressed multilayer perceptron. , 256-dimensional temporal hidden state Compressed to 32-dimensional low-dimensional embedding During training, information bottleneck regularization is employed, and the temporal hidden state is minimized through variational mutual information estimation. With low-dimensional embedding Upper bound of mutual information between them, while maximizing low-dimensional embedding. The mutual information between the label and the true label y is used to force the compression layer to filter out background noise, irrelevant features, and spurious correlation patterns, retaining only the effective information that is truly predictive for the three categories of normal, attentional, and abnormal. The upper bound regularization term of the mutual information uses a combination of maximum mean difference (MMD) and InfoNCE. ; In the formula, This is due to information bottleneck regularization loss; It is the square of the maximum mean difference, used to measure the difference between two distributions; The weighting factor is 0.05. This is a lower bound for mutual information based on InfoNCE estimation; The hyper-opinion projection module described in S3.3 is configured to take the hidden state as input, map it through a multilayer perceptron to a hyper-opinion representation with a dimension greater than the number of categories of the final Dirichlet distribution concentration parameter, and then map the hyper-opinion representation back to the initial Dirichlet distribution concentration parameter through a learnable projection matrix; specifically, the input is the 32-dimensional low-dimensional embedding output from the information bottleneck layer. In other words, the hidden state is first transformed by a multilayer perceptron to map the low-dimensional hidden state into an 8-dimensional hyper-opinion representation vector (32→128→K, K=8). The dimension of this representation vector is greater than the number of categories of the Dirichlet distribution concentration parameter corresponding to the condition category determination, which can comprehensively encompass both fine and fuzzy evidence information. Then, relying on a pre-set learnable projection matrix... The high-dimensional hyperview representation vector is linearly mapped to the corresponding category dimension space, completing the dimension adaptation transformation, and finally outputting the initial Dirichlet distribution concentration parameters; specifically: ; In the formula, The initial concentration parameter vector (dimension 3) output by the super-projection module; Use the Softplus activation function to ensure a positive output; The projective matrix is ​​a learnable matrix (K=8). For the hyper-opinion mapping network (32-dimensional input, K=8-dimensional output); 1 represents an all-one vector. The hyperopinion simultaneously encodes precise single-class support and fuzzy multi-class support, providing an additional gradient path for backpropagation and alleviating the gradient vanishing problem. S3.4 The modulated concentration parameter With the output of the super-opinion projection module (That is, the initial Dirichlet distribution concentration parameters) are based on the learnable fusion coefficients. The final Dirichlet distribution concentration parameters are obtained by performing a weighted summation. Ultimately, the Dirichlet distribution corresponds to three state categories: normal, attention, and abnormal. Calculate the reliability quality of each state. (in (Corresponding to normal, attention, and abnormal, respectively) and overall uncertainty. The reliability of the quantitative model's judgment on the current production state (three state categories) is assessed by the model's reliability quality and overall uncertainty calculated based on the final Dirichlet distribution concentration parameters, combined with multi-level rules (such as reliability quality). b异常 ≥0.3, 6 consecutive frames b 正常 A value <0.3 indicates an anomaly; reliability quality b 正常 <0.6 (If the overall uncertainty u>0.4, then pay attention; otherwise, it is normal.) Automatic judgment, such as: stewing temperature 118°C, uniformity 0.96, model output. and according to The values ​​are further used to calculate the reliability quality and overall uncertainty. No warning conditions met → Normal; Stewing temperature 121°C, uniformity 0.88. ,satisfy b 正常 <0.6， Notice the warning; a slight temperature adjustment of -1°C is recommended. Local overheating to 130°C may cause a sudden pressure drop. , satisfy b 异常 ≥0.3, abnormal shutdown; The loss function of the temporal evidence deep learning network includes a squared loss term based on projected probability after removing the variance minimization term and the global KL regularization term, and a KL divergence penalty term with threshold truncation between the final Dirichlet distribution concentration parameters at adjacent time steps. The squared loss term based on projected probability (Re-EDL simplified loss), after removing the variance minimization term and the global KL regularization term, is as follows: ; In the formula, Discrete time t Loss in state discrimination; One-hot encoding for real labels ( Indicates the first k The class is a real category, otherwise ); Discrete time t The k One concentration parameter; The sum of concentration parameters, the first term is the squared error of the projected probability, and the second term is the variance adjustment term; The threshold-trimmed KL divergence penalty term between the final Dirichlet distribution concentration parameters at adjacent time points is: ; In the formula, The total time series length of the integrated state representation vector input to the temporal evidence deep learning network. For time indexing, This is a truncation function. For the first The final Dirichlet distribution concentration parameters at time t, For the first The final Dirichlet distribution concentration parameters at time t, Let KL divergence be the distance between two Dirichlet distributions. Indicated by α For the Dirichlet distribution of the concentration parameter, To allow for fluctuation thresholds, The KL divergence between the final Dirichlet distribution concentration parameters at adjacent time points under normal operating conditions is determined by the 95th quantile (taken as 0.05). It has an analytical closed form: ; In the formula, For the Gamma function, It is the Digamma function; Total loss function for: ; In the formula, This represents the average EDL loss at each time point; ; Training was performed using the AdamW optimizer with a learning rate of 0.001 and a batch size of 64. S4, input the comprehensive state representation vector and the final Dirichlet distribution concentration parameter into the pre-trained deep evidence regression network, and output the predicted mean and predicted variance of the key quality indicators; introduce physical gradient constraint term and differentiable digit calibration loss term into the loss function of the pre-trained deep evidence regression network. In this embodiment, to achieve accurate prediction of key quality indicators (sialic acid content, solid content), ensure that the prediction results conform to physical laws, and improve the reliability of prediction interval calibration, this invention constructs a multi-task deep evidence regression network. It takes the concatenated vector of the comprehensive state representation vector and the Dirichlet concentration parameter as input, and simultaneously outputs the predicted mean and variance of key quality indicators such as sialic acid and solid content. The loss function introduces a physical gradient constraint term and a density-weighted differentiable quantile calibration loss term to achieve joint optimization of accuracy, physical consistency, and statistical coverage. Specifically, the deep evidence regression network has a multi-task structure. It takes the concatenated vector (131 dimensions) of the comprehensive state representation vector (128 dimensions) and the final Dirichlet distribution concentration parameter (3 dimensions) as input. After extracting and fusing features through a three-layer fully connected network (131→128→64→32, GELU activation), it independently outputs four hyperparameters of the normal-inverse gamma distribution (NIG) for each key quality indicator, including the location parameter (of the NIG distribution). Degrees of freedom parameters Inverse scaling parameters and scale parameters The prediction mean, the prediction variance, and the total amount of evidence are calculated from the four hyperparameters in the following manner: The predicted mean (optimal estimate of the key quality indicator) is equal to the location parameter. ; The prediction variance (the range of fluctuation in the prediction result, quantifying random uncertainty) is equal to the sum of the scaling parameter multiplied by the sum of the degrees of freedom parameter plus one, divided by the product of the degrees of freedom parameter and the inverse scaling parameter minus one. ; The total amount of evidence, equal to twice the number of degrees of freedom plus the inverse scaling parameter, is used to assess the overall reliability of the predicted mean and the predicted variance. The total amount of evidence is used to assess the overall credibility of the predicted mean and predicted variance. The larger the total amount of evidence, the higher the credibility. During the stewing of fresh bird's nest, the thermal degradation of sialic acid follows a pseudo-first-order kinetic, with degradation accelerating as temperature increases. A physical gradient constraint is introduced to force the predicted mean gradient to conform to physical laws. Specifically, the physical gradient constraint term is: Calculate the predicted mean of sialic acid content Temperature during stewing (Taken from S2) The partial derivative of (converted to Kelvin temperature) is denoted as the first partial derivative. The rate of change of sialic acid with temperature as predicted by the model; The partial derivative of sialic acid concentration with respect to temperature, as determined by the pseudo-first-order reaction kinetic equation, is calculated and denoted as the second partial derivative; wherein the pseudo-first-order reaction kinetic equation includes the pre-exponential factor, activation energy, and molar gas constant; the expression for the partial derivative of sialic acid concentration with respect to temperature is: ,in, Pre-exponential factor, For activation energy, Let be the molar gas constant, T be the absolute temperature of stewing (K), SA be the sialic acid concentration (unit: mg / g), and exp( ) is an exponential function; The square of the difference between the first and second partial derivatives yields the physical gradient constraint term. By forcing the model to predict gradients to match the theoretical dynamic gradients through squared error, the prediction results are avoided from violating physical laws. The differentiable calibrator loss term is as follows: A preset set of quantiles is selected, wherein the preset set of quantiles contains at least one lower quantile, one middle quantile, and one upper quantile, such as {0.05, 0.5, 0.95}; For each quantile q, calculate the corresponding standard normal quantile. zq ( Multiply by the predicted variance The square root of this is denoted as the offset. ; Calculate the predicted mean of the output of the deep evidence regression network. Adding the offset, we obtain the predicted quantile value corresponding to that quantile. ; Substituting the predicted quantile value and the true value y corresponding to each quantile into the Pinball loss function, we obtain the Pinball loss corresponding to each quantile: ; In the formula, u To predict residuals; For indicator functions, when u If the value is less than 0, take 1; otherwise, take 0. The weighted summation of the Pinball loss corresponding to each quantile is used to obtain the differentiable quantile calibration loss term. Before weighted summation, each quantile pinball loss of the differentiable quantile calibration loss term is multiplied by the density scalar output by the density estimation pathway. The power of; where the power exponent. The temperature coefficient is greater than zero, so that the calibration loss weight in the high-density region approaches 1 and the original calibration strength is maintained, while the calibration loss weight in the low-density region is automatically reduced to suppress overcalibration in the noisy region. ; in, Quantile weights , It is a density scalar (value from 0 to 1). This is the temperature coefficient (default value is 0.5). Total regression loss for: ; In the formula, The negative log-likelihood of the NIG distribution (using the standard form proposed by Amini et al.); For evidence regularization weights; This represents the absolute value of the prediction error. The total amount of evidence; These are the weights for the physical gradient constraints. The loss is due to physical gradient constraints. To calibrate the loss weights; The calibration loss is for the differentiable digits. For density loss weights; Estimate the loss for density in S2; A two-stage training approach was adopted: the first stage froze S2 and S3, and only trained the deep evidence regression network head and density head (50 epochs, learning rate 0.001, batch size 128); the second stage unfroze all parameters and performed end-to-end fine-tuning (100 epochs, learning rate from 1e-4 cosine annealing to 1e-6). S5, based on the reliability quality, the overall uncertainty and the prediction variance, output state warning information and decision instructions through preset multi-level rules; In this embodiment, to address the issues of single early warning and delayed decision-making, a three-level rule is established based on reliability, uncertainty, and variance to output tiered early warning and closed-loop decision-making. Specifically, during online operation, each time a new observation vector o of S1 is received... t The system sequentially executes the S2→S3→S4 forward process, with a single inference time of approximately 15ms (measured by NVIDIA Jetson Orin), meeting the real-time processing requirements of a maximum sensing frame rate of 2Hz. Based on the S3 output, the reliability quality b is normal, and the overall uncertainty is... u The prediction variance Var[y] from S4 output, combined with the density estimate ρt and gating factor gt from S2 output, are used to output early warnings and decisions according to the following multi-level rules: Decoupling anomaly detection: When the density estimate ρt < 0.3, regardless of the state classification result, an "outside-distribution input" warning is triggered, indicating that the operator's current working condition deviates significantly from the training distribution and the prediction may be unreliable; Status warning: If the reliability b is normal <0.6 or the overall uncertainty u>0.4, the "attention" warning is triggered, and it is recommended to fine-tune the temperature ±1℃ and the time ±5min; if b is normal <0.3 within 3 consecutive seconds (6 consecutive sampling points), the "abnormal" shutdown signal is triggered, the production line is automatically suspended and offline sampling and retesting are requested; Prediction reliability hint: When the relative standard deviation of any key quality indicator is predicted... At that time, the system displayed "Insufficient prediction confidence, offline re-examination recommended"; Enhanced interpretability traceability: The time-series curve of the gating factor gt indicates the joint health of physical consistency and distribution density; the density estimate ρt is displayed independently to help operators distinguish between "physical violations" and "unknown operating conditions"; the multi-dimensional radar chart of the modulation vector mt is displayed in groups by sensor channel, intuitively indicating the consistency between each feature dimension and physical laws in the current data; Thus, through S1 multi-sensor deployment and spatiotemporal alignment, a multimodal observation vector covering temperature, moisture, image, and odor is constructed, solving the problems of single monitoring dimension and asynchronous temporal sequence in traditional monitoring. Through S2 decoupling evidence parameterization and density-aware physical modulation, the mixed evidence in traditional EDL is split into category prediction and density estimation paths, achieving source separation of accidental uncertainty and cognitive uncertainty. A gating factor is introduced to smoothly attenuate the evidence amplitude in low-density areas using physical residual signals, fundamentally solving the problems of overconfidence in out-of-distribution inputs and poor physical consistency. Through S3 temporal evidence deep learning network, Re-EDL simplification loss, HEDL super-opinion projection, IB-EDL information bottleneck regularization, and temporal KL smoothing penalty with threshold truncation are employed. This systemic approach overcomes the shortcomings of traditional EDL, such as overconfidence, vanishing gradients, spurious correlations, and time-series jumps, outputting smooth and reliable state reliability and overall uncertainty. Through S4 decoupling calibration-enhanced evidence regression, only the predicted mean is constrained by a quasi-first-order reaction kinetic gradient, while density-weighted differentiable quantile calibration loss is introduced to the predicted variance. This allows for coverage calibration of the prediction interval during the training phase, achieving end-to-end joint optimization of accuracy, adherence to physical laws, statistical coverage, and evidence consistency. Finally, through S5 multi-level decision rules, reliability, uncertainty, variance, and density estimation are linked to output out-of-distribution warnings, state warnings, prediction reliability hints, and interpretability traceability, effectively solving the problems of single warnings, delayed decision-making, and inability to distinguish between physical violations and unknown operating conditions. Starting from the mechanism of evidence parameterization, this invention completely breaks away from the limitations of traditional deep learning, which relies on experience-based parameter tuning and is unreliable due to uncertainty. It can stably monitor the entire production status of fresh stewed bird's nest and has advantages such as reliable evidence generation, smooth state discrimination, accurate prediction interval calibration, and lightweight deployment. It can completely replace manual inspection and single-point threshold monitoring, and realize the automation, standardization and intelligent upgrade of fresh stewed bird's nest quality monitoring.

[0012] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process and related explanations of the methods described above can be found in the corresponding processes in the foregoing system embodiments, and will not be repeated here.

[0013] Those skilled in the art will recognize that the modules and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. The programs corresponding to the software modules and method steps can be placed in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the art. To clearly illustrate the interchangeability of electronic hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in electronic hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the invention.

[0014] The terms “first”, “second”, etc., are used to distinguish similar objects, not to describe or indicate a specific order or sequence.

[0015] The term "comprising" or any other similar term is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus / device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent in such process, method, article, or apparatus / device.

[0016] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A state monitoring method using multi-sensor data fusion during the production of freshly stewed bird's nest, characterized in that, Includes the following steps: Multiple types of sensors are deployed on the fresh stewed bird's nest production line to collect multimodal data, which are then spatiotemporally aligned to form a multidimensional observation vector; Based on the multidimensional observation vector and the current process stage, a low-dimensional physical state vector is calculated using the pre-established process physical equations, and a temporal convolutional network is used to extract data-driven features. The data-driven features are linearly projected and subtracted from the low-dimensional physical state vector to obtain the physical residual, which is then used to generate a modulation vector through a learnable network. The modulation vector is element-wise scaled to the data-driven features and then fused with the low-dimensional physical state vector to obtain a comprehensive state representation vector. The time series of the comprehensive state representation vector is input into a pre-trained temporal evidence deep learning network, which outputs the final Dirichlet distribution concentration parameter, and then calculates the confidence quality and overall uncertainty. The loss function of the temporal evidence deep learning network includes a squared loss term based on projection probability after removing the variance minimization term and the global KL regularization term, and a KL divergence penalty term with threshold truncation between the final Dirichlet distribution concentration parameters at adjacent time points. The comprehensive state representation vector and the final Dirichlet distribution concentration parameters are input into the pre-trained deep evidence regression network, which outputs the predicted mean and predicted variance of key quality indicators. The loss function of the pre-trained deep evidence regression network introduces a physical gradient constraint term and a differentiable quantile calibration loss term. Based on the reliability quality, the overall uncertainty, and the prediction variance, state warning information and decision instructions are output through preset multi-level rules.

2. The multi-sensor data fusion state monitoring method for the production process of freshly stewed bird's nest according to claim 1, characterized in that, The threshold-trunculated KL divergence penalty term between the final Dirichlet distribution concentration parameters at adjacent time points is: ； In the formula, The total time series length of the integrated state representation vector input to the temporal evidence deep learning network. For time indexing, For the first The final Dirichlet distribution concentration parameters at time t, For the first The final Dirichlet distribution concentration parameters at time t, Let KL divergence be the KL divergence. The allowable fluctuation threshold; It is determined by the 95th quantile of the KL divergence between the final Dirichlet distribution concentration parameters at adjacent time points under normal operating conditions.

3. The multi-sensor data fusion state monitoring method for the production process of freshly stewed bird's nest according to claim 1, characterized in that, The physical residual is obtained by subtracting the data-driven features from the low-dimensional physical state vector through linear projection, including the following steps: The data-driven features are mapped to the physical state space using a learnable linear projection matrix to obtain projected physical quantities. Subtract the projected physical quantity from the low-dimensional physical state vector to obtain the physical residual; The linear projection matrix is ​​initialized as a PCA matrix during training, such that the principal component variance structure of the data-driven features in the physical state space is aligned with the low-dimensional physical state vector.

4. The state monitoring method for multi-sensor data fusion during the production process of freshly stewed bird's nest according to claim 1, characterized in that, The physical gradient constraint term is specifically as follows: The partial derivative of the predicted mean sialic acid content with respect to the temperature during the stewing stage is calculated and denoted as the first partial derivative. The partial derivative of sialic acid concentration with respect to temperature, as determined by the pseudo-first-order reaction kinetic equation, is calculated and denoted as the second partial derivative; wherein, the pseudo-first-order reaction kinetic equation includes the pre-exponential factor, activation energy, and molar gas constant; The physical gradient constraint term is obtained by taking the square of the difference between the first partial derivative and the second partial derivative; the physical gradient constraint term forces the first partial derivative to conform to the pseudo-first-order reaction kinetic equation.

5. The state monitoring method using multi-sensor data fusion during the production of freshly stewed bird's nest according to claim 1, characterized in that, The differentiable calibrator loss term is as follows: A preset set of quantiles is selected, wherein the preset set of quantiles contains at least one lower quantile, one middle quantile, and one upper quantile; For each quantile, calculate the standard normal quantile corresponding to that quantile multiplied by the square root of the prediction variance, and record it as the offset; The predicted mean output by the deep evidence regression network is added to the offset to obtain the predicted quantile value corresponding to that quantile. Substitute the predicted and actual values ​​of each quantile into the Pinball loss function to obtain the Pinball loss for each quantile. The weighted summation of the Pinball losses corresponding to each quantile yields the differentiable quantile calibration loss term.

6. The state monitoring method using multi-sensor data fusion during the production of freshly stewed bird's nest according to claim 1, characterized in that, The deep evidence regression network has a multi-task structure. It takes the concatenated vector of the integrated state representation vector and the final Dirichlet distribution concentration parameter as input. After extracting fused features through a three-layer fully connected network, it independently outputs four hyperparameters of a normal-inverse gamma distribution for each key quality indicator, including location parameter, degrees of freedom parameter, inverse scaling parameter, and scaling parameter. The predicted mean, predicted variance, and total evidence quantity are calculated from these four hyperparameters in the following manner: The predicted mean is equal to the location parameter; The prediction variance is equal to the sum of the scaling parameter multiplied by the degree of freedom parameter plus one, divided by the product of the degree of freedom parameter and the inverse scaling parameter minus one; The total amount of evidence is equal to twice the number of degrees of freedom plus the inverse scaling parameter, and is used to assess the overall confidence of the predicted mean and the predicted variance.

7. The state monitoring method using multi-sensor data fusion during the production of freshly stewed bird's nest according to claim 1, characterized in that, The temporal evidence deep learning network includes a temporal network layer, an information bottleneck layer, and a hyper-opinion projection module. The temporal network layer is configured to extract temporal dependencies from the time series of the comprehensive state representation vector and output the hidden state. The information bottleneck layer is configured to compress the hidden state into a low-dimensional embedding. During training, by maximizing the mutual information between the low-dimensional embedding and the true label, while minimizing the upper bound of the mutual information between the hidden state and the low-dimensional embedding, the information bottleneck layer is forced to retain only the information that is effective for category prediction. The hyper-opinion projection module is configured to take the hidden state as input, map it through a multilayer perceptron to a hyper-opinion representation with a dimension greater than the number of categories of the final Dirichlet distribution concentration parameter, and then map the hyper-opinion representation back to the initial Dirichlet distribution concentration parameter through a learnable projection matrix.

8. The state monitoring method for multi-sensor data fusion during the production process of freshly stewed bird's nest according to claim 7, characterized in that, It also includes a dual-channel decoupled evidence header; the dual-channel decoupled evidence header includes a category prediction path and a density estimation path; the category prediction path takes the data-driven features as input and outputs a conditional category distribution through a multilayer perceptron; the density estimation path independently estimates the density scalar of the data-driven features in the training distribution. The initial Dirichlet distribution concentration parameter output by the super-projection module is weighted and summed with the modulated concentration parameter obtained by fusing and modulating the conditional category distribution and the density scalar to obtain the final Dirichlet distribution concentration parameter, specifically: The density scalar is multiplied by the preset total number of equivalent samples, and then multiplied by the conditional class distribution. The resulting product is added element-wise to the all-1 vector to obtain the decoupled concentration parameter. The decoupled concentration parameter is then passed through a density-aware physical modulation gate to obtain the modulated concentration parameter. The modulated concentration parameters and the output of the super-projection module are weighted and summed using learnable fusion coefficients to obtain the final Dirichlet distribution concentration parameters.

9. The state monitoring method using multi-sensor data fusion during the production of freshly stewed bird's nest according to claim 8, characterized in that, Before weighted summation, each quantile pinball loss of the differentiable quantile calibration loss term is multiplied by the power of the density scalar output by the density estimation path; where the power exponent is a temperature coefficient greater than zero, so that the calibration loss weight in the high-density region approaches 1 and maintains the original calibration intensity, while the calibration loss weight in the low-density region is automatically decayed to suppress over-calibration in the noisy region.

10. The state monitoring method using multi-sensor data fusion during the production process of freshly stewed bird's nest according to claim 9, characterized in that, The decoupled concentration parameter is modulated by a density-sensing physical modulation gate to obtain the modulated concentration parameter. The method is as follows: The density scalar output by the density estimation pathway is raised to a preset density influence index. The physical residual is taken as the L2 norm and divided by the preset temperature parameter. The negative value is then mapped to a range of 0 to 1 by the Sigmoid function to obtain the physical consistency factor. The density scalar after raising it to the power of 1 is multiplied by the physical consistency factor to obtain the gating factor; Subtract the all-1 vector from the decoupling concentration parameter, multiply by the gating factor, and then add back the all-1 vector to obtain the modulated decoupling concentration parameter. When the data-driven feature falls into a low-density region of the training distribution, the density scalar approaches 0, the gating factor approaches 0, and the modulated decoupling concentration parameter shrinks to an all-1 vector; when the physical residual is large, the physical consistency factor approaches 0, the gating factor approaches 0, and the modulated decoupling concentration parameter shrinks to an all-1 vector.

Images