Intelligent monitoring method for dough fermentation state in bread processing

Abstract

The application discloses an intelligent monitoring method for dough fermentation state in a bread processing process, and relates to the technical field of dough monitoring. The method comprises the following steps: performing time sequence feature extraction on a multi-source data set, obtaining a time sequence feature matrix, dividing an internal space of a fermentation box into N monitoring grids, independently analyzing spectrum data and sound wave signal data, temperature and humidity in each monitoring grid, extracting a spatial distribution feature matrix, inputting the time sequence feature matrix and the spatial distribution feature matrix into a dynamic self-adaptive feature fusion unit, generating a dough fermentation state feature matrix by coupling an oscillation neural network and an attention mechanism, inputting the dough fermentation state feature matrix into an ST-GNN model, obtaining dough fermentation state data, converting the dough fermentation state data into a three-dimensional distribution graph, and presenting the three-dimensional distribution graph in real time on a display terminal. The method enhances the transparency monitoring of a dough fermentation process by using a dough monitoring method.

Description

Technical Field

[0001] This invention relates to the field of dough monitoring technology, and in particular to an intelligent monitoring method for the fermentation state of dough during bread processing. Background Technology

[0002] With the continuous advancement of the food industry and consumers' increasing demands for bread quality, effective monitoring of dough fermentation has become crucial. In recent years, advancements in sensor technology, spectral analysis, and data processing have made intelligent monitoring of dough fermentation during bread processing possible. The application of these advanced technologies not only enhances the controllability and transparency of the fermentation process but also lays the foundation for automated and standardized production. Furthermore, modern spectral analysis techniques, such as terahertz spectral scanning, enable non-contact monitoring of changes in the internal structure of dough, representing a significant step towards more intelligent, efficient, and precise bread production processes.

[0003] Nevertheless, existing dough monitoring methods still have room for improvement. Currently, most fermentation monitoring parameters on the market focus on only a few (such as temperature, humidity, CO2 concentration, etc.), while ignoring changes in the physicochemical properties of the dough itself, such as elastic modulus, internal moisture migration, and sugar conversion. This one-sided data collection method makes it difficult to provide a comprehensive understanding of the dough's fermentation state, making it difficult to adjust fermentation conditions in a timely manner, which in turn affects the quality of the final product. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides an intelligent monitoring method for the fermentation state of dough during bread processing to solve the problem of the single method of dough fermentation monitoring.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, the present invention provides an intelligent monitoring method for the fermentation state of dough during bread processing, comprising, Strain data, spectral data, acoustic signal data, temperature, humidity, and carbon dioxide concentration data of dough in the fermentation chamber were collected and time-stamped to form a multi-source dataset. Temporal feature extraction is performed on multi-source datasets to obtain a temporal feature matrix; The internal space of the fermentation chamber is divided into N monitoring grids. The spectral data, acoustic signal data, temperature and humidity in each monitoring grid are analyzed independently to extract the spatial distribution feature matrix. The temporal feature matrix and spatial distribution feature matrix are input into the dynamic adaptive feature fusion unit, and the dough fermentation state feature matrix is ​​generated by coupling an oscillating neural network and an attention mechanism. Input the dough fermentation state feature matrix into the ST-GNN model to obtain dough fermentation state data; The dough fermentation status data is converted into a three-dimensional distribution map and displayed in real time on the display terminal.

[0007] As a preferred embodiment of the intelligent monitoring method for the fermentation state of dough during bread processing according to the present invention, the method for collecting strain data, spectral data, acoustic signal data, temperature, humidity, and carbon dioxide concentration data of the dough in the fermentation chamber specifically includes: By setting a strain sensor array on the inner wall of the fermentation chamber, the surface deformation of the dough is monitored in real time to obtain strain data; A terahertz spectral scanner was used to perform non-contact scanning of the dough in the fermentation chamber to obtain spectral data on moisture migration and sugar conversion inside the dough. Acoustic sensors capture sound wave signals generated by the movement of air bubbles inside the dough. A gas sensor was used to monitor the rate of change of carbon dioxide concentration in the fermentation chamber, obtain carbon dioxide concentration data, and simultaneously record temperature and humidity data in the fermentation chamber.

[0008] As a preferred embodiment of the intelligent monitoring method for the fermentation state of dough during bread processing described in this invention, the step of performing timestamp alignment to form a multi-source dataset refers to forming a multi-source dataset by aligning strain data, spectral data, acoustic signal data, temperature, humidity, and carbon dioxide concentration data through a time protocol.

[0009] As a preferred embodiment of the intelligent monitoring method for dough fermentation state during bread processing according to the present invention, the step of extracting temporal features from multi-source datasets to obtain a temporal feature matrix specifically involves: The elastic modulus of the dough was calculated using a moving average algorithm based on the strain data. Principal component analysis is used to extract principal component coefficients from spectral data to form spectral feature vectors. After performing a Fourier transform on the acoustic signal data to obtain a time-frequency spectrum matrix, the Mel frequency cepstral coefficients are calculated. Temperature and humidity data are respectively processed using Z-score normalization to obtain standardized temperature and humidity; Perform a first-order difference operation on the carbon dioxide concentration data to generate the gas production rate; The dough elastic modulus, spectral feature vector, Mel frequency cepstral coefficient, gas production rate, standardized temperature and humidity are aligned and spliced ​​along the time axis to form a time-series feature matrix.

[0010] As a preferred embodiment of the intelligent monitoring method for the fermentation state of dough during bread processing according to the present invention, the method of dividing the internal space of the fermentation box into N monitoring grids refers to collecting the internal geometric dimensions of the fermentation box to establish a three-dimensional coordinate system, and dividing it into N monitoring grids at equal intervals along the length, width, and height directions.

[0011] As a preferred embodiment of the intelligent monitoring method for dough fermentation status during bread processing according to the present invention, the step of independently analyzing the spectral data, acoustic signal data, temperature, and humidity within each monitoring grid to extract a spatial distribution feature matrix specifically involves: Kriging space interpolation is performed on the spectral data within each monitoring grid to generate a three-dimensional distribution field of spectral absorption peak intensity. After the acoustic signal data is processed by beamforming, the acoustic energy flux density vector at the center point of each monitoring grid is calculated. Finite element analysis was performed on the temperature and humidity within each monitoring grid to obtain the temperature gradient and humidity gradient values. The spatial distribution feature matrix within the fermenter is constructed by the three-dimensional distribution field of the spectral absorption peak intensity, the acoustic energy flux density vector, the temperature gradient value, and the humidity gradient value within each monitoring grid.

[0012] As a preferred embodiment of the intelligent monitoring method for dough fermentation state during bread processing according to the present invention, wherein: the temporal feature matrix and spatial distribution feature matrix are input into a dynamic adaptive feature fusion unit, and a dough fermentation state feature matrix is ​​generated by coupling an oscillating neural network and an attention mechanism, specifically, The time-series feature matrix is ​​input into the time-varying wavelet packet transform processor, and multi-scale decomposition is performed using the Daubechies wavelet basis function to extract time-frequency features of different frequency bands and generate a joint time-frequency feature matrix. The spatial distribution feature matrix is ​​input into a nonlocal mean filter to smooth the three-dimensional distribution field of spectral absorption peak intensity, acoustic energy flux density vector, temperature gradient value, and humidity gradient value, thereby generating an enhanced spatial distribution feature matrix. Using the joint time-frequency feature matrix as the input to the driving oscillator and the enhanced spatial distribution feature matrix as the input to the response oscillator, frequency locking and phase synchronization between oscillators are achieved through phase difference feedback adjustment, thereby obtaining synchronization feature pairs. The synchronous feature pairs are converted into query vectors. The temporal feature matrix and the spatial distribution feature matrix constitute the key matrix and the value matrix. By calculating the relevance score between the key matrix and the query vector and normalizing it, a dynamic attention weight matrix is ​​generated. The dynamic attention weight matrix is ​​used to perform Hadamard product operation on the synchronous feature pairs to obtain the spatiotemporal fusion feature tensor. The spatiotemporal fusion feature tensor is input into a two-layer fully connected network to obtain the dough fermentation state feature matrix.

[0013] As a preferred embodiment of the intelligent monitoring method for dough fermentation state during bread processing described in this invention, the step of inputting the spatiotemporal fusion feature tensor into a two-layer fully connected network to obtain the dough fermentation state feature matrix refers to inputting the spatiotemporal fusion feature tensor into the first layer network and performing a nonlinear transformation using the ReLU activation function, and then inputting it into the second layer network and performing a dimension mapping using a linear activation function to generate the dough fermentation state feature matrix.

[0014] As a preferred embodiment of the intelligent monitoring method for dough fermentation state during bread processing according to the present invention, wherein: the step of inputting the dough fermentation state feature matrix into the ST-GNN model to obtain dough fermentation state data specifically involves, Based on the feature matrix of dough fermentation state, a bidirectional long short-term memory network layer is constructed, and a two-layer graph convolutional network is configured. The fully connected output layer is set with 5 output neurons corresponding to the yeast activity, dough volume, CO2 concentration and dough temperature and humidity, respectively. The activation functions are linear activation function, ReLU activation function and Softmax, forming the ST-GNN model. The dough fermentation state feature matrix is ​​input into the ST-GNN model for feature fusion mapping to generate dough fermentation state data.

[0015] As a preferred embodiment of the intelligent monitoring method for dough fermentation status during bread processing according to the present invention, the step of converting dough fermentation status data into a three-dimensional distribution map and presenting it in real time on a display terminal specifically involves: The dough fermentation state data is converted into point cloud data in three-dimensional space using a 3D distribution map generation algorithm; The point cloud data is rendered in real time using a graphics rendering engine to generate a 3D distribution map, which is then presented through a visualization interface on a display terminal.

[0016] The beneficial effects of this invention are as follows: By acquiring data from multiple sources, comprehensive monitoring of physical, chemical, and environmental factors during dough fermentation is achieved, providing a data foundation for optimizing fermentation conditions. At the same time, temporal feature extraction and spatial distribution feature analysis utilize advanced data analysis methods to simplify complex data structures and accurately capture spatiotemporal changes during fermentation, improving the relevance and interpretability of the data. Subsequently, a dough fermentation state feature matrix is ​​generated through feature weighted fusion, comprehensively considering the influence of time and spatial variables, which significantly improves the accuracy and reliability of the prediction model. Finally, the fermentation state data is converted into a three-dimensional distribution map for real-time presentation, enhancing the transparency of the dough fermentation process monitoring. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart for an intelligent monitoring method for the fermentation state of dough during bread processing.

[0019] Figure 2 A flowchart for generating dough fermentation status data.

[0020] Figure 3 A flowchart for generating the spatial distribution feature matrix.

[0021] Figure 4 A flowchart illustrating the architecture of an intelligent monitoring method for the fermentation state of dough during bread processing. Detailed Implementation

[0022] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0023] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0024] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0025] Reference Figures 1-4 This is one embodiment of the present invention, which provides an intelligent monitoring method for the fermentation state of dough during bread processing, comprising the following steps: S1. Collect strain data, spectral data, acoustic signal data, temperature, humidity and carbon dioxide concentration data of dough in the fermentation chamber, and align the timestamps to form a multi-source dataset; By setting a strain sensor array on the inner wall of the fermentation chamber, the surface deformation of the dough is monitored in real time to obtain strain data; It should be noted that a strain sensor array is evenly arranged on the inner wall of the fermentation chamber. Each strain sensor is a microelectromechanical force-sensitive element that continuously collects the normal pressure applied during the dough expansion process at a fixed sampling frequency. The strain sensor array converts the normal pressure into a normal pressure signal, which is then converted into a voltage signal by a Wheatstone bridge. After being quantized by an analog-to-digital converter, the real-time strain value inside the fermentation chamber, i.e., strain data, is obtained.

[0026] A terahertz spectral scanner was used to perform non-contact scanning of the dough in the fermentation chamber to obtain spectral data on moisture migration and sugar conversion inside the dough. It should be noted that a terahertz spectroscopy scanner emits terahertz waves into the fermentation chamber at a fixed scanning frequency. The terahertz waves penetrate the observation window of the fermentation chamber and illuminate the inside of the dough. After the terahertz waves interact with the water molecules and sugars inside the dough, the terahertz waves carrying the material characteristics return to a high-sensitivity detector. The high-sensitivity detector transmits the terahertz waves carrying the material characteristics to the data processing unit. The data processing unit converts the terahertz waves into spectral data through Fourier transform. The spectral data includes spectral scan samples, absorbance values, and spectral absorption peak intensities.

[0027] Acoustic sensors capture sound wave signals generated by the movement of air bubbles inside the dough. It should be noted that a broadband acoustic sensor array is arranged on the inner wall of the fermentation chamber. The broadband acoustic sensor continuously collects the sound wave signals generated by bubble bursting and gas diffusion during the dough fermentation process at a fixed sampling frequency. After using a filter to retain the effective frequency band in the sound wave signal, it is converted into a digital signal by an analog-to-digital converter. The digital signal is then input into a wavelet transform processor for multi-scale decomposition to obtain the sound wave signal data, including the sound pressure signal and the particle velocity vector.

[0028] A gas sensor was used to monitor the rate of change of carbon dioxide gas concentration in the fermentation chamber, obtain carbon dioxide concentration data, and simultaneously record temperature and humidity data in the fermentation chamber. It should be noted that distributed digital temperature and humidity sensors are used to synchronously collect temperature and humidity data within the fermentation chamber. A non-dispersive infrared carbon dioxide gas sensor is installed on the top of the fermentation chamber. This sensor measures the carbon dioxide concentration within the chamber at fixed sampling intervals and outputs a current signal. The current signal is converted into a voltage signal by an instrumentation amplifier and then input to a temperature compensation circuit. This circuit corrects the current signal based on the temperature and humidity data within the fermentation chamber. The corrected current signal is then quantized into a digital value for the carbon dioxide concentration using an analog-to-digital converter, thus obtaining the carbon dioxide concentration data. The collected strain data, spectral data, acoustic signal data, temperature data, humidity data, and carbon dioxide concentration data are timestamped using the IEEE 1588 precise time protocol and then aligned according to the timestamps to form a multi-source dataset.

[0029] S2. Extract time-series features from the multi-source dataset to obtain the time-series feature matrix; The elastic modulus of the dough was calculated using a moving average algorithm based on the strain data. It should be noted that the strain data was first processed by a moving average filter with a sliding window length of 30 sampling points to eliminate high-frequency noise interference; the filtered strain data was then combined with the mechanical loading conditions in the fermentation chamber, and the elastic modulus of the dough was calculated based on Hooke's Law, expressed by the formula: ; in, Indicates the elastic modulus of dough. Indicates normal pressure, Represents strain data (strain values); Principal component analysis is used to extract principal component coefficients from spectral data to form spectral feature vectors. It should be noted that the spectral data is first preprocessed using a standard normal transformation to eliminate baseline drift. The preprocessed spectral data is then organized into a spectral data matrix along both row and column directions. The spectral scan samples (i.e., different characters on the dough) are used as the row directions, and the absorbance values ​​at wavelength points are used as the column directions to establish the spectral data matrix. Spectral data matrix Interchange rows and columns to form the transpose matrix. Based on spectral data matrix transpose matrix Using principal component analysis, the covariance matrix among the spectral wavelength variables is calculated, expressed by the formula: ; in, Represents the covariance matrix. Represents a spectral data matrix. Represents the transpose matrix. Indicates the number of spectral scan samples; The covariance matrix is ​​decomposed into eigenvalues ​​and eigenvectors. The first three principal component directions are selected by sorting the eigenvalues ​​from largest to smallest. The spectral data is projected onto the three principal component directions to obtain the three principal component coefficients. The combination of the three principal component coefficients constitutes the spectral eigenvector. The first principal component coefficient reflects the intensity of water migration, the second principal component coefficient characterizes the degree of carbohydrate conversion, and the third principal component coefficient indicates the change in gluten network.

[0030] After performing a Fourier transform on the acoustic signal data to obtain a time-frequency spectrum matrix, the Mel frequency cepstral coefficients are calculated. It should be noted that the acoustic signal data is converted into a discrete-time series using an analog-to-digital converter. The discrete-time series is then divided into fixed-length frames, and a fast Fourier transform is performed on each frame to obtain a short-time spectrum. The power spectral density is obtained by squaring the short-time spectrum. The logarithm of the power spectral density is then arranged in chronological order to form a time-spectrum matrix with rows representing time frames and columns representing frequency points. The time-spectrum matrix is ​​multiplied by each Mel filter to obtain the Mel filter energy output. The logarithm of the Mel filter bank's energy output is then subjected to a discrete cosine transform to obtain the Mel frequency cepstral coefficients, expressed by the formula: ; in, Represents the Mel frequency cepstral coefficients. The order index of Mel frequency cepstral coefficients. This indicates the total number of channels in the Mel filter. This indicates the channel index of the Mel filter. This indicates the energy output of the Mel filter bank. Pi is a constant. Temperature and humidity data are respectively processed using Z-score normalization to obtain standardized temperature and humidity; It should be noted that after calculating the arithmetic mean and standard deviation of the time series of temperature and humidity data respectively, Z-score standardization was performed to obtain standardized temperature and humidity, expressed by the formula: ; ; in, Indicates standardized temperature. Indicates standardized humidity. Representing temperature data, This represents the arithmetic mean of the temperature data. This represents the standard deviation of temperature data. Represents humidity data. This represents the arithmetic mean of the humidity data. Indicates the standard deviation of humidity data; Perform a first-order difference operation on the carbon dioxide concentration data to generate the gas production rate; It should be noted that a first-order difference operation is performed on the carbon dioxide concentration data to calculate the difference between carbon dioxide concentration data at adjacent time points, and the gas production rate is expressed by the formula: ; in, Indicates the gas production rate. This represents the numerical value of carbon dioxide gas concentration. The numerical value of carbon dioxide gas concentration at the current time. Indicates the sampling time interval. This indicates the time sequence number at which the digital value of carbon dioxide concentration was collected. This represents the numerical value of the carbon dioxide concentration at the previous moment. Each data point in the gas production rate represents the change in carbon dioxide concentration per unit time; a positive value indicates an increase in carbon dioxide concentration, and a negative value indicates a decrease in carbon dioxide concentration. The dough elastic modulus, spectral feature vector, Mel frequency cepstral coefficient, gas production rate, standardized temperature and humidity are aligned and spliced ​​along the time axis to form a time-series feature matrix. Furthermore, the dough elastic modulus data, spectral feature vector, Mel frequency cepstral coefficient, gas production rate data, standardized temperature data, and standardized humidity data are first aligned using a unified timestamp, and then concatenated into a two-dimensional matrix according to the time series. The rows of the two-dimensional matrix represent the time series, and the columns are arranged sequentially as dough elastic modulus, spectral feature vector, Mel frequency cepstral coefficient, gas production rate, standardized temperature, and humidity, ultimately yielding a time series feature matrix.

[0031] S3. Divide the internal space of the fermentation chamber into N monitoring grids, and independently analyze the spectral data, acoustic signal data, temperature and humidity in each monitoring grid to extract the spatial distribution feature matrix. A three-dimensional coordinate system was established by collecting the internal geometric dimensions of the fermentation chamber, and the system was divided into N monitoring grids at equal intervals along the length, width, and height directions. It should be noted that a right-handed rectangular coordinate system (X, Y, Z) is established inside the fermentation chamber. The origin of the right-handed rectangular coordinate system is located at the lower left corner of the inner wall of the fermentation chamber, with the X-axis parallel to the length direction of the fermentation chamber, the Y-axis parallel to the width direction of the fermentation chamber, and the Z-axis parallel to the height direction of the fermentation chamber. The length of the fermentation chamber is divided into N segments at equal intervals along the X-axis, the width of the fermentation chamber is divided into N segments at equal intervals along the Y-axis, and the height of the fermentation chamber is divided into N segments at equal intervals along the Z-axis, ultimately forming N monitoring grids. Each monitoring grid is assigned a unique code in sequence. The equal intervals in the length, width, and height directions can be expressed by the following formula: ; in, Indicates equal spacing. This indicates the internal geometric dimensions (length, width, height) of the fermentation chamber. This indicates the number of monitoring grids used to divide the target area; Kriging space interpolation is performed on the spectral data within each monitoring grid to generate a three-dimensional distribution field of spectral absorption peak intensity. It should be noted that a spatial location dataset is constructed based on a right-handed Cartesian coordinate system of the monitoring grid. A semi-variogram is established using this dataset and the Kriging interpolation algorithm. A spherical mathematical model is applied to the semi-variogram, and then fitted using weighted least squares. The optimal weighting coefficients for the unsampled monitoring grid are calculated using the fitted semi-variogram. Based on these optimal weighting coefficients, the spectral data (spectral absorption peak intensities) of the known sampling points are weighted and summed to generate a three-dimensional distribution field of spectral absorption peak intensities in the fermentation chamber space. Here, the unsampled monitoring grid refers to the monitoring grid that was not directly measured by the terahertz spectrometer; the known sampling points refer to the monitoring grid that was directly measured and whose spectral data was recorded by the terahertz spectrometer; the spherical mathematical model serves as a theoretical model for the semi-variogram in Kriging spatial interpolation, used to describe and quantify the spatial correlation structure of spectral absorption peak intensities, thus providing a statistical basis for interpolation calculations at unknown locations, and is not an algorithmic model.

[0032] After the acoustic signal data is processed by beamforming, the acoustic energy flux density vector at the center point of each monitoring grid is calculated. It should be noted that, within the monitoring grid inside the fermentation chamber, the acoustic signal data collected by the broadband acoustic sensor array is first processed by beamforming to generate directional acoustic signal data. The sound pressure signal from this directional acoustic signal data is then multiplied in the time domain with the particle velocity vector to obtain the acoustic energy flux density vector, which is related to the density of air bubbles in the dough. The acoustic energy flux density vector is expressed by the formula: ; in, Represents the acoustic energy flux density vector. Indicates sound pressure signal, Represents the particle velocity vector; Finite element analysis was performed on the temperature and humidity within each monitoring grid to obtain the temperature gradient and humidity gradient values. It should be noted that within the monitoring grid, temperature and humidity are discretized using the finite element method, dividing each monitoring grid into finite elements. Interpolation functions for temperature and humidity are constructed within each element. Then, control equations for temperature and humidity are established based on the heat conduction equation and the moisture diffusion equation. The weighted residual method is used to transform the control equations into a system of algebraic equations. The temperature and humidity distribution within the monitoring grid is obtained by solving these algebraic equations. Finally, the temperature gradient and humidity gradient values ​​are calculated using the numerical difference method, expressed by the following formula: ; in, Indicates the temperature gradient value. Indicates the humidity gradient value. Indicates monitoring grid Axial step size, Indicates monitoring grid Axial step size, β This indicates the temperature within the monitored grid. This indicates the humidity within the monitored grid. This indicates the position index of the monitoring grid in the horizontal direction (X-axis direction). Indicates the position index of the monitoring grid in the horizontal direction (Y-axis direction); The interpolation functions for constructing temperature and humidity are used to approximate the temperature and humidity distribution of each cell within the monitoring grid, thereby accurately calculating the temperature gradient and humidity gradient values.

[0033] The spatial distribution feature matrix within the fermenter is constructed by the three-dimensional distribution field of the spectral absorption peak intensity, the acoustic energy flux density vector, the temperature gradient value, and the humidity gradient value within each monitoring grid. Furthermore, the three-dimensional distribution field of spectral absorption peak intensity, acoustic flux density vector, temperature gradient value, and humidity gradient value within each monitoring grid are aligned using a unified spatial coordinate system (right-hand rectangular coordinate system) and then organized into a multi-dimensional matrix according to the monitoring grid number order. The row direction of the multi-dimensional matrix corresponds to the spatial index of the monitoring grid, and the column direction is arranged with the three-dimensional distribution field of spectral absorption peak intensity, acoustic flux density vector, temperature gradient value, and humidity gradient value in sequence, ultimately forming a spatial distribution feature matrix.

[0034] S4. Input the temporal feature matrix and spatial distribution feature matrix into the dynamic adaptive feature fusion unit, and generate the dough fermentation state feature matrix by coupling the oscillating neural network and the attention mechanism. The time-series feature matrix is ​​input into the time-varying wavelet packet transform processor, and multi-scale decomposition is performed using the Daubechies wavelet basis function to extract time-frequency features of different frequency bands and generate a joint time-frequency feature matrix. It should be noted that, firstly, a 5-level decomposition depth is configured in the wavelet packet transform processor to establish the analysis frequency band, and the wavelet filter bank coefficients are initialized; then, a hierarchical decomposition is performed. The first level of the wavelet filter separates the time-series feature matrix into low-frequency approximation coefficients and high-frequency detail coefficients. The second to fifth levels synchronously process the time-varying characteristics of the dough's elastic modulus and stiffness, the frequency domain energy distribution of the spectral feature vector, the resonant peak migration of the Mel frequency cepstral coefficients, the gas production rate, and the normalized temperature and humidity periodic fluctuations. Subsequently, the wavelet filter extracts the gluten network relaxation characteristics, yeast metabolic activity characteristics, bubble dynamics characteristics, and equipment vibration noise through instantaneous energy analysis, zero-crossing rate detection, and inter-band cross-correlation calculation, and constructs a three-dimensional time-frequency feature tensor with time alignment characteristics. The three-dimensional time-frequency feature tensor is then reduced in dimension by principal component analysis to generate a time-frequency joint feature matrix.

[0035] The spatial distribution feature matrix is ​​input into a nonlocal mean filter to smooth the three-dimensional distribution field of spectral absorption peak intensity, acoustic energy flux density vector, temperature gradient value, and humidity gradient value, thereby generating an enhanced spatial distribution feature matrix. It should be noted that a weighted similarity matrix is ​​constructed based on the distance and spatial distribution characteristics similarity between monitoring grids (including the curve matching degree of the three-dimensional distribution field of spectral absorption peak intensity, the directional consistency of the acoustic energy flow density vector, and the spatial correlation of temperature gradient values ​​and humidity gradient values). Subsequently, a 7×7×7 cube search window is used, combined with a three-dimensional Gaussian weighting function to perform filtering processing on the similarity matrix. Finally, while retaining the position of spectral characteristic peaks and the main direction of acoustic energy flow, local outliers of temperature and humidity gradient values ​​are eliminated, generating an enhanced spatial distribution characteristic matrix.

[0036] Using the joint time-frequency feature matrix as the input to the driving oscillator and the enhanced spatial distribution feature matrix as the input to the response oscillator, frequency locking and phase synchronization between oscillators are achieved through phase difference feedback adjustment, thereby obtaining synchronization feature pairs. It should be noted that, firstly, the fundamental frequency and phase response coefficient of the driving oscillator, as well as the fundamental frequency and phase response coefficient of the response oscillator, are configured. The driving oscillator is used to extract the characteristic instantaneous phase from the joint time-frequency feature matrix, and the response oscillator is used to extract the spatial frequency phase from the enhanced spatial distribution feature matrix. Synchronization feature pairs are obtained by real-time monitoring of the phase difference between the characteristic instantaneous phase and the spatial frequency phase.

[0037] The synchronous feature pairs are converted into query vectors. The temporal feature matrix and the spatial distribution feature matrix constitute the key matrix and the value matrix. By calculating the relevance score between the key matrix and the query vector and normalizing it, a dynamic attention weight matrix is ​​generated. It should be noted that the synchronous feature pairs are transformed into query vectors, and the temporal feature matrix and spatial distribution feature matrix are linearly transformed into key and value matrices (both with dimensions 512×512), respectively. Then, the dot product relevance score between the query vector and the key matrix is ​​calculated using the following formula: ; in, This represents the dot product correlation score. Represents the query vector. Represents the key matrix. This represents the matrix transpose operation. This represents the feature dimension scaling factor.

[0038] The dot product correlation score is normalized using the Softmax function to obtain the normalized attention weight distribution; then the normalized attention weights are multiplied by the value matrix to obtain the dynamic attention weight matrix.

[0039] The dynamic attention weight matrix is ​​multiplied with the synchronous feature pairs by Hadamard to obtain the spatiotemporal fusion feature tensor. It should be noted that the dynamic attention weight matrix (dimension 512×32×6) and the synchronization feature pair (dimension 512×32×6) are multiplied element-wise. During the multiplication, each weight value (range 0-1) in the dynamic attention weight matrix is ​​weighted according to the importance of the corresponding feature value in the synchronization feature pair. For example, the weighting coefficient for the three-dimensional distribution field of spectral absorption peak intensity is 0.35±0.05, the weighting coefficient for the acoustic energy flux density vector is 0.30±0.03, and the weighting coefficients for temperature gradient and humidity gradient are 0.20±0.02 and 0.15±0.02, respectively. After the multiplication operation, a spatiotemporal fusion feature tensor is generated.

[0040] The spatiotemporal fusion feature tensor is input into a two-layer fully connected network to obtain the dough fermentation state feature matrix. Furthermore, sample data of past dough fermentation conditions were collected as a training set and input into a two-layer fully connected network for training. During the training process, the Dropout rate and L2 regularization coefficient were combined with the NAdam optimizer for 100-150 training cycles to finally obtain the trained two-layer fully connected network.

[0041] After flattening the spatiotemporal fusion feature tensor, it is input into the first fully connected network (1024 neurons, LeakyReLU activation function, Kaiming normal distribution initialization) after training. After Layer Normalization and Dropout processing, it is then passed through the second fully connected network (256 neurons, linear output, Xavier uniform distribution initialization) for dimensionality reduction. Finally, a dough fermentation state feature matrix is ​​generated, which includes yeast activity index, dough volume change rate, carbon dioxide concentration gradient and temperature and humidity coupling coefficient.

[0042] S5. Input the dough fermentation state feature matrix into the ST-GNN model to obtain dough fermentation state data; Based on the dough fermentation state feature matrix, a bidirectional long short-term memory network layer was constructed, and a two-layer graph convolutional network was configured. The fully connected output layer was set with 5 output neurons corresponding to the dough yeast activity index, dough volume change rate, carbon dioxide concentration gradient, and temperature and humidity coupling coefficient, respectively. The activation functions adopted were linear activation function, ReLU activation function, and Softmax, forming the ST-GNN model. Historical dough fermentation state data were collected and divided into training set and test set. The training set was input into the ST-GNN model for training. The ST-GNN model extracted the temporal features of the dough fermentation state data in the training set through the bidirectional long short-term memory network layer, and then learned the spatial correlation of different parts of the dough in the training set through the two-layer graph convolutional network. During the training process, the backpropagation algorithm was used to optimize the loss function and iteratively update the ST-GNN model parameters. When the performance of the ST-GNN model on the validation set tended to be stable, the training was stopped, and the finally trained ST-GNN model was obtained.

[0043] The dough fermentation state feature matrix is ​​input into the ST-GNN model for feature fusion mapping to generate dough fermentation state data. It should be noted that after the dough fermentation state feature matrix is ​​input into the trained ST-GNN model, a bidirectional long short-term memory network layer performs bidirectional temporal feature extraction, and two graph convolutional networks capture the local and global spatial features in the dough fermentation state feature matrix, respectively. The temporal features output by the bidirectional long short-term memory network layer and the spatial features output by the two graph convolutional networks are input together into the fully connected output layer. The fully connected output layer combines a linear activation function, a ReLU activation function, and a Softmax pair to decouple multiple tasks. Specifically, the yeast activity branch outputs the yeast activity value through a linear activation function, the dough volume branch outputs the non-negative volume change value through a ReLU activation function, the CO2 concentration branch generates the concentration value through a linear activation function, and the dough temperature and humidity branch is directly output through the fully connected output layer, ultimately forming dough fermentation state data including yeast activity value, dough volume change value, CO2 concentration value, and dough temperature and humidity value.

[0044] S6. Convert the dough fermentation status data into a three-dimensional distribution map and display it on the display terminal in real time; The dough fermentation state data is converted into point cloud data in three-dimensional space using a 3D distribution map generation algorithm; It should be noted that the dough fermentation status data is first cleaned and standardized to remove noise and unify dimensions before being input into a 3D distribution map generation algorithm. This algorithm maps yeast activity values ​​to X-axis coordinates, dough volume change values ​​to Y-axis coordinates, and CO2 concentration values ​​to Z-axis coordinates, forming a 3D coordinate system. Yeast activity values, dough volume change values, and CO2 concentration values ​​are each mapped to a point cloud on their respective X, Y, and Z axes. The color of the point cloud is converted from dough temperature and humidity values ​​to point cloud color channel values ​​through a linear transformation. Specifically, the dough temperature and humidity values ​​are first normalized and mapped to the range [0,1]. Then, because the color channel value range is 0-255, the product of the dough temperature value and 255 is converted to a red channel value, the product of the dough humidity value and 255 is converted to a green channel value, and the complementary value of the difference between dough temperature and humidity is converted to a blue channel value, ultimately forming point cloud data reflecting the dynamic changes in fermentation.

[0045] The point cloud data is rendered in real time using a graphics rendering engine to generate a 3D distribution map, which is then presented through a visualization interface on a display terminal. Furthermore, the graphics rendering engine receives point cloud data output from the 3D distribution map generation algorithm, converts the point cloud coordinates into screen space positions using the vertex shader, and calculates the final pixel color based on the point cloud color channel values. The X-axis coordinates in the point cloud data control the horizontal position offset, the Y-axis coordinates control the vertical position offset, and the Z-axis coordinates generate a depth visual effect through perspective projection, ultimately forming a 3D distribution map. The rendered 3D distribution map is output through a visualization interface on the display terminal, allowing users to interactively view the evolution of the dough fermentation process.

[0046] This embodiment also provides a computer device applicable to the intelligent monitoring method for the fermentation state of dough during bread processing, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to realize the intelligent monitoring method for the fermentation state of dough during bread processing as proposed in the above embodiment.

[0047] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0048] This embodiment also provides a storage medium storing a computer program. When executed by a processor, the program implements the intelligent monitoring method for the fermentation state of dough during bread processing, as proposed in the above embodiment. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0049] In summary, this invention achieves comprehensive monitoring of physical, chemical, and environmental factors during dough fermentation through multi-source data acquisition, providing a data foundation for optimizing fermentation conditions. Simultaneously, temporal feature extraction and spatial distribution feature analysis utilize advanced data analysis methods to simplify complex data structures and accurately capture spatiotemporal changes during fermentation, enhancing data relevance and interpretability. Subsequently, a dough fermentation state feature matrix is ​​generated through feature weighted fusion, comprehensively considering the influence of temporal and spatial variables, significantly improving the accuracy and reliability of the prediction model. Finally, the fermentation state data is converted into a three-dimensional distribution map for real-time presentation, enhancing the transparency of dough fermentation monitoring.

[0050] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. An intelligent monitoring method for the fermentation state of dough during bread processing, characterized in that: include, Strain data, spectral data, acoustic signal data, temperature, humidity, and carbon dioxide concentration data of dough in the fermentation chamber were collected and time-stamped to form a multi-source dataset. Temporal feature extraction is performed on multi-source datasets to obtain a temporal feature matrix; The internal space of the fermentation chamber is divided into N monitoring grids. The spectral data, acoustic signal data, temperature and humidity in each monitoring grid are analyzed independently to extract the spatial distribution feature matrix. The temporal feature matrix and spatial distribution feature matrix are input into a dynamic adaptive feature fusion unit. A dough fermentation state feature matrix is ​​generated through a coupled oscillating neural network and an attention mechanism. Specifically... The time-series feature matrix is ​​input into the time-varying wavelet packet transform processor, and multi-scale decomposition is performed using the Daubechies wavelet basis function to extract time-frequency features of different frequency bands and generate a joint time-frequency feature matrix. The spatial distribution feature matrix is ​​input into a nonlocal mean filter to smooth the three-dimensional distribution field of spectral absorption peak intensity, acoustic energy flux density vector, temperature gradient value, and humidity gradient value, thereby generating an enhanced spatial distribution feature matrix. Using the joint time-frequency feature matrix as the input to the driving oscillator and the enhanced spatial distribution feature matrix as the input to the response oscillator, frequency locking and phase synchronization between oscillators are achieved through phase difference feedback adjustment, thereby obtaining synchronization feature pairs. The synchronous feature pairs are converted into query vectors. The temporal feature matrix and the spatial distribution feature matrix constitute the key matrix and the value matrix. By calculating the relevance score between the key matrix and the query vector and normalizing it, a dynamic attention weight matrix is ​​generated. The dynamic attention weight matrix is ​​used to perform Hadamard product operation on the synchronous feature pairs to obtain the spatiotemporal fusion feature tensor. The spatiotemporal fusion feature tensor is input into a two-layer fully connected network to obtain the dough fermentation state feature matrix, specifically... After inputting the spatiotemporal fusion feature tensor into the first layer network and performing a nonlinear transformation using the ReLU activation function, it is then input into the second layer network and performed a dimension mapping using a linear activation function to generate the dough fermentation state feature matrix. The dough fermentation state feature matrix is ​​input into the ST-GNN model to obtain dough fermentation state data. Specifically... Based on the feature matrix of dough fermentation state, a bidirectional long short-term memory network layer is constructed, and a two-layer graph convolutional network is configured. The fully connected output layer is set with 5 output neurons corresponding to the yeast activity, dough volume, CO2 concentration and dough temperature and humidity, respectively. The activation functions are linear activation function, ReLU activation function and Softmax, forming the ST-GNN model. The dough fermentation state feature matrix is ​​input into the ST-GNN model for feature fusion mapping to generate dough fermentation state data. The dough fermentation status data is converted into a three-dimensional distribution map and displayed in real time on the display terminal.

2. The intelligent monitoring method for dough fermentation status during bread processing as described in claim 1, characterized in that: The collection of strain data, spectral data, acoustic signal data, temperature, humidity, and carbon dioxide concentration data of the dough within the fermentation chamber specifically includes: By setting a strain sensor array on the inner wall of the fermentation chamber, the surface deformation of the dough is monitored in real time to obtain strain data; A terahertz spectral scanner was used to perform non-contact scanning of the dough in the fermentation chamber to obtain spectral data on moisture migration and sugar conversion inside the dough. Acoustic sensors capture sound wave signals generated by the movement of air bubbles inside the dough. A gas sensor was used to monitor the rate of change of carbon dioxide concentration in the fermentation chamber, obtain carbon dioxide concentration data, and simultaneously record temperature and humidity data in the fermentation chamber.

3. The intelligent monitoring method for dough fermentation status during bread processing as described in claim 2, characterized in that: The aforementioned timestamp alignment to form a multi-source dataset refers to the process of aligning strain data, spectral data, acoustic signal data, temperature, humidity, and carbon dioxide concentration data with timestamps using a time protocol to form a multi-source dataset.

4. The intelligent monitoring method for dough fermentation status during bread processing as described in claim 3, characterized in that: The step of extracting temporal features from multi-source datasets to obtain a temporal feature matrix specifically involves: The elastic modulus of the dough was calculated using a moving average algorithm based on the strain data. Principal component analysis is used to extract principal component coefficients from spectral data to form spectral feature vectors. After performing a Fourier transform on the acoustic signal data to obtain a time-frequency spectrum matrix, the Mel frequency cepstral coefficients are calculated. Temperature and humidity data are respectively processed using Z-score normalization to obtain standardized temperature and humidity; Perform a first-order difference operation on the carbon dioxide concentration data to generate the gas production rate; The dough elastic modulus, spectral feature vector, Mel frequency cepstral coefficient, gas production rate, standardized temperature and humidity are aligned and spliced ​​along the time axis to form a time-series feature matrix.

5. The intelligent monitoring method for dough fermentation status during bread processing as described in claim 4, characterized in that: The process of dividing the internal space of the fermentation chamber into N monitoring grids refers to collecting the internal geometric dimensions of the fermentation chamber to establish a three-dimensional coordinate system, and dividing it into N monitoring grids at equal intervals along the length, width, and height directions.

6. The intelligent monitoring method for dough fermentation status during bread processing as described in claim 5, characterized in that: The process involves independently analyzing the spectral data, acoustic signal data, temperature, and humidity within each monitoring grid to extract a spatial distribution feature matrix. Specifically... Kriging space interpolation is performed on the spectral data within each monitoring grid to generate a three-dimensional distribution field of spectral absorption peak intensity. After the acoustic signal data is processed by beamforming, the acoustic energy flux density vector at the center point of each monitoring grid is calculated. Finite element analysis was performed on the temperature and humidity within each monitoring grid to obtain the temperature gradient and humidity gradient values. The spatial distribution feature matrix within the fermenter is constructed by the three-dimensional distribution field of the spectral absorption peak intensity, the acoustic energy flux density vector, the temperature gradient value, and the humidity gradient value within each monitoring grid.

7. The intelligent monitoring method for dough fermentation status during bread processing as described in claim 6, characterized in that: The process of converting dough fermentation status data into a three-dimensional distribution map and displaying it in real time on a display terminal specifically involves: The dough fermentation state data is converted into point cloud data in three-dimensional space using a 3D distribution map generation algorithm; The point cloud data is rendered in real time using a graphics rendering engine to generate a 3D distribution map, which is then presented through a visualization interface on a display terminal.

Images