A neural network method and apparatus for modeling multidimensional time-series data in a feature space.

Abstract

This invention discloses a neural network method and apparatus for multidimensional temporal data modeling within a feature space. The invention standardizes and organizes raw observation data to obtain standardized observation data with a uniform form. Based on this standardized observation data, training examples for artificial neural network training are organized, the structure of the artificial neural network is designed, and an artificial neural network model is established. An encoder module (E) is used to extract features from the observation data, reducing the data dimensionality. A generative mechanism model is constructed within the feature space, thereby reducing the complexity of the generative mechanism model, suppressing the possibility of overlearning, and allowing for better training results with fewer samples. By adding encoders and decoders on both sides of the cascaded generative mechanism model, the generative model operating in the feature space can be trained and predicted using actual observation data, while ensuring that the feature data output by the generative mechanism model can be accurately reconstructed from the observation data.

Description

Technical Field

[0001] This invention belongs to the field of computational methods and is a neural network method and apparatus for modeling multidimensional time-series data in a feature space. Background Technology

[0002] Establishing a method for building neural network models of multidimensional time series data, enabling the learning of the driving mechanism of system state changes per unit time, and using neural network models to simulate the temporal changes of the system's internal mechanisms is an important goal of time series analysis in the field of artificial intelligence.

[0003] For example, time-series data modeling and prediction of bio-fermentation processes is an important technology for achieving efficient biomanufacturing. Time-series data of fermentation processes refers to time-point-tagged sequence data collected at various time points during microbial fermentation. Each time point includes a set of fixed indicators, commonly including metabolomics data, fermentation process data, and transcriptomics data. The purpose of time-series data modeling and prediction of fermentation processes is to explore fermentation patterns through simulation, optimize the fermentation process, and achieve better economic and social benefits.

[0004] In multidimensional time-series data modeling applications, there are scenarios involving high-dimensional, small-sample data. This means the data dimensionality is high, but the absolute number of samples is very small or far less than the data dimensionality. Simultaneously, the data across all dimensions are not independent; there are strong correlations between data dimensions, and the number of data dimensions is far greater than the number of independent factors determining data differences. For example, transcriptomic and metabolomic characterization of fermentation processes can measure tens of thousands of different indicators to represent the state of the fermentation system at a specific time point. However, these indicators are not all independent; complex correlations exist between them.

[0005] Modeling directly based on observed data can lead to an excessively large input dimensionality, resulting in excessive model complexity and potentially causing the "curse of dimensionality." The curse of dimensionality refers to the rapid increase in computational complexity as the input dimensionality of the model increases. Increased input dimensionality also leads to a larger sample size required for model training. When the amount of actual observed data is insufficient, the model is prone to overfitting, meaning it only remembers the features of the training data itself, rather than fitting the intrinsic relationships between the training data; overfitting reduces the model's generalization ability. To eliminate or mitigate the curse of dimensionality while improving the generalization ability of the learning model, one strategy is to first extract features from the observed data during modeling. Feature data has a lower dimensionality than observed data and can be reconstructed from observed data with high fidelity using fixed methods. Representing higher-dimensional observed data with lower-dimensional feature data, and then modeling based on the feature data, can effectively reduce the input dimensionality of the model, reduce model complexity, reduce the need for training examples, suppress overfitting, and improve the model's generalization ability.

[0006] Current dimensionality reduction methods are typically performed independently as a preprocessing step in the modeling of multidimensional time series data. These methods mainly include principal component analysis (PCA), independent component analysis (ICA), filtering analysis, and factor analysis. As a preprocessing step, the dimensionality reduction process is independent of the model learning process. The goal of dimensionality reduction methods is to extract feature factors that contain the main information of the data under specific statistical significance, such as feature factors that determine the changes or independence of indicators. However, these feature factors may not necessarily correspond to the key factors driving the changes in the time series and cannot well support the modeling of time series mechanisms in a low-dimensional feature factor space. Furthermore, these dimensionality reduction methods, as preprocessing, require post-processing methods to convert the feature factors into observed values ​​in application. The conversion process may suffer from error amplification; that is, small errors in the feature factors may lead to large errors in predicting the observed data.

[0007] In summary, current methods for modeling high-dimensional time series data with small samples have the following limitations:

[0008] 1) The large dimensionality of the observed data leads to high model complexity, which can easily cause the "curse of dimensionality," that is, as the dimensionality increases, the computational complexity increases significantly.

[0009] 2) Feature factors extracted by feature extraction methods independent of model learning methods may not necessarily match the key factors of the mechanism driving time series changes, and cannot well support the modeling of time series mechanisms in a low-dimensional feature factor space.

[0010] 3) Post-processing methods that convert feature factors into observed values ​​may have the problem of error amplification during the conversion process, that is, small errors in feature factors may lead to large errors in predicting observed data.

[0011] To address the aforementioned issues, we propose a neural network approach for modeling multi-dimensional state change time-series data of a system within a feature space. By embedding a cascaded time-series generation mechanism network within an autoencoder neural network, the autoencoder neural network extracts features from the observed time series, while the generation mechanism network learns and models the time-series generation mechanism within the feature space. In this approach, feature extraction is not a pre-processing step for modeling, nor is the reconstruction of features into observed data a post-processing step for model prediction. The embedding of the autoencoder neural network and the cascaded generation mechanism network enables feature extraction and time-series generation mechanism learning to be completed within the same framework. This not only solves the problems of high model complexity and overfitting through dimensionality reduction but also addresses the issues of adapting feature extraction to the mechanism learning objectives and the error amplification during the transformation from feature data to observed data.

[0012] The neural network method described above, which models multi-dimensional state changes of a system within a feature space, can be used to model the working mechanism of a biofermentation system. This neural network model of the fermentation system's working mechanism can calculate the system state one unit of time later, based on the current system state, for multi-omics time series data of the fermentation process (a type of high-dimensional, small-sample data). Compared to general numerical fitting models of time series state changes, this model of the driving mechanism of time series state changes in a biofermentation system can better reflect the influence of various regulatory mechanisms of engineered bacteria on the fermentation process, and can better guide the genome modification of engineered bacteria to obtain better production performance. Summary of the Invention

[0013] This invention discloses a neural network method and apparatus for multidimensional time-series data modeling in a feature space, specifically implemented through the following technical solution:

[0014] A neural network method for modeling multidimensional time-series data in a feature space includes the following steps:

[0015] 1) The original observation data is standardized and organized to obtain standardized observation data with a uniform format;

[0016] 2) Training examples for artificial neural network training based on standardized observation data;

[0017] 3) Design the structure of the artificial neural network and establish the artificial neural network model;

[0018] 4) Use the training examples from step 2) to train the artificial neural network model established in step 3) to obtain the parameter matrix of the artificial neural network;

[0019] 5) Using the training examples in step 2), evaluate the impact of the parameters used in designing the artificial neural network structure and the parameters used in training the established artificial neural network model on the accuracy of the obtained artificial neural network model, and select the optimal artificial neural network model under different parameter combinations as the final result model.

[0020] The neural network structure designed in step 3) has the following characteristics:

[0021] Neural networks have a chimeric training structure (E-(G)). n -D); The chimeric training structure consists of an encoder module (E) and a concatenated generation mechanism module (G). n The system consists of three modules: (1) encoder module and (2) decoder module. The encoder module and decoder module respectively realize the conversion from observation data to feature data and from feature data to observation data, so that the serial generation mechanism module can learn the working mechanism of the system in the feature space.

[0022] A neural network unit within a cascaded generation mechanism module can model the mechanism by which the system transforms from its current state to its state after one unit of time. Such a network unit is called a unit-time positive change generator (G unit). By cascading G units, it is possible to model the changes that occur in the system over multiple units of time. A cascaded generation mechanism module (G) is composed of multiple cascaded G units. n ).

[0023] As a further improvement, the chimeric training structure described in this invention is established as follows: Chimeric Training Structure (E-(G)) n -D) consists of the encoder module (E) and the cascaded generation mechanism module (G) in sequence. n The system consists of three modules: (1) encoder module (D), (2) decoder module (G), (3) encoder module (4) encoder module (5) decoder module (6) decoder module (7) encoder module (8) encoder module (9) decoder module (10) decoder module (11) decoder module (12) decoder module (13) decoder module (14) decoder module (15) decoder module (16) decoder module (17) decoder module (18) decoder module (19 ... n The G unit is obtained by cascading multiple G units. The number of neurons in each layer of the G unit is equal to the dimension of the feature data. The number of neurons in the input layer of the decoder module is equal to the dimension of the feature data, and the number of neurons in the output layer is equal to the dimension of the observation data.

[0024] As a further improvement, the chimeric training structure with n time intervals described in this invention is called E-(G). n -D: When n=0, ED does not contain G units, and its training data consists of training examples where the start and end time points are the same; when n=1, E-(G) 1 -D contains only one G unit, and its training data consists of training examples with a time interval of 1; when n>1, E-(G) n -D's concatenation generation mechanism module ((G)) n It is obtained by concatenating n G units, and its training data consists of training examples with n time intervals.

[0025] As a further improvement, the training method in step 4) of the present invention is specifically as follows:

[0026] 4.1) Calculate the loss value for each chimeric training architecture: For a specific chimeric training architecture: E-(G) n -D, calculates the model loss using only examples with time intervals of n. It inputs the observation data from the starting time point of the training examples into a cohesive training structure with the corresponding number of time intervals to calculate the loss value (i.e., inputs the observation data from the starting time point of the training examples with n time intervals into the cohesive training structure (E-(G)). n-D)). After the training data is input into the splicing training structure, the network output data is obtained through forward propagation. The loss value of the splicing training structure is calculated from the network output data and the real data. That is, T i Data at specific times Input into the chimeric training structure E-(G) n -D retrieves network output data. Data output from the network With T i+b Real data at specific times The loss value, Loss, can be obtained using the following formula:

[0027] 4.2) Based on the model loss value, gradient calculation is performed to limit the number of backpropagation layers for loss: For the splice training structure with 0 time intervals, which only contains encoder and decoder modules, the start and end time point data of the training examples are the observation data at the same time point. The weight parameters of the decoder and encoder modules are updated, which is essentially equivalent to training an autoencoder network. The updated weight parameters are shared with encoder and decoder modules in other splice training structures. For the splice training structure with 1 time interval, which contains only 1 G-unit, the network uses the error backpropagation training mechanism to directly obtain the update gradients of the G-unit and decoder module. In the splice training structure with more than 1 time intervals, the cascade generation mechanism module consists of multiple G-units connected in series. The gradient is calculated using the error backpropagation mechanism, and the gradients on the decoder module and the last G-unit are taken as the update gradients of the G-unit and decoder module.

[0028] 4.3) Based on the gradient calculation method that limits the number of backpropagation layers of loss, multiple chimeric training structures are updated in sequence, and the network weight parameters are shared during the sequential weight update process. Specifically, for chimeric training structures with different time intervals, the gradient of the chimeric training structure is calculated in a certain order, and the gradient descent method is used to update the weights. The updated weight parameters are shared with all other chimeric training structures.

[0029] 4.4) For K+1 chimeric training structures ED, E-(G) 1 -D、E-(G) 2 -D、E-(G) 3 -D…E-(G) K -D calculates the loss value of each chimeric training structure and determines whether the loss value of each chimeric training structure has converged. If all have converged, the result model is obtained; otherwise, repeat steps 4.1), 4.2), and 4.3) until the loss value of each cascaded training structure has converged, and the result model is obtained.

[0030] As a further improvement, step 1) of the present invention specifically involves: organizing the multidimensional time series observation data into a four-tuple organization form, namely batch, time, indicator, and value; obtaining multiple batches of multidimensional time series observation data through multiple batches of observation; each batch of observation data includes a set of time points; each time point of observation data includes a set of indicators; and the observation data of each indicator is a specific value.

[0031] As a further improvement, step 2) of the present invention specifically involves: organizing the system state changes at any two time points within the same batch into a training example, resulting in a set of training examples with different time intervals. Each training example includes the state at two time points. When the two time points are not the same, the data at the earlier time point is called the starting time point data of the training example, and the data at the later time point is called the ending time point data of the training example. When the two time points are the same, the starting time point data and the ending time point data are the same. The starting time point data and the ending time point data of each training example are represented in the form of an observation data quadruple.

[0032] As a further improvement, the chimeric training structure described in this invention adopts a fully connected structure.

[0033] As a further improvement, step 5) of this invention specifically refers to optimizing the modeling process from the perspective of model hyperparameters. During the optimization process, the hyperparameters used in the remodeling process are adjusted, including: feature dimension; structural parameters of network modules (number of nodes per layer in the encoder module, concatenated generation mechanism module, and decoder module, and number of hidden layers); learning rate of the gradient descent method; number of training data examples input for each training iteration; and number of iterations for cyclic training. By using different combinations of the above hyperparameters, and based on steps 3) and 4), the work of establishing the neural network structure and training the neural network is completed, resulting in a new model. The fitting accuracy of the model to the observed data under each hyperparameter combination is evaluated, and the optimal model is selected.

[0034] The present invention also discloses an apparatus for performing multidimensional time-series data modeling in a feature space, comprising:

[0035] Acquisition Unit: Used to standardize and organize raw observation data to obtain standardized observation data with uniform format;

[0036] Organizational unit: A training example used to organize canonical observation data for training artificial neural networks;

[0037] Building blocks: used to design the structure of artificial neural networks and build artificial neural network models;

[0038] Training unit: Used to train the established artificial neural network model using training examples to obtain the parameter matrix of the artificial neural network;

[0039] Optimization unit: Used to evaluate the impact of parameters used in designing the artificial neural network structure and in training the established artificial neural network model on the accuracy of the resulting artificial neural network model using training examples, and select the optimal artificial neural network model under different parameter combinations as the final result model.

[0040] The designed neural network structure has the following characteristics:

[0041] Neural networks have a chimeric training structure (E-(G)). n -D); The chimeric training structure consists of an encoder module (E) and a concatenated generation mechanism module (G). n The system consists of three modules: (1) encoder module and (2) decoder module. The encoder module and decoder module respectively realize the conversion from observation data to feature data and from feature data to observation data, so that the serial generation mechanism module can learn the working mechanism of the system in the feature space.

[0042] A neural network unit within a cascaded generation mechanism module can model the mechanism by which the system transforms from its current state to its state after one unit of time. Such a network unit is called a unit-time positive change generator (G unit). By cascading G units, it is possible to model the changes that occur in the system over multiple units of time. A cascaded generation mechanism module (G) is composed of multiple cascaded G units. n ).

[0043] The beneficial effects of this invention are as follows:

[0044] 1) Feature extraction of observed data is achieved through the encoder module (E), reducing data dimensionality and constructing a generative mechanism model in the feature space, thereby reducing the complexity of the generative mechanism model, suppressing the possibility of overlearning, and allowing better training results with fewer samples.

[0045] 2) By using the network structure of the generative mechanism model embedded in the autoencoder neural network, feature extraction and generative mechanism model training can be completed in a unified neural network. The training error is calculated at the overall network level. The network weights that complete the feature extraction function and generative mechanism model function are trained in a coordinated manner, which can achieve the adaptation of the feature extraction network and the generative mechanism model and improve the model fitting accuracy.

[0046] 3) By adding encoders and decoders on both sides of the cascaded generative mechanism model, the generative model working in the feature space can be trained and predicted using actual observation data. At the same time, it ensures that the feature data output by the generative mechanism model can be accurately restored to the observation data, which is beneficial to the application of the model. Attached Figure Description

[0047] Figure 1 This is a schematic diagram of the model training steps;

[0048] Figure 2 Example diagram showing how training examples are organized;

[0049] Figure 3 This is a schematic diagram of the serial generation mechanism module structure;

[0050] Figure 4 This is a schematic diagram of loss-restricted backpropagation for a chimeric training structure. Detailed Implementation

[0051] The chimeric training structure (E-(G)) designed in this invention n -D) contains three modules: encoder module (E), concatenated generation mechanism module (G) n The decoder module (D) can be used to model the overall mechanism of a fermentation biological system, including the mechanisms of cell physiology, catalytic reactions, and the regulation of corresponding processes and nutritional conditions. Analysis of this mechanism model can yield possible optimization schemes, guiding the optimization of the fermentation process, including predicting the yield of the target product, modifying the genome of engineered bacteria, and designing fermentation conditions. The method disclosed in this invention models the overall mechanism of a fermentation biological system through the following steps; Figure 1 This is a schematic diagram of the model training steps;

[0052] 1. Organize the time series data into a quaternary organization format.

[0053] The object of modeling and analysis is the "biofermentation system," which includes the microbial strain and the fermentation environment. Therefore, the model needs to be able to learn and predict indicators reflecting the state of the microbial strain and the fermentation environment. The state of the microbial strain and the fermentation environment can be characterized using various omics technologies. The fermentation process is a time series composed of multiple time points; therefore, this fermentation time series data can be organized into a four-element structure: batch, time, indicator, and value. The fermentation time series data is divided into training data and test data. The training data is used to build the model, and the test data is used to evaluate the model's accuracy.

[0054] Suppose the data consists of time-series data measured during the fermentation of actinomycetes with acarbose as the target product. There are (M+U) batches (e.g., M=7, U=3), and each batch has (K+1) time points (e.g., K=10, each time point is denoted as T1, T2, T3…T).11 ).

[0055] For (M+U) batches of actinomycete fermentation, samples at (K+1) time points corresponding to each batch were analyzed by mass spectrometry to determine the abundance of various compounds in each sample. q compounds (e.g., q = 196) related to acarbose synthesis were selected. The abundance of these q compounds was extracted from the compound abundance report of each time point sample and used as the observation value of the bio-fermentation system state at that time point. Compound abundances not observed by mass spectrometry were recorded as missing. Data for time points T3 of batch 2 and T2 of batch 3 were missing due to experimental failure in sample determination.

[0056] The (M+U) batches of actinomycete fermentation data are divided into training data and test data, with M batches as training data and U batches as test data. Therefore, there are a total of (M+U) batches of data, and each batch has (K+1) time points (T1, T2, T3…T…). K+1 Each time point has q-dimensional index data values, where the q-dimensional data of the p-th time point in the m-th batch is represented as follows:

[0057]

[0058] 2. Examples of organizing training data

[0059] Figure 2 This is an example diagram illustrating the organization of training examples; using M batches of actinomycete fermentation data as training data, and based on the quaternary organization method in step 1, the system state changes at any two time points within the same batch are organized as an example, T. a With T b The example training data is organized as follows, representing the start and end times of the example, respectively:

[0060] Batch 1 example set D1:

[0061] Batch 2 example set D2:

[0062] Batch 3 example set D3:

[0063] Batch m example set D m :

[0064] Where 4≤m≤M-1, m∈N *

[0065] Batch M example set D M :

[0066] Then summarize the examples from each batch, according to example T a With T b The interval between the two time points is reorganized as follows:

[0067]

[0068]

[0069]

[0070]

[0071]

[0072] 3. Establish a chimeric training structure for G-units for training examples with different time intervals.

[0073] Figure 3 This is a schematic diagram of the cascaded generation mechanism module structure; an encoder module (E) and a decoder module (D) are constructed: both modules are fully connected neural networks. In the encoder module (E), the number of nodes in the input layer is equal to the dimension r of the observed data, the number of nodes in each hidden layer is s, the number of hidden layers is f1, and the number of nodes in the output layer is equal to the dimension e of the feature data. Similarly, in the decoder module (D), the number of nodes in the input layer is equal to the dimension e of the feature data, the number of nodes in each hidden layer is s, the number of hidden layers is f1, and the number of nodes in the output layer is equal to the dimension r of the observed data. This represents the nth chimeric training structure E-(G). n - The network weight parameters of the encoder module (E) and decoder module (D) in D (for convenience, network weight parameters and network bias parameters are collectively referred to as network weight parameters).

[0074] Construct a serial generation mechanism module ((G)) n The network of a G-unit consists of an input layer, hidden layers, and an output layer, where each layer has e nodes and the number of hidden layers is f². For training examples with different time intervals, the G-units are concatenated according to the number of time intervals to obtain the corresponding concatenated generation mechanism modules: G, (G). 2 G) 3 …(G) K ,use Let it be represented as the nth chimeric training structure E-(G) n -D Serial Generation Mechanism Module (G) i The network weight parameters of the j-th G-unit.

[0075] Then, the encoder module (E) and the serial generation mechanism module (G) are used. nThe three modules (E, G), the decoder module (D), and the coaxial training structure (E-G) are sequentially integrated. n -D).

[0076] The specific parameters of each chimeric training structure are as follows:

[0077] There is no G element in ED: the weight parameters are...

[0078] E-(G) 1 -D contains 1 G unit: weight parameters include

[0079] E-(G) 2 -D contains 2 G units: weight parameters include

[0080] E-(G) 3 -D contains 3 G units: weight parameters include

[0081]

[0082] E-(G) K -D contains K G units: weight parameters include

[0083] The initial weight parameters of the G unit, E module, and D module of each chimeric training structure are set to the same random values: Right now: Before training begins, all are equal to Before training begins, all are equal to Before training begins, all are equal to

[0084] 4. Calculate the model loss value

[0085] The starting time point T of the training examples in group c a The multidimensional data is input into a spliced ​​training structure with corresponding time intervals to calculate the loss value. (The set is...) Multidimensional data at c starting time points T Enter E-(G) n In the -D option, the output data is obtained. Where n≤K, n∈N. With T b Multidimensional real data at specific time points The loss value can be calculated. n The calculation formula is:

[0086] 5. Calculate the gradient through loss-restricted backpropagation, and update the network weight parameters through sequential iterative optimization.

[0087] Figure 4 This is a schematic diagram of loss-restricted backpropagation for a chimeric training structure. For a chimeric training structure with zero time intervals, which contains only encoder and decoder modules, the weight parameters of the decoder and encoder modules are updated.

[0088] For a chimeric training structure with a time interval of 1, which contains only 1 G unit, the network uses an error backpropagation training mechanism to directly obtain the update gradients of the G unit and the decoder module.

[0089] In a spliced ​​training structure with a time interval greater than 1, the cascaded generation mechanism module consists of multiple G-units cascaded together. The gradient is calculated using the backpropagation mechanism, and the gradients at the decoder module and the last G-unit are extracted as the update gradients for the G-units and decoder module, respectively. The specific calculation formula is as follows:

[0090] For the splice training structure with zero time intervals, ED only calculates the update gradients of the encoder and decoder modules, i.e.

[0091]

[0092]

[0093] For the chimeric training structure E-(G) with a time interval of 1... 1 -D, calculate The update gradient of the decoder module, i.e.

[0094]

[0095]

[0096] For a chimeric training structure E-(G) with two time intervals... 2 -D, calculate The update gradient of the decoder module, i.e.

[0097]

[0098]

[0099] For the chimeric training structure E-(G) with 3 time intervals... 3 -D, calculate The update gradient of the decoder module, i.e.

[0100]

[0101]

[0102]

[0103] For a chimeric training structure E-(G) with K time intervals... K -D, calculate The update gradient of the decoder module, i.e.

[0104]

[0105]

[0106] The gradient calculation method described above only calculates the gradients of the last two structural units of the spliced ​​training structure each time (when the time interval is 0, only the gradients of the decoder module and the encoder module are calculated; when the number of time intervals is greater than 1, only the gradients of the last G unit and the encoder module are calculated). Therefore, this invention calls this method loss-restricted backpropagation.

[0107] Based on the gradients calculated using the above method, for chimeric training structures with different time intervals, the gradients of the chimeric training structures with smaller time intervals are calculated first, in order of the number of time intervals. Then, the SGD stochastic gradient descent method is used to update the weights. The updated weights are shared with all other chimeric training structures. This invention refers to this training method as sequential loop optimization. The number of loops in the loop training is set to H, where the training process for the t-th loop is as follows:

[0108] First, calculate the update gradients of the encoder and decoder modules in the ED, i.e. Where α represents the learning rate of the SGD gradient descent method; This represents the updated weight parameters of the encoder and decoder modules at the end of the (t-1)th training iteration. Represents the updated weight parameters of the t-th cyclic encoder and decoder modules; represents the updated gradients of the encoder and decoder modules. The weight parameters are shared with other cohesive training structures, i.e.:

[0109] All equal to All equal to

[0110] Then calculate E-(G) 1 -D The update weights of the unit and decoder module, i.e. in, This indicates that at the end of the (t-1)th training cycle, G1... The updated weight parameters of the unit In the t-th iteration of G1 The updated weight parameters of the unit;

[0111] Will Gradients updated by unit and decoder modules The weight parameters are shared with other cohesive training structures, i.e.:

[0112] All equal to All equal to

[0113] Calculate E-(G) 2 -D The update weights of the unit and decoder module, i.e. Then Gradients updated by unit and decoder modules The weight parameters are shared with all other chiral training structures.

[0114] Update E-(G) using the same calculation method. 3 -D、E-(G) 4 -D、E-(G) 5 -D…E-(G) K -D, when the current training loop calculates up to E-(G) K -D After the unit and decoder modules update their weights and share them with other chimeric training structures, the current training loop ends, and the next training loop begins.

[0115] 6. Obtain the final model after all training structure losses have converged.

[0116] After each training cycle in step 5, for the K+1 chimeric training structures ED, E-(G) 1 -D、E-(G) 2 -D、E-(G) 3 -D…E-(G) K The -D option, following the method described in step 4, inputs all training examples for the corresponding time interval, yielding the full data loss value for each concatenated training module. The specific calculation is as follows:

[0117] Calculate the loss value of ED, i.e.

[0118] Calculate E-(G) 1 The loss value of -D, i.e.

[0119] Calculate E-(G) 2 The loss value of -D, i.e.

[0120] Calculate E-(G) 3 The loss value of -D, i.e.

[0121]

[0122] Calculate E-(G) K The loss value of -D, i.e.

[0123] Determine Loss0, Loss1, Loss2, Loss3,…,Loss K If all values ​​converge, proceed to the next step; otherwise, continue with steps 4 and 5 until Loss0, Loss1, Loss2, Loss3, ..., Loss K All converge. Let the overall loss of the chimeric training structure converge after H iterations of the training process.

[0124] 7. Adjust the model hyperparameters and optimize the resulting model.

[0125] After obtaining the model through training in step 6, adjust the model's hyperparameters: feature dimension e, network module structural parameters (number of hidden layer nodes s and number of hidden layers f1 in the encoder and decoder modules; number of nodes per layer e and number of hidden layers f2 in the concatenated generation mechanism module), learning rate α for the SGD stochastic gradient descent method, number of training examples c in a single training iteration, and number of training iterations H. After adjusting the hyperparameters, repeat steps 4-6 until the network loss difference converges, then terminate training.

[0126] 8. Obtain the optimal model

[0127] After hyperparameter optimization in step 7, the optimal G-unit, encoder module, and decoder module are obtained. The network structure and network weights of the G-unit, encoder module, and decoder module are saved. The prediction accuracy of the model is then verified using test data.

[0128] The fermentation data of U batches of actinomycetes described in the first step of the application case are used as test data, with each batch having K+1 time points (T1, T2, T3…T…). K+1 ), with q metrics at each time point, the test data is organized as follows, in steps 1 and 2:

[0129]

[0130]

[0131]

[0132]

[0133]

[0134] in, There is Total N l For example, l≤K, l∈N * .

[0135] Calculate the prediction accuracy of ED, i.e.

[0136] Calculate E-(G) 1 -D's prediction accuracy, i.e.

[0137] Calculate E-(G) 2 -D's prediction accuracy, i.e.

[0138] Calculate E-(G) 3 -D's prediction accuracy, i.e.

[0139]

[0140] Calculate E-(G) K -D's prediction accuracy, i.e.

[0141] Thus, the encoder module (E), decoder module (D), and system mechanism model (G unit) in the bio-fermentation system mechanism model are used to predict the accuracy of system state changes in the bio-fermentation system at different time intervals.

[0142] This invention also discloses an apparatus for modeling multidimensional time series data using a method for modeling multidimensional time series data, comprising:

[0143] Acquisition Unit: Used to standardize and organize raw observation data to obtain standardized observation data with uniform format;

[0144] Organizational unit: A training example used to organize canonical observation data for training artificial neural networks;

[0145] Building blocks: used to design the structure of artificial neural networks and build artificial neural network models;

[0146] Training unit: Used to train the established artificial neural network model using training examples to obtain the parameter matrix of the artificial neural network;

[0147] Optimization unit: Used to evaluate the impact of parameters used in designing the artificial neural network structure and in training the established artificial neural network model on the accuracy of the resulting artificial neural network model using training examples, and select the optimal artificial neural network model under different parameter combinations as the final result model.

[0148] The designed neural network structure has the following characteristics:

[0149] Neural networks have a chimeric training structure (E-(G)). n -D); The chimeric training structure consists of an encoder module (E) and a concatenated generation mechanism module (G). n The system consists of three modules: (1) encoder module and (2) decoder module. The encoder module and decoder module respectively realize the conversion from observation data to feature data and from feature data to observation data, so that the serial generation mechanism module can learn the working mechanism of the system in the feature space.

[0150] A neural network unit within a cascaded generation mechanism module can model the mechanism by which the system transforms from its current state to its state after one unit of time. Such a network unit is called a unit-time positive change generator (G unit). By cascading G units, it is possible to model the changes that occur in the system over multiple units of time. A cascaded generation mechanism module (G) is composed of multiple cascaded G units. n ).

[0151] The above is not intended to limit the specific embodiments of this patent. It should be noted that those skilled in the art can make various changes, modifications, additions, or substitutions without departing from the essential scope of this invention, and these improvements and refinements should also be considered within the scope of protection of this invention.

Claims

1. A method for modeling time series data of multidimensional state changes in a fermentation system, characterized in that, Includes the following steps: 1) The original observation data is standardized and organized to obtain standardized observation data with a uniform format. The observation data includes multidimensional state data related to the fermentation system, including multi-omics time series data of metabolomics data, fermentation process data and transcriptomics data; 2) Training examples for artificial neural network training are organized based on standardized observation data. The training examples take the multidimensional state data of the fermentation system at the current time point as input and the multidimensional state data at the next time point as output. 3) Design the structure of the artificial neural network and establish the artificial neural network model; 4) Use the training examples from step 2) to train the artificial neural network model established in step 3) to obtain the parameter matrix of the artificial neural network; 5) Using the training examples in step 2), evaluate the impact of the parameters used in designing the artificial neural network structure and the parameters used in training the established artificial neural network model on the accuracy of the obtained artificial neural network model, and select the optimal artificial neural network model under different parameter combinations as the final result model. The neural network structure designed in step 3) has the following characteristics: The neural network described has a chimeric training structure (E The described chimeric training structure consists of an encoder module (E) and a cascaded generation mechanism module (E). ), decoder module ( It consists of three modules; the encoder module and the decoder module respectively realize the conversion of observation data to feature data and feature data to observation data, so that the serial generation mechanism module can learn the working mechanism of the system in the feature space; One neural network unit in the aforementioned cascade generation mechanism module can model the working mechanism of the fermentation system transitioning from its current state to its state after a unit time. It is used to calculate the system state after a unit time based on the current fermentation system state. Its input is the multi-omics, multi-dimensional state data at the current fermentation time point, and its output is a prediction of the state data at the next fermentation time point. Such a network unit is called a unit-time positive change generator. Unit); through The cascading of units allows for modeling of changes in a system over multiple time units, consisting of multiple The series connection of units forms a series generation mechanism module ( ).

2. The method for modeling multi-dimensional state change time series data of a fermentation system according to claim 1, characterized in that, The chimeric training structure is established as follows: Chimeric Training Structure (E ) consists of an encoder module (E) and a serial generation mechanism module (E) in sequence. ), decoder module ( It consists of three modules: the encoder module has an input layer with the number of neurons equal to the dimension of the observed data, an output layer with the number of neurons equal to the dimension of the feature data, and a concatenated generation mechanism module. ) composed of multiple Units are connected in series. The number of neurons in each layer of the unit is equal to the dimension of the feature data. The number of neurons in the input layer of the decoder module is equal to the dimension of the feature data, and the number of neurons in the output layer is equal to the dimension of the observation data.

3. The method for modeling multi-dimensional state change time series data of a fermentation system according to claim 2, characterized in that, The number of time intervals is The chimeric training structure is called E : =0 E Not included A unit is a training example whose start and end time points are the same. =1 E It contains only 1 A unit, whose training data consists of training examples with a time interval of 1; When >1, E The concatenation generation mechanism module in the middle ( )Depend on indivual The training data is obtained by concatenating units, and the number of time intervals is... Training examples.

4. The method for modeling multi-dimensional state change time series data of a fermentation system according to claim 1, 2, or 3, characterized in that, The training method in step 4) is as follows: 4.1) Calculate the loss value for each chimeric training architecture: For a specific chimeric training architecture: E Only use time intervals For example, to calculate the model loss, the observed data at the start time of the training examples are input into a spliced ​​training structure with a corresponding number of time intervals to calculate the loss value. That is, the number of time intervals is... The observation data at the starting time point of the training examples are input into the chimeric training structure (E). After the training data is input into the splicing training structure, the network output data is obtained through forward propagation. The loss value of the splicing training structure is calculated by comparing the network output data with the real data. Data at specific times Input into the chimeric training structure E Obtain network output data E Data E output from the network and Real data at specific times Loss value The formula is ; 4.2) Based on the model loss value, perform gradient calculation for the number of backpropagation layers to limit the loss: For a spliced ​​training structure with 0 time intervals, which only contains encoder and decoder modules, the start and end time point data of the training examples are observation data at the same time point. Update the weight parameters of the decoder and encoder modules, which is essentially equivalent to training an autoencoder network. The updated weight parameters are shared with encoder and decoder modules in other spliced ​​training structures; For a spliced ​​training structure with 1 time interval, which only contains 1... The unit, this network uses an error backpropagation training mechanism to directly obtain Gradient updates for unit and decoder modules; the concatenated generation mechanism module in a chimeric training structure with a time interval greater than 1 consists of multiple Units are cascaded, and the gradient is calculated using the backpropagation mechanism. The decoder module and the last unit are then extracted. The gradient on the unit is used as Update gradients for the unit and decoder modules; 4.3) Based on the gradient calculation method for limiting the number of backpropagation layers of loss, multiple chimeric training structures are updated in sequence, and the network weight parameters are shared during the sequential weight update process. Specifically, for chimeric training structures with different time intervals, the gradient of the chimeric training structure is calculated in a certain order, and the gradient descent method is used to update the weights. The updated weight parameters are shared with all other chimeric training structures. 4.4) To A chimeric training structure E E E E ... E Calculate the loss value for each chimeric training structure and determine whether the loss value of each chimeric training structure has converged. If all have converged, the resulting model is obtained; otherwise, repeat steps 4.1), 4.2), and 4.3) until the loss value of each cascaded training structure has converged, and the resulting model is obtained.

5. The method for modeling multi-dimensional state change time series data of a fermentation system according to claim 1, characterized in that, Step 1) specifically involves organizing the multidimensional time series observation data into a four-tuple structure: batch, time, index, and value. The batch represents the batch number of the fermentation experiment, the time represents the fermentation sampling time point, the index represents a certain state variable of the fermentation system, and the value represents the specific measured value of the index. Multiple batches of multidimensional time series observation data are obtained through observation of multiple batches. The observation data of each batch includes a set of fermentation time points, the observation data of each fermentation time point includes a set of indices, and the observation data of each index is a specific value.

6. The method for modeling multi-dimensional state change time series data of a fermentation system according to claim 1 or 5, characterized in that, Step 2) specifically involves: organizing the system state changes at any two fermentation time points within the same batch into a training example. By comparing the state data at the two fermentation time points, the training example captures the state change patterns of the fermentation system. This method is suitable for handling the problem of incomplete time-series data caused by uneven sampling frequencies, resulting in a set of training examples with different time intervals. Each training example includes the state at two fermentation time points. The data at the earlier fermentation time point is called the starting fermentation time point data of the training example; the data at the later fermentation time point is called the ending fermentation time point data of the training example. The starting fermentation time point data and the ending fermentation time point data of each training example are represented in the form of an observation data quadruple.

7. The method for modeling multi-dimensional state change time series data of a fermentation system according to claim 1, characterized in that, The described chimeric training structure adopts a fully connected structure.

8. The method for modeling multi-dimensional state change time series data of a fermentation system according to claim 7, characterized in that, Step 5) specifically refers to optimizing the modeling process from the perspective of model hyperparameters. During optimization, the hyperparameters used in the remodeling process are adjusted, including: feature dimension; structural parameters of the network modules, such as the number of nodes per layer and the number of hidden layers in the encoder, concatenated generation mechanism, and decoder modules; the learning rate of the gradient descent method; the number of training data examples input for each training iteration; and the number of iterations in the cyclic training. Different combinations of these hyperparameters are used, based on steps 3) and 4), to complete the establishment of the neural network structure and the training of the neural network, resulting in a new model. The fitting accuracy of the model to the observed data under each hyperparameter combination is evaluated, and the optimal model is selected.

Images