Long sequence prediction method for key production variables in polyester fiber polymerization process

Abstract

The invention relates to a long-sequence prediction method for key production variables in a polyester fiber polymerization process, and the method comprises the steps: inputting the historical data of a certain time length of a plurality of key production variables in the polyester fiber polymerization process into a proposed time anchor network model, and outputting a predicted value of a future time step of the plurality of key production variables; an encoder of the time anchor point network model is a plurality of time anchor point layers, and each time anchor point layer is formed by connecting a time anchor point sampling module and a graph mask full-connection neural network in series; a decoder of the time anchor network model is a two-layer full-connection neural network. According to the method, modeling is carried out on a multivariable prediction task of an aggregation process from the time scale and spatial correlation, the proposed anchor point sampling-based technology contains the idea of division and treatment, data is subjected to block sampling, different neural networks are adopted to extract features, a time-based anchor point network model is established, and the time-based prediction task of the aggregation process is obtained. The problem of long sequence modeling can be effectively solved, and accurate prediction of key production variables is realized.

Description

Technical Field

[0001] The invention belongs to the technical field of intelligent production of chemical fibers and relates to a long sequence prediction method for key production variables in a polyester fiber polymerization process. Background Art

[0002] The polymerization process is used to produce polyethylene terephthalate (PET) melt for spinning. The process indicators of the polymerization process and the stability of the production process directly determine the quality of the PET melt. However, the production process of the polymerization process is complex and there are many types of processes. Typical polymerization processes include three-pot polymerization to five-pot polymerization. Taking three-pot polymerization as an example, it includes esterification process, pre-polycondensation and final polycondensation process. The raw materials between each sub-process can be used for each other. For example, part of the ethylene glycol (EG) produced in the final polycondensation process will be fed back to the esterification process for reuse. In addition, the polymerization process is very dependent on the stable heat medium temperature provided by the heat medium system and the stable catalyst flow provided by the additive system. The characteristic viscosity of the PET melt finally generated by the polymerization process is a key quality indicator, and its size is closely related to the pressure of the final polycondensation reactor. Therefore, the polymerization process of polyester fiber is a complex nonlinear system with coupled mechanism and process. In addition to the complex operation process of the system itself, it is also affected by external environmental factors, making it difficult to implement mechanism modeling of the polymerization process.

[0003] Due to its strong data fitting ability, deep learning models can be applied to various application environments that are difficult to model explicitly. At present, models developed based on deep learning technology for polyester fiber polymerization processes include tasks such as soft measurement, anomaly detection, and prediction. In the prediction task, there is currently little research on the realization of multi-output prediction of multiple variables. In addition, the residence time of different reactors in the polymerization process is long, and the final performance indicators of the polymer melt are from the raw materials being fed into the esterification reactor to the PET melt being transported to the spinning process, and the overall reaction cycle in the middle is long. The current sampling rate of DCS data is recorded once per minute, but the entire reaction cycle may be as long as more than ten hours. Therefore, in order to utilize the historical data of the entire cycle, the length of the input data determined according to the production cycle should be able to improve the prediction accuracy of the model. However, deep learning models (such as recurrent neural networks) have the well-known problem of forgetting historical information, which makes how to avoid the forgetting problem a research hotspot in general time series long sequence modeling.

[0004] Another important issue is the relationship between different variables. For a long time, for multivariate time series data given by the data set, modelers often assume that the variables are completely coupled to each other. For each time series prediction, all other covariates in the given data set are used to predict the variable. However, some studies have found that this complete mixing of variables may have negative effects. The most advanced model currently considers each variable separately and only uses the data of each variable itself to predict its future time trajectory. Although graph convolutional neural networks (GCNs) have modeled sparse connections between variables in fields such as traffic flow, their essence is a model based on network diffusion dynamics and can only be used for pure diffusion networks. However, the polyester fiber polymerization process is a complex system in which multiple dynamics interact with each other. It is difficult to use GCN to extract relationship information between variables without graph structure information.

[0005] Therefore, the aggregation process data is more complex than any other public dataset, and there are certain challenges in modeling the aggregation process.

[0006] Therefore, it is of great significance to study a long sequence prediction method for key production variables in the polyester fiber polymerization process to solve the above problems. Summary of the invention

[0007] The purpose of the present invention is to solve the problems existing in the prior art and provide a long sequence prediction method for key production variables in a polyester fiber polymerization process.

[0008] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0009] Based on the long sequence prediction method of key production variables in the polyester fiber polymerization process with multi-scale sampling in time and space, the historical data of a certain time length of multiple key production variables in the polyester fiber polymerization process are input into the Time Anchor Network (TANK, Time Anchor NetworkworK) model, and the predicted values ​​of the future time steps of the multiple key production variables are obtained as output;

[0010] The encoder of the time anchor network model is a multi-layer time anchor layer (TAL), each of which is composed of a time anchor sampling module (TASM) and a graph mask fully connected neural network (GMFCNN) in series; the decoder of the time anchor network model is a 2-layer fully connected neural network; when applied to the key variable prediction task of the aggregation process, the series of multi-layer time anchor layers can further deepen the depth of the network, forming a time anchor network (TANK) framework for multi-variable prediction;

[0011] The input object of the time anchor sampling module is the historical data X of the key production variables sampled from the DCS, and the output object is the time series feature

[0012] The graph masked fully connected neural network is formed by masking the weights of a fully connected neural network using the adjacency matrix between multiple key production variables; the time series features output by the time anchor sampling module are Input to the graph mask fully connected neural network to achieve feature aggregation operations between different key production variables, specifically:

[0013]

[0014]

[0015] H o =σ(H2+H1);

[0016] Among them, H1 and H2 are temporary variables, and Cat is the splicing operation on the feature channel. is the parameter of the fully connected neural network, σ is the ReLU activation function, ⊙ is the element-by-element multiplication of the matrix, H o is the output of the graph mask fully connected neural network, and Z is the adjacency matrix reflecting the connection relationship between different key production variables.

[0017] As the preferred technical solution:

[0018] The long sequence prediction method of key production variables of polyester fiber polymerization process based on spatiotemporal multi-scale sampling as mentioned above has the following specific steps:

[0019] (1) First, the historical data X of the key production variables sampled from the DCS and the text description data W of each key production variable are input into the multimodal fusion graph structure learning module, and an adjacency matrix Z reflecting the connection relationship between the variables is output; the multimodal fusion graph structure learning module fuses the historical data information of the variables with the text description information of the variables to infer the graph structure information. Compared with the existing graph structure learning neural network, the multimodal fusion graph structure learning module uses the text description information between variables for the first time to learn the graph structure; then the adjacency matrix Z is combined with the weights of a fully connected neural network Perform point-by-point multiplication to implement mask operation and obtain a graph mask fully connected neural network;

[0020] (2) Constructing a time anchor network model; the encoder of the time anchor network model is a multi-layer series of time anchor layers, each of which is formed by a time anchor sampling module and a graph mask fully connected neural network in series, and the decoder of the time anchor network model is a 2-layer fully connected neural network;

[0021] (3) First, the historical data X of the key production variables sampled from the DCS is input into the time anchor sampling module in the first time anchor layer, and the time series features are output. Then the time series features Input into the graph mask fully connected neural network in the first layer of time anchor layer, each layer of time anchor layer outputs H o As the input of the next layer of time anchor layer, the output feature of the final multi-layer time anchor layer is H Q , H Q Then input it into the decoder of the time anchor network model, and output the predicted value F of the future time step of multiple key production variables; by using the adjacency matrix describing the graph structure to mask the weight parameters of the multi-layer fully connected neural network, the graph structure information can be easily integrated. In addition, the masked fully connected neural network can act as a nonlinear learner of the connection strength between variables, which can improve the model performance.

[0022] As described above, the long sequence prediction method for key production variables of polyester fiber polymerization process based on spatiotemporal multi-scale sampling, the process of step (1) includes: firstly, X and W are merged into a feature fusion matrix H through word embedding and variable embedding, and then H is input into a neural network dedicated to graph structure learning and outputs a graph structure Z representing the internal relationship between sensors; the specific steps of constructing the adjacency matrix Z are as follows:

[0023] (1.1) Collect and analyze data and clean the data;

[0024] The data preprocessing process is a key step in effectively modeling and determining the accuracy of the model. After discussing and deciding with the company's engineers, the original sampled data determines the multivariate variables that need to be predicted. According to the determined variable names, the variable description data and time series data are searched and extracted from the company database. Since the sampled data may have problems such as missing variable descriptions, inaccurate descriptions, missing time series data, anomalies, etc., the missing values ​​of the data are interpolated by manually checking the variable description information and using the data translation or mean method.

[0025] (1.2) Multimodal data embedding;

[0026] Since the original data includes Chinese and English text data W and time series data X, it is necessary to embed the two modal data separately. At present, large language models with multilingual capabilities have been widely studied. The present invention uses a large language model only to obtain a good word embedding vector, so the multilingual version of the Bert model is selected to obtain the word embedding vector H. w The Bert model does not participate in subsequent model training.

[0027] H W= Bert(W);

[0028] Among them, W represents the text description data of key production variables, Bert represents a large language deep learning model, and H w is the output word embedding feature.

[0029] For time series data, since the original sampling data only contains sensor scalar values, the position information of the data in the time series is not encoded. Therefore, the following time position encoding method is used:

[0030]

[0031]

[0032] Where PE is the position embedding vector, t is the time information, i is a dimension of embedding, d is the total embedding dimension, sin(·) and cos(·) represent sine and cosine functions respectively.

[0033] The obtained embedding vector PE is concatenated with the sensor historical data X to obtain the multi-channel input historical data after time embedding. A convolutional network is used to Perform feature fusion to obtain the embedded feature H of historical data X :

[0034]

[0035]

[0036] Among them, X is the historical data of key production variables obtained by sampling, Cat represents the concatenation operation of features, and Conv represents the convolution operation. represents the data after the sensor data and time information are embedded, H X Historical data embedding features representing key production variables.

[0037] (1.3) Multimodal Fusion Graph Structure Learning Module (GSLM)

[0038] The purpose of graph structure learning is to infer a sparse graph structure. The structure is represented by a binary matrix Z with the same dimension as the number of variables. Since there is no prior knowledge, the pairwise relationship between all variables is first constructed. If there are N variables, the number of common variable pairs is E = N 2-N, E is the number of edges in the graph. Based on the constructed node pairs, the message passing method is used to pass features on the edges and vertices respectively, and neural networks are used to model edge embedding and node embedding on the edges and nodes respectively. Finally, a Softmax function is used to map the edge features to a discrete probability distribution, which belongs to the set {0,1}. Among them, 1 means that the edge exists, and 0 means it does not exist. The specific formula is as follows:

[0039] H=Cat(H W ,H X );

[0040]

[0041]

[0042]

[0043]

[0044]

[0045] Among them, Cat is the concatenation operation of the features, H is the feature fusion matrix for structure learning, and f emb is a feature dimension reduction convolutional neural network, is the feature of node j and is used as the input of the first layer of edge convolutional neural network, is the feature of node i and is used as the input of the first layer of edge convolutional neural network, is the first layer of edge convolutional neural network, for The edge features between the output nodes i and j are: is the first layer node convolutional neural network, for The output feature of node j is used as the input of the second layer of edge convolutional neural network. for The output feature of node i is used as the input of the second layer of edge convolutional neural network. To learn the second layer of edge convolutional neural network of graph structure, for The edge features between nodes i and j are output, g is the independently sampled Gumbel distribution noise, τ is the hyperparameter, z ij represents the edge probability between node i and node j.

[0046] As described above, the long sequence prediction method for key production variables of polyester fiber polymerization process based on spatiotemporal multi-scale sampling, the present invention introduces the idea of ​​ensemble sampling for the first time, thereby abandoning the point-by-point data input form of recurrent neural network, and in addition, by specifying the number of sampling anchor points, the number of sampling subsets can be controlled, so as to adapt to data with different input history lengths; the anchor-based multi-scale sampling time feature extraction layer is named Time Anchor Sampling Module (TASM); for example, the time length of a single sample of polyester fiber polymerization process data input to TASM is 12 hours of data, and the sampling interval is 1 minute, then the sample has 720 data points; the number of anchor points can be set according to the length of subset sampling. If each anchor point samples 3 consecutive data points, then in order to ensure that all data are included in the subset, the number of anchor points can be set to 360; in order to control each anchor point to aggregate information of different scales in the time dimension, a multi-scale aggregation method is designed; multiple subsets are sampled at different distances from each anchor point, and the size of the subset gradually increases as the distance from the anchor point increases; therefore, for each anchor point, if O subsets are sampled, O neural networks are used to extract features from the O subsets respectively; therefore, for each neural network, it captures information of different scales and different time positions, thereby greatly reducing the possibility of neural network forgetting. The time anchor point sampling module in step (2) is mainly divided into three construction steps. First, the anchor point set is sampled according to the coordinates of the time point. Secondly, according to With manually specified distance set Sample multi-scale features of each anchor point from X to form a feature subset Finally Input into the neural network f and output the new features of X Through the subset sampling technology based on anchor points, multi-scale information of long sequences can be aggregated at different anchor points, which solves the widespread forgetting problem in time series modeling and can flexibly integrate task-related prior information. The construction steps of the time anchor sampling module are as follows:

[0047] (2.1) Data encoding: First, embed each discrete data point of the historical data X of the key production variable into the time coordinate in chronological order. The data sample after embedding the time coordinate is expressed as:

[0048]

[0049] where s j :=(t j ,x j ) represents the encoding tuple of each time point, including the coordinates of a time point With vector value C represents the number of channels of the variables input into the time anchor network model, L represents the length of the historical data X of the key production variables, represents a positive real number; a data set consisting of N samples is represented as According to the time coordinate t j , each data sample Divide into a fixed length A subset of represents a positive integer, and the subset is defined as in

[0050] (2.2) Sampling anchor point: According to the time coordinate t j Sample a set of anchor points at equal intervals M is the number of anchor points; note that the anchor point set is based on the embedded time coordinate t j Sampling, used for subsequent data x j Indexing; then, according to the distance anchor point Distance Sampling a subset of samples with fixed length k Finally, a neural network is used to extract features from each anchor subset and pool each anchor subset into a single time point with multiple channels. After a layer of TAL feature extraction, a new set of M subsets is formed The above process can be restated as follows:

[0051]

[0052]

[0053]

[0054] Where G is the sampling function, According to the anchor point With parameters (d, k) The sampling function for sampling, f is the feature mapping function, which can be a convolutional neural network or a multi-layer perceptron. The number of elements in is equal to the number of sampled anchor points, and each element is the feature of each anchor point, specifically:

[0055]

[0056]

[0057] in, D represents the number of channels of the output feature of f;

[0058] (2.3) Multi-scale feature aggregation: for each anchor point p m Define a distance set where d i Indicates the distance anchor point The distance, k i Represents the distance d from each anchor point i The length of the continuous data points that need to be sampled at the distance, O represents the number of subsets that need to be aggregated for each anchor point, that is, the number of scales of the multi-scale aggregation described here; Note that the distance set For each anchor point p m All the same;

[0059] For anchor point collection All anchor points in are equivalent to the sampled subset set as For anchor point collection Any anchor point p in m ,according to O subset samplings were performed, and O independent neural networks were used to extract multi-scale features of the time dimension for each of the O subsets. Concatenate to the channel dimension:

[0060]

[0061] Among them, f i represents the i-th neural network, ∪ represents the channel splicing operation, Represents the anchor point p m The output after multi-scale feature aggregation, σ is the ReLU activation function;

[0062] The time series features output by the time anchor sampling module (i.e., the anchor set The set of outputs after multi-scale feature aggregation of all anchor points in

[0063] As described above, the long sequence prediction method based on spatiotemporal multi-scale sampling of key production variables in the polyester fiber polymerization process, the process of step (3) is specifically expressed as follows:

[0064]

[0065]

[0066] H o =σ(H2+H1);

[0067]

[0068]

[0069] Among them, H1 and H2 are temporary variables. is the parameter of the fully connected neural network, σ is the ReLU activation function, ⊙ is the element-by-element multiplication of the matrix, H o is the output of the graph mask fully connected neural network, Z is the adjacency matrix reflecting the connection relationship between different key production variables, represents the kth time anchor point layer, k∈{1,2,…Q}, H Q It represents the final output feature after the Q-layer temporal anchor layer feature extraction. represents the 2-layer fully connected neural network that constitutes the decoder, and F represents the final prediction result.

[0070] Beneficial effects:

[0071] (1) The present invention models the multivariate prediction task of the polymerization process from the perspective of time scale and spatial correlation. The proposed anchor sampling-based technology contains the idea of ​​divide and conquer. By sampling the data in blocks and using different neural networks to extract features, the problem of long sequence modeling can be effectively solved. In addition, the multimodal data existing in the polyester production process is used to jointly infer a sparse graph structure that considers the equipment structure and dynamic mechanism. This structure can be used to realize the function of feature aggregation between variables and adaptive learning of the relationship strength between variables through the parameter matrix of the graph masked fully connected neural network, thereby achieving accurate prediction of key production variables.

[0072] (2) In order to model the relationship between some correlated variables among multiple variables and use these relationships to improve the prediction accuracy of the model, a method is proposed to model the spatial correlation between multiple variables by performing dot product between graph structure information and the parameter matrix of a fully connected neural network. The difference between this method and the general graph convolutional neural network is that the graph mask fully connected layer here is used to learn the correlation between related variables, which is equivalent to learning a weighted graph; the adjacency matrix Z pre-performs hard deletion on irrelevant edges, and no feature aggregation is performed between irrelevant time series. For two variables represented by edges in the binary graph, their correlation is learned using a graph mask fully connected neural network;

[0073] (3) The present invention is a deep learning modeling method for long-period, nonlinear coupling variables in the polyester fiber polymerization process. This method is mainly used for multi-variable prediction tasks, which solves the problem of forgetting historical information in traditional deep learning models and solves the problem of coupling modeling between multiple variables.

[0074] (4) The present invention adopts the set method to model time series for the first time. The current mainstream sequence modeling methods are the point-by-point recursive form of recurrent neural networks and the sliding convolution method of temporal convolutional networks. The present invention is completely different from the current mainstream methods and has a more flexible modeling method. The degree of compression of input information can be controlled by reasonably setting hyperparameters. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 A schematic diagram of the process flow for modeling the present invention;

[0076] Figure 2 is a process flow chart of the polymerization process;

[0077] Figure 3 This is the schematic diagram of the Graph Structured Learning Module (GSLM);

[0078] Figure 4 This is the overall framework diagram of the TANK model;

[0079] Figure 5 It is the schematic diagram of the Time Anchor Sampling Module (TASM);

[0080] Figure 6 The waveform tracking comparison of the predicted value and the true value for 1000 consecutive minutes at the 15th time step of the predicted value; the left figure is the variable LIC_0401.MV, and the right figure is the variable LIT_0401A.PV;

[0081] Figure 7 The waveform tracking comparison of the predicted value and the true value for 1000 consecutive minutes at the 15th time step of the predicted value; the left figure is the variable LIT_0401B.PV, and the right figure is the variable TE_0402.PV;

[0082] Figure 8 This is the variable connection graph learned by the graph structure learning module. DETAILED DESCRIPTION

[0083] The present invention will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall within the scope limited by the appended claims of the application equally.

[0084] The long sequence prediction method of key production variables in the polyester fiber polymerization process based on spatiotemporal multi-scale sampling is as follows:

[0085] (1) The historical data X of key production variables sampled from DCS and the text description data W of each key production variable are merged into a feature fusion matrix H through word embedding and variable embedding. Then H is input into the multimodal fusion graph structure learning module (GSLM) to output an adjacency matrix Z that reflects the connection relationship between variables. The adjacency matrix Z is then combined with the weights of a fully connected neural network. Perform point-by-point multiplication to implement mask operation and obtain a graph mask fully connected neural network;

[0086] Among them, the construction method of the adjacency matrix Z is as follows:

[0087]

[0088]

[0089]

[0090]

[0091]

[0092]

[0093]

[0094]

[0095]

[0096] Where PE is the position embedding vector, t is the time information, i is a dimension of embedding, d is the total embedding dimension, sin(·) and cos(·) represent sine and cosine functions respectively, W represents the text description data of key production variables, X represents the historical data of key production variables, Bert is a large language deep learning model, Conv is the convolution operator, Cat is the concatenation operation on the feature channel, and h j is the jth column of matrix H, H is the feature fusion matrix for structure learning, f emb is a feature dimension reduction convolutional neural network, is the feature of node j and is used as the input of the first layer of edge convolutional neural network, is the feature of node i and is used as the input of the first layer of edge convolutional neural network, is the first layer of edge convolutional neural network, for The edge features between the output nodes i and j are: is the first layer node convolutional neural network, for The output feature of node j is used as the input of the second layer of edge convolutional neural network. for The output feature of node i is used as the input of the second layer of edge convolutional neural network. To learn the second layer of edge convolutional neural network of graph structure, for The edge features between nodes i and j are output, g is the independently sampled Gumbel distribution noise, τ is the hyperparameter, z ij represents the edge probability between node i and node j;

[0097] (2) Constructing a time anchor network model; the encoder of the time anchor network model is a multi-layer series of time anchor layers, each of which is formed by a time anchor sampling module and a graph mask fully connected neural network in series, and the decoder of the time anchor network model is a 2-layer fully connected neural network;

[0098] Among them, the construction steps of the time anchor sampling module are as follows:

[0099] (2.1) Data encoding: First, embed each discrete data point of the historical data X of the key production variable into the time coordinate in chronological order. The data sample after embedding the time coordinate is expressed as:

[0100]

[0101] where s j :=(t j ,x j ) represents the encoding tuple of each time point, including the coordinates of a time point With vector value C represents the number of channels of the variables input into the time anchor network model, L represents the length of the historical data X of the key production variables, represents a positive real number; a data set consisting of N samples is represented as According to the time coordinate t j , each data sample Divide into a fixed length A subset of represents a positive integer, and the subset is defined as in

[0102] (2.2) Sampling anchor point: According to the time coordinate t j Sample a set of anchor points at equal intervals M is the number of anchor points; the anchor point set is used for subsequent data x j Indexing; then, according to the distance anchor point Distance Sampling a subset of samples with fixed length k Finally, a neural network is used to extract features from each anchor subset and pool each anchor subset into a single time point with multiple channels. After a layer of temporal anchor layer (TAL) feature extraction, a new set of M subsets is formed The above process can be restated as follows:

[0103]

[0104]

[0105]

[0106] Where G is the anchor point sampling function, Indicates that according to the anchor point With parameters (d, k) The sampling function for sampling, f is the feature mapping function, which can be a convolutional neural network or a multi-layer perceptron. The number of elements in is equal to the number of sampled anchor points, and each element is the feature of each anchor point, specifically:

[0107]

[0108]

[0109] in, D represents the number of channels of the output feature of f;

[0110] (2.3) Multi-scale feature aggregation: for each anchor point p m Define a distance set where d i Indicates the distance anchor point The distance, k i Represents the distance d from each anchor point i The length of the continuous data points that need to be sampled at the distance, O represents the number of subsets that need to be aggregated for each anchor point;

[0111] For anchor point collection All anchor points in are equivalent to the sampled subset set as For anchor point collection Any anchor point p in m ,according to O subset samplings were performed, and O independent neural networks were used to extract multi-scale features of the time dimension for each of the O subsets. Concatenate to the channel dimension:

[0112]

[0113] Among them, f i represents the i-th neural network, ∪ represents the channel splicing operation, Represents the anchor point p m The output after multi-scale feature aggregation, σ is the ReLU activation function;

[0114] The time series features output by the time anchor sampling module (i.e., the anchor set The set of outputs after multi-scale feature aggregation of all anchor points in

[0115] (3) First, the historical data X of the key production variables sampled from the DCS is input into the time anchor sampling module in the first time anchor layer, and the time series features are output. Then the time series features Input into the graph mask fully connected neural network in the first layer of time anchor layer, each layer of time anchor layer outputs H o As the input of the next layer of time anchor layer, the output feature of the final multi-layer time anchor layer is H Q , H Q Then input it into the decoder of the time anchor network model, and output the predicted value F of the future time step of multiple key production variables, specifically:

[0116]

[0117]

[0118] H o =σ(H2+H1);

[0119]

[0120]

[0121] Among them, H1 and H2 are temporary variables. is the parameter of the fully connected neural network, σ is the ReLU activation function, ⊙ is the element-by-element multiplication of the matrix, H o is the output of the graph mask fully connected neural network; represents the kth time anchor point layer, k∈{1,2,…Q}, H Q It represents the final output feature after the Q-layer temporal anchor layer feature extraction. represents the 2-layer fully connected neural network that constitutes the decoder, and F represents the final prediction result.

[0122] The present invention is described in detail below in conjunction with specific cases:

[0123] Long sequence prediction method for key production variables of polyester fiber polymerization process based on spatiotemporal multi-scale sampling, such as Figure 1 , 4 As shown, the specific process is as follows:

[0124] (1) Select key process variables and construct data sets;

[0125] The selection of key process variables requires rich experience in production processes. The selection of variables in the present invention is jointly selected by multiple engineers of the enterprise. The process analysis of the esterification, pre-polycondensation and final polycondensation processes of the polymerization process is carried out. The process flow chart of the polymerization process is shown in FIG. Figure 2 As shown in Table 1, 30 key process variables are selected. The sampling timestamp is used to encode the position of the time point.

[0126] Table 1 Description of polyester polymerization process variables

[0127]

[0128]

[0129] As can be seen from Table 1, the selected process variable description is missing. This problem is solved by combining with the specific process of the production process and adding variable descriptions. The present invention selects data sampled from the real production process to construct a data set for training, verification and testing. The specific data set description is shown in Table 2 below.

[0130] Table 2 Description of polyester polymerization process data set

[0131]

[0132] According to the selected variable name, connect to the enterprise database to find the corresponding production process historical data. When constructing the data set, you need to normalize the variable data first, using the following method:

[0133]

[0134] Since the dimensions of different variables are different, each variable is normalized separately. In the above formula, μ is the mean of each variable, and σ is the variance.

[0135] The normalized data is used to construct training samples using the sliding window method. The important point is to add the encoding of the sampling time point to each constructed sample. The time length of the data sample constructed using the sliding window is L = 720, then the normalized node coordinates of each sample are embedded, that is, the embedded coordinate sequence is The coordinates are used for anchor point selection and distance-based multi-scale sampling, and subset sampling is performed based on the distance.

[0136] (2) Graph structure learning;

[0137] The description information between variables shown in Table 1 reflects the sensor type and layout, and describes the topological structure information between certain variables from the perspective of the device. However, this structure may not be accurate for the prediction task, so the present invention also uses the historical data of the sensor. The principle of graph structure learning is as follows Figure 3 shown.

[0138] a) Convert the variable description information text into a tensor form that can be used by the model, and use the open source BERT-Base Multilingual Cased version model from Google to obtain the text embedding vector.

[0139] H W = Bert(W);

[0140] The word embedding H W The tensor dimension is C is the number of variables, L is the time length of a time series instance, and S is the channel dimension of word embedding. In this case, C is 30, L is 720, and S is 256.

[0141] b) To embed historical data, first embed the time according to the following formula, where time i is a dimension of embedding, and d is the total embedding dimension. In this case, d is 256.

[0142]

[0143]

[0144] Concatenate the position embedding with each node's time series data in dimension to get a new tensor A convolutional neural network is used to map the concatenated vectors to a new tensor H X middle:

[0145]

[0146] Among them, Conv is a convolutional neural network.

[0147] c) Concatenate the outputs of steps a) and b) above to obtain the embedding vector H for inputting the graph structure learning module:

[0148] H=Cat(H W ,H X ).

[0149] (3) Model hyperparameter setting;

[0150] like Figure 5 The figure shows the principle diagram of multi-scale feature extraction of the time anchor sampling module (TASM). For the prediction of multivariate time series, independent time dimension feature extraction is performed on each time series, and then the masked perceptron is used to extract spatial dimension features based on the learned graph structure. On the polyester polymerization process dataset, the hyperparameter τ of the graph structure learning module is set to 0.5, the number of encoder cascaded time anchor layers (TAL) is set to 3, the input time series length is 720 minutes, and the prediction time step is 120 minutes; the parameter settings for each TAL are shown in Table 3.

[0151] Table 3 TSAM parameter settings

[0152]

[0153] (4) Model training;

[0154] The model is trained based on the constructed model and data set. The training process is the same as that of a general neural network. The learning rate is set to 0.001, the batch size of the input model is set to 50, the loss function is the widely used MSE loss function, the model training cycle (epoch) is set to 100, the model training framework is Pytorch1.8, the stochastic gradient descent method is used, and the Adam optimizer is used for optimization; during the training process, the model with the smallest prediction error on the validation set will be saved, and the model finally saved is the optimal model that appears within 100 training cycles;

[0155] (5) Model validation;

[0156] In order to verify the predictive ability of the model, the optimal model saved in the training process is tested. The data set used is the validation set data shown in Table 2. On the validation set, 720 minutes of historical data of the 30 key variables shown in Table 1 are input into TANK, and the output prediction waveform should be equivalent to the real waveform of the 30 variables in the future.

[0157] Figure 6 , Figure 7 The following are the prediction results of some key production variables, where the dotted line is the true value. The waveform tracking of the predicted value and the true value for 1000 consecutive minutes at the 15th time step of the predicted value is plotted here. By comparison, it can be found that the overall trend of the above TANK prediction results is consistent with the trajectory changes of the real variables. Figure 8 This is the graph structure information learned by the graph structure learning module. The nodes of the graph represent the key production variables in Table 1, and the node numbers correspond to the variable numbers. The edges in the graph indicate that there is a correlation between two variables, and this relationship is learned through the joint prediction of multiple variables, so it is more suitable for prediction tasks.

Claims

1. A long sequence prediction method for key production variables in polyester fiber polymerization process based on spatiotemporal multi-scale sampling, characterized by: Inputting historical data of a certain time length of multiple key production variables in the polyester fiber polymerization process into a time anchor network model, and outputting predicted values ​​of the multiple key production variables in future time steps; The encoder of the temporal anchor network model is a multi-layer temporal anchor layer, each of which is composed of a temporal anchor sampling module and a graph mask fully connected neural network in series; the decoder of the temporal anchor network model is a 2-layer fully connected neural network; The input object of the time anchor sampling module is the historical data X of the key production variables sampled from the DCS, and the output object is the time series feature The graph masked fully connected neural network is formed by masking the weights of a fully connected neural network using the adjacency matrix between multiple key production variables; the time series features output by the time anchor sampling module are Input to the graph mask fully connected neural network to achieve feature aggregation operations between different key production variables, specifically: H. =o(H2+H); Among them, H1 and H2 are temporary variables, and Cat is the splicing operation on the feature channel. is the parameter of the fully connected neural network, σ is the ReLU activation function, ⊙ is the element-by-element multiplication of the matrix, H o is the output of the graph mask fully connected neural network, and Z is the adjacency matrix reflecting the connection relationship between different key production variables.

2. The long sequence prediction method for key production variables of polyester fiber polymerization process based on spatiotemporal multi-scale sampling according to claim 1 is characterized in that: The specific steps are as follows: (1) First, the historical data X of the key production variables sampled from the DCS and the text description data W of each key production variable are input into the multimodal fusion graph structure learning module, and an adjacency matrix Z reflecting the connection relationship between the variables is output; then the adjacency matrix Z is combined with the weights of a fully connected neural network Perform point-by-point multiplication to implement mask operation and obtain a graph mask fully connected neural network; (2) Constructing a time anchor network model; the encoder of the time anchor network model is a multi-layer series of time anchor layers, each of which is formed by a time anchor sampling module and a graph mask fully connected neural network in series, and the decoder of the time anchor network model is a 2-layer fully connected neural network; (3) First, the historical data X of the key production variables sampled from the DCS is input into the time anchor sampling module in the first time anchor layer, and the time series features are output. Then the time series features Input into the graph mask fully connected neural network in the first layer of time anchor layer, each layer of time anchor layer outputs H o As the input of the next layer of time anchor layer, the output feature of the final multi-layer time anchor layer is H Q , H Q It is then input into the decoder of the time anchor network model, and the output is the predicted value F of the future time step of multiple key production variables.

3. The long sequence prediction method for key production variables of polyester fiber polymerization process based on spatiotemporal multi-scale sampling according to claim 2 is characterized in that: The construction method of the adjacency matrix Z in step (1) is as follows: Among them, PE is the position embedding vector, t is the time information, i is a dimension of embedding, d is the total embedding dimension, W represents the text description data of key production variables, X represents the historical data of key production variables, Bert is a large language deep learning model, Conv is the convolution operator, Cat is the splicing operation on the feature channel, represents the data after the sensor data and time information are embedded, H is the feature fusion matrix for structure learning, and h j is the jth column of matrix H, f emb is a feature dimension reduction convolutional neural network, is the feature of node j and is used as the input of the first layer of edge convolutional neural network, is the feature of node i and is used as the input of the first layer of edge convolutional neural network, f e 1 is the first layer of edge convolutional neural network, f e 1 The edge feature between the output node i and node j, f v 1 is the first layer node convolutional neural network, f v 1 The output feature of node j is used as the input of the second layer of edge convolutional neural network. f v 1 The output feature of node i is used as the input of the second layer of edge convolutional neural network, f e 2 To learn the second layer of edge convolutional neural network of graph structure, f e 2 The edge features between nodes i and j are output, g is the independently sampled Gumbel distribution noise, τ is the hyperparameter, z ij represents the edge probability between node i and node j.

4. The long sequence prediction method for key production variables of polyester fiber polymerization process based on spatiotemporal multi-scale sampling according to claim 2 is characterized in that: The construction steps of the time anchor sampling module in step (2) are as follows: (2.1) Data encoding: First, embed each discrete data point of the historical data X of the key production variable into the time coordinate in chronological order. The data sample after embedding the time coordinate is expressed as: where s j :=(t j ,x j ) represents the encoding tuple of each time point, including the coordinates of a time point With vector value C represents the number of channels of the variables input into the time anchor network model, L represents the length of the historical data X of the key production variables, represents a positive real number; (2.2) Sampling anchor point: According to the time coordinate t j Sample a set of anchor points at equal intervals M is the number of anchor points; (2.3) Multi-scale feature aggregation: for each anchor point p m Define a distance set where d i Indicates the distance anchor point The distance, k i Represents the distance d from each anchor point i The length of the continuous data points that need to be sampled at the distance, O represents the number of subsets that need to be aggregated for each anchor point; For anchor point collection All anchor points in are equivalent to the sampled subset set as For anchor point collection Any anchor point p in m ,according to O subset samplings were performed, and O independent neural networks were used to extract multi-scale features of the time dimension for each of the O subsets. Concatenate to the channel dimension: Among them, f i represents the i-th neural network, U represents the channel splicing operation, Represents the anchor point p m The output after multi-scale feature aggregation, σ is the ReLU activation function; Time series features output by the time anchor sampling module 5. The long sequence prediction method for key production variables of polyester fiber polymerization process based on spatiotemporal multi-scale sampling according to claim 2 is characterized in that: The process of step (3) is specifically expressed as follows: H o =σ(H2+H1); Among them, H1 and H2 are temporary variables. is the parameter of the fully connected neural network, σ is the ReLU activation function, ⊙ is the element-by-element multiplication of the matrix, H o is the output of the graph mask fully connected neural network, Z is the adjacency matrix reflecting the connection relationship between different key production variables, represents the kth time anchor point layer, k∈{1,2,…Q}, H Q It represents the final output feature after the Q-layer temporal anchor layer feature extraction. represents the 2-layer fully connected neural network that constitutes the decoder, and F represents the final prediction result.

Images