A prediction method, device and storage medium for multi-data value collaboration in manufacturing industry

Through the embedding, coding and attention mechanism processing of the multi-time sequence prediction model, the prediction problem of high-noise data in manufacturing is solved, high-accuracy prediction is achieved, and the support capabilities of intelligent data applications in manufacturing industry are improved.

CN114936676BActive Publication Date: 2024-12-06SHANGHAI JIAOTONG UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210451403.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-04-26
Publication Date
2024-12-06
Estimated Expiration
2042-04-26

AI Technical Summary

Technical Problem

In the manufacturing industry, data mining technology analysis capabilities are weak, and it is difficult to find potential correlations and causal relationships in high-noise, multi-modal massive data, resulting in low prediction accuracy and ineffective support for intelligent perception and data intelligent applications.

Method used

Using a multivariate timing prediction model, data processing is carried out through embedding, encoding, attention and autoregressive modules, high-dimensional vector embedding, encoding and nonlinear fusion of target variables and environmental variables is achieved, and combined with supervised learning training models, manufacturing production scenarios are carried out. data predictions under.

Benefits of technology

Without the need for additional feature extraction and selection, the prediction accuracy is improved, and it has flexibility and scalability, effectively reducing the interference of environmental noise on the target variables, and improving the prediction effect.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114936676B_ABST
    Figure CN114936676B_ABST
Patent Text Reader

Abstract

The present invention relates to a prediction method, device and storage medium for multi-data value collaboration in the manufacturing industry, the method comprising: constructing a multivariate time series prediction model, the input of the model being a time series sequence of multivariate variables in a historical period, the output of the model being a predicted value of the multivariate variables at the time to be predicted, the multivariate variables including target variables and environmental variables; training the multivariate time series prediction model; using the trained multivariate time series prediction model to predict data in a manufacturing production scenario. Compared with the prior art, the present invention uses an end-to-end multivariate time series prediction model to predict data, and does not require additional feature extraction, feature selection and other operations when processing high-noise, highly correlated manufacturing big data, and has flexibility and scalability.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of manufacturing data mining and processing, and in particular to a prediction method, device and storage medium for collaborative multi-data value prediction in manufacturing. Background Art

[0002] In the comprehensive value chain covering the entire cycle of manufacturing product production, transportation, and sales, there will be multiple types of data such as time series data, process data, production network structure data, and user data. To achieve the synergy of the value chain, it is necessary to deeply explore the intrinsic connection of information within and across the value chain, identify the patterns within multi-source heterogeneous data, and further achieve value-driven collaborative optimization tasks. At present, manufacturing companies generally face the challenges of weak data mining technology analysis capabilities and weak comprehensive capabilities of intelligent application services. Due to the high requirements of the manufacturing industry for model reliability, the commonly used data mining technologies in the industry are mainly traditional mathematical and statistical methods that can ensure good robustness and consistency. However, these models are difficult to discover potential correlations and causal relationships when faced with massive multimodal and high-noise data. Therefore, how to effectively discover the complex nonlinear relationships between multivariate time series, improve the accuracy of predictions, and provide support for intelligent perception and data intelligence applications in more key links of the manufacturing industry has become a technical problem that needs to be solved urgently. Summary of the invention

[0003] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a prediction method, device and storage medium for the collaborative value of multiple data in the manufacturing industry. When processing high-noise and highly correlated manufacturing big data, no additional feature extraction, feature selection and other operations are required, and it has flexibility and scalability.

[0004] The purpose of the present invention can be achieved by the following technical solutions:

[0005] A method for predicting the value of multi-data collaboration in manufacturing industry, the method comprising:

[0006] Constructing a multivariate time series prediction model, wherein the input of the model is a time series sequence of multivariate variables in a historical period, and the output of the model is a predicted value of the multivariate variables at the time to be predicted, wherein the multivariate variables include a target variable and an environmental variable;

[0007] Training multivariate time series forecasting models;

[0008] The trained multivariate time series forecasting model is used to predict data in manufacturing production scenarios.

[0009] Preferably, the multivariate time series prediction model includes:

[0010] Embedding module: transforms the target variable and environmental variable in the time series of multivariate variables in the input historical period into high-dimensional vectors using two different embedding operations;

[0011] Encoding module: Encode the transformed high-dimensional vectors of the target variable and the environment variable to obtain the target variable hidden state matrix H1 and the environment variable hidden state matrix H2;

[0012] Attention module: Based on the target variable hidden state matrix H1 and the environment variable hidden state matrix H2, the attention mechanism is used to perform data fusion processing to obtain nonlinear fusion data;

[0013] Autoregression module: Perform linear autoregression processing on the time series of multivariate variables in the input historical period to obtain linear autoregression data;

[0014] Aggregation output module: Aggregate nonlinear fusion data and linear autoregressive data to obtain prediction results.

[0015] Preferably, the encoding module includes a long short-term memory network.

[0016] Preferably, the target variable hidden state matrix H1 is an m1*n dimensional matrix, and the environment variable hidden state matrix H2 is an m2*n dimensional matrix, where n is the total number of sampling moments of the input time series, and m1 and m2 are constants;

[0017] The execution process of the attention module includes:

[0018] Take each row vector of H1 as a query, calculate the dot product of each row vector of H1 and the row vector of H2 as the attention weight of each row vector of H2, use the attention weight to weight the sum of the row vectors of H2 to get the context vector, and concatenate all the context vectors to H1 along the feature dimension. The concatenated matrix is ​​recorded as H3;

[0019] Take the column vector h3 at the end of H3 as the query, calculate the dot product of all column vectors in H3 as the attention weight of each column vector of H3, and use the attention weight to find the weighted sum of all column vectors of H3 to get a context vector c3;

[0020] Take the column vector h2 at the end of H2 as the query, calculate the dot product of all column vectors in H2 as the attention weight of each column vector of H2, and use the attention weight to find the weighted sum of all column vectors of H2 to get a context vector c2;

[0021] Concatenate h3 and c3 along the feature dimension to obtain a nonlinear fusion vector X1, and concatenate h2 and c2 along the feature dimension to obtain a nonlinear fusion vector X2.

[0022] Preferably, the execution process of the aggregation output module includes:

[0023] The linear autoregressive data processed by the autoregressive module is recorded as c1, where c1 is a k-dimensional vector, and each dimension corresponds to a dimension of the multivariate variable;

[0024] Perform linear transformation and concatenation on the nonlinear fusion vector X1 and the nonlinear fusion vector X2 to form a k-dimensional vector r1;

[0025] Add c1 and r1 to get the prediction results of the multivariate variable.

[0026] Preferably, the multivariate time series prediction model is trained using supervised learning.

[0027] Preferably, the mean square error is used during training to calculate the loss between the network prediction result and the actual output, and the network parameters are updated by gradient back propagation.

[0028] Preferably, after a round of training, the prediction results are verified using the validation set. If the loss on the validation set is less than the current minimum loss, the model parameters are temporarily stored. Training continues until the loss value converges.

[0029] A prediction device for collaborative multi-data value in the manufacturing industry includes a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to implement the prediction method when executing the computer program.

[0030] A storage medium stores a computer program, which implements the prediction method when executed by a processor.

[0031] Compared with the prior art, the present invention has the following advantages:

[0032] (1) The present invention adopts an end-to-end multivariate time series prediction model to predict data. When processing high-noise and highly correlated manufacturing big data, no additional feature extraction, feature selection and other operations are required, and it has flexibility and scalability;

[0033] (2) The present invention embeds and predicts the target variable and the environmental variable separately, effectively reducing the interference of environmental noise on the prediction of the target variable;

[0034] (3) The present invention uses the attention mechanism to fuse multivariate data, selects environmental variables that are more relevant to the target variable, and simultaneously realizes the fusion of data at the sampling time to improve the prediction effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a schematic diagram of the overall framework of the multivariate time series prediction model of the present invention;

[0036] Figure 2 This is a flow chart of calculating the H3 vector in an embodiment of the present invention;

[0037] Figure 3 This is a flow chart of using the attention mechanism for data fusion and data aggregation in an embodiment of the present invention. DETAILED DESCRIPTION

[0038] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. Note that the following embodiments are merely illustrative in nature, and the present invention is not intended to limit its applicable objects or uses, and the present invention is not limited to the following embodiments.

[0039] Example

[0040] This embodiment provides a prediction method for the value coordination of multiple data in the manufacturing industry. According to the needs of the specific manufacturing process, the method collects data from the manufacturing process and divides it into a target data (target variable) that needs to be predicted and a variety of data that have a potential impact on the prediction target (environmental variables). For example, the machine monitoring indicators such as temperature, humidity, and pressure on the production line are environmental variables, and the product yield or yield is the target variable. The purpose of the present invention is to use the target variable and environmental variable data of the historical period to predict the target variable and environmental variable at the future moment.

[0041] The method provided in this embodiment includes:

[0042] Construct a multivariate time series forecasting model. The input of the model is the time series of multivariate variables in the historical period. The output of the model is the predicted value of the multivariate variables at the time to be predicted. The multivariate variables include target variables and environmental variables.

[0043] Training multivariate time series forecasting models;

[0044] The trained multivariate time series forecasting model is used to predict data in manufacturing production scenarios.

[0045] Among them, Figure 1 As shown,

[0046] Multivariate time series forecasting models include:

[0047] Embedding module: The target variable and the environment variable in the time series of the input historical period multivariate variable are respectively embedded using two different operations (i.e. Figure 1 The target embedding layer and the environment embedding layer in are transformed into high-dimensional vectors;

[0048] Encoding module: The high-dimensional vectors of the target variable and the environment variable after transformation are encoded to obtain the target variable hidden state matrix H1 and the environment variable hidden state matrix H2. The encoding module includes a long short-term memory network (LSTM), that is, Figure 1The target LSTM module and the environment LSTM module in;

[0049] Attention module: Based on the target variable hidden state matrix H1 and the environment variable hidden state matrix H2, the attention mechanism is used to perform data fusion processing to obtain nonlinear fusion data;

[0050] Autoregression module: Perform linear autoregression processing on the time series of multivariate variables in the input historical period to obtain linear autoregression data;

[0051] Aggregation output module: Aggregate nonlinear fusion data and linear autoregressive data to obtain prediction results.

[0052] Assuming that the target variable hidden state matrix H1 is an m1*n dimensional matrix, and the environment variable hidden state matrix H2 is an m2*n dimensional matrix, where n is the total number of sampling moments of the input time series, and m1 and m2 are constants, the execution process of the attention module includes:

[0053] Take each row vector of H1 as a query, calculate the dot product of each row vector of H1 and the row vector of H2 as the attention weight of each row vector of H2, use the attention weight to weight the sum of the row vectors of H2 to get the context vector, and concatenate all the context vectors to H1 along the feature dimension. The concatenated matrix is ​​recorded as H3;

[0054] Take the column vector h3 at the end of H3 as the query, calculate the dot product of all column vectors in H3 as the attention weight of each column vector of H3, and use the attention weight to find the weighted sum of all column vectors of H3 to get a context vector c3;

[0055] Take the column vector h2 at the end of H2 as the query, calculate the dot product of all column vectors in H2 as the attention weight of each column vector of H2, and use the attention weight to find the weighted sum of all column vectors of H2 to get a context vector c2;

[0056] Concatenate h3 and c3 along the feature dimension to obtain a nonlinear fusion vector X1, and concatenate h2 and c2 along the feature dimension to obtain a nonlinear fusion vector X2.

[0057] The execution process of the aggregation output module includes:

[0058] The linear autoregressive data processed by the autoregressive module is recorded as c1, where c1 is a k-dimensional vector, and each dimension corresponds to a dimension of the multivariate variable;

[0059] Perform linear transformation and concatenation on the nonlinear fusion vector X1 and the nonlinear fusion vector X2 to form a k-dimensional vector r1;

[0060] Add c1 and r1 to get the prediction results of the multivariate variable.

[0061] The present invention adopts a supervised learning method to train a multivariate time series prediction model. The specific process of training the multivariate time series prediction model includes:

[0062] 1. Preprocess all data: including filling missing values, removing outliers, and normalizing using mean and variance to obtain a multivariate time series data as input.

[0063] 2. Split the time series: After dividing and combining the time series into some (input, output) pairs, supervised learning is used for training. The input is a multivariate time series with a length equal to the window size, and the output is the actual multivariate variable value after a fixed interval of time. The training set, validation set, and test set are divided into a ratio of 0.6, 0.2, and 0.2.

[0064] 3. Initialize the parameters of the deep neural network and input the training set into the network to start training. During the training process, the mean square error is used to calculate the loss between the network prediction result and the actual output, and the network parameters are updated through gradient back propagation. After a round of training, the prediction result is verified using the validation set. If the loss on the validation set is less than the current minimum loss, the model parameters are temporarily stored. Continue training until the loss value converges.

[0065] After training is completed, the model results are tested on the test set, and the pre-trained model can be used for practical applications.

[0066] refer to Figure 1-3 A specific implementation process of the method of the present invention is given. First, refer to Figure 1-3 , it is easy for those skilled in the art to understand the internal structure of the multivariate time series prediction model (deep neural network). On this basis, refer to Figure 2-3 , the model training method of the present invention is introduced. Taking a multi-stage continuous flow data set of manufacturing industry as an example, it contains 1 target variable and 9 environmental variables. The sampling frequency of the data set is 1 second / time, and the window size is 60, that is, the data of the previous minute is used to predict the current data.

[0067] like Figure 2 As shown:

[0068] (1) The target variable and environment variable are transformed into 8*60 and 16*60 dimensional vectors respectively through two different embedding layers.

[0069] (2) These two vectors pass through two hidden layers of LSTM modules (Long Short-Term Memory Network) with dimensions of 8 and 16 respectively, and output 8*60 dimensional hidden state matrix H1 and 16*60 dimensional hidden state matrix H2.

[0070] (3) Take each row vector of H1 as a query and calculate the dot product between them and the row vector of H2 as the attention weight. The specific calculation method is: calculate the dot product between a row vector of H1 and each row vector of H2 as the similarity score, and then normalize it to between 0 and 1 through Softmax to obtain the attention weight of each row vector of H2. Then, take the weighted sum of all row vectors of H2 to obtain a context vector corresponding to the row vector of H1. Repeat the above process to calculate the context vector corresponding to each row vector of H1, and concatenate all context vectors under H1 along the feature dimension to obtain a 16*60-dimensional matrix H3.

[0071] like Figure 3 As shown:

[0072] (4) Take the column vector h3 at the end of H3 as the query and calculate the dot product attention weights for all column vectors in H3. The specific calculation method is: calculate the dot product between h3 and the column vectors at each previous moment as the similarity score, and then normalize it to between 0 and 1 through Softmax to obtain the attention weight of each column vector, and then weighted sum all column vectors of H3 to obtain a 16*1 dimensional context vector c3.

[0073] (5) Perform the same operation as step (4) on H2. Take the column vector h2 at the end of H2 as the query, calculate the dot product attention weight for all column vectors in H2, and use the attention weight to calculate the weighted sum of all column vectors of H2 to obtain a 16*1 dimensional context vector c2.

[0074] (6) The original multivariate time series input is passed through an autoregressive layer to obtain a 10*1 dimensional autoregressive vector c1.

[0075] (7) h2 and c2, h3 and c3 are concatenated into two vectors along the feature dimension, compressed to 16 dimensions through a linear transformation, the two results are concatenated and compressed to 10 dimensions through a linear transformation, and finally added to c1 to obtain the final prediction result as the output.

[0076] The mean square error between the actual values ​​corresponding to the output and input is calculated as the loss of network training, and the network parameters are updated iteratively.

[0077] After the training is completed, the trained multivariate time series prediction model is used to perform predictions in practical applications. For example, the temperature, humidity, pressure and other machine monitoring indicators on the production line are environmental variables, and the product yield or yield is the target variable. The above method is used to predict the target variable and the environmental variable.

[0078] The present invention adopts an end-to-end multivariate time series prediction model to predict data. When processing high-noise and highly correlated manufacturing big data, it does not require additional feature extraction, feature selection and other operations, and has flexibility and scalability.

[0079] Based on the above prediction method, this embodiment also provides a prediction device for multi-data value collaboration in the manufacturing industry, which includes a memory and a processor, the memory is used to store computer programs, and the processor is used to implement the above-mentioned prediction method when executing the computer program.

[0080] In addition, this embodiment further provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned prediction method is implemented.

[0081] The above embodiments are merely examples and do not limit the scope of the present invention. These embodiments can also be implemented in various other ways, and various omissions, substitutions, and changes can be made without departing from the technical concept of the present invention.

Claims

1. A prediction method for multi-data value collaboration in manufacturing industry, characterized in that: The method includes: Constructing a multivariate time series prediction model, wherein the input of the model is a time series sequence of multivariate variables in a historical period, and the output of the model is a predicted value of the multivariate variables at the time to be predicted, wherein the multivariate variables include a target variable and an environmental variable; Training multivariate time series forecasting models; Use the trained multivariate time series prediction model to predict the data of environmental variables and target variables in the manufacturing production scenario, where the environmental variables are the temperature, humidity or pressure on the production line, and the target variable is the yield or yield of the product; The multivariate time series prediction model includes: Embedding module: transforms the target variable and environmental variable in the time series of multivariate variables in the input historical period into high-dimensional vectors using two different embedding operations; Encoding module: Encode the transformed high-dimensional vectors of the target variable and the environment variable to obtain the target variable hidden state matrix H1 and the environment variable hidden state matrix H2; Attention module: Based on the target variable hidden state matrix H1 and the environment variable hidden state matrix H2, the attention mechanism is used to perform data fusion processing to obtain nonlinear fusion data; Autoregression module: Perform linear autoregression processing on the time series of multivariate variables in the input historical period to obtain linear autoregression data; Aggregation output module: Aggregate nonlinear fusion data and linear autoregressive data to obtain prediction results; The encoding module includes a long short-term memory network; Assume that the target variable hidden state matrix H1 is an m1*n dimensional matrix, and the environment variable hidden state matrix H2 is an m2*n dimensional matrix, where n is the total number of sampling moments of the input time series, and m1 and m2 are constants; The execution process of the attention module includes: Take each row vector of H1 as a query, calculate the dot product of each row vector of H1 and the row vector of H2 as the attention weight of each row vector of H2, use the attention weight to weight the sum of the row vectors of H2 to get the context vector, and concatenate all the context vectors to H1 along the feature dimension. The concatenated matrix is ​​recorded as H3; Take the column vector h3 at the end of H3 as the query, calculate the dot product of all column vectors in H3 as the attention weight of each column vector of H3, and use the attention weight to find the weighted sum of all column vectors of H3 to get a context vector c3; Take the column vector h2 at the end of H2 as the query, calculate the dot product of all column vectors in H2 as the attention weight of each column vector of H2, and use the attention weight to find the weighted sum of all column vectors of H2 to get a context vector c2; Concatenate h3 and c3 along the feature dimension to obtain a nonlinear fusion vector X1, and concatenate h2 and c2 along the feature dimension to obtain a nonlinear fusion vector X2; The execution process of the aggregation output module includes: The linear autoregressive data processed by the autoregressive module is recorded as c1. c1 is k dimensional vector, each dimension corresponds to a dimension of a multivariate variable; The nonlinear fusion vector X1 and the nonlinear fusion vector X2 are linearly transformed and concatenated to form k dimensional vector r1; Add c1 and r1 to get the prediction results of the multivariate variable.

2. The method for predicting the value of multi-data collaboration in manufacturing industry according to claim 1 is characterized in that: The multivariate time series prediction model is trained using supervised learning.

3. The method for predicting the value of multi-data collaboration in manufacturing industry according to claim 2 is characterized in that: During the training process, the mean square error is used to calculate the loss between the network prediction results and the actual output, and the network parameters are updated through gradient back propagation.

4. The method for predicting the value of multi-data collaboration in manufacturing industry according to claim 2 is characterized in that: After a round of training, the prediction results are verified using the validation set. If the loss on the validation set is less than the current minimum loss, the model parameters are temporarily stored; training continues until the loss value converges.

5. A prediction device for multi-data value collaboration in manufacturing industry, characterized in that: The method comprises a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to implement the prediction method according to any one of claims 1 to 4 when executing the computer program.

6. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the prediction method according to any one of claims 1 to 4 is implemented.