Production quality prediction method, device, equipment, medium and program for production equipment

Abstract

This disclosure relates to the field of forecasting technology, and in particular to a method, apparatus, equipment, medium, and program for predicting the production quality of production equipment. The method includes: acquiring a first historical multivariate time series, which is a multivariate time series used for prediction, and includes the variable to be predicted and its influencing factor variables; generating fitted values ​​of the first historical multivariate time series using a vector autoregressive moving average model; generating a time vector of the first historical multivariate time series based on its time information; and generating predicted values ​​of the variable to be predicted using a neural network model based on the fitted values ​​and the time vector of the first historical multivariate time series. Implementing the technical solution of this disclosure can improve the accuracy of variable prediction.

Description

Technical Field

[0001] This disclosure relates to the field of prediction technology, and in particular to a method, apparatus, equipment, medium, and program for predicting the production quality of production equipment. Background Technology

[0002] Currently, the prediction of production quality variables during equipment manufacturing primarily relies on human experience. Operators typically predict production quality variables based on their long-accumulated subjective judgment in order to assess the future production status of the equipment.

[0003] However, subjective judgment methods rely heavily on the operator's personal skills and experience, lacking systematicity and consistency, which leads to inaccurate prediction results. Summary of the Invention

[0004] This disclosure provides a method, apparatus, equipment, medium, and program for predicting the production quality of production equipment, thereby improving the accuracy of variable prediction.

[0005] In a first aspect, this disclosure provides a method for predicting the production quality of production equipment, comprising: obtaining a first historical multivariate time series, wherein the first historical multivariate time series is a multivariate time series for prediction, the multivariate time series includes a variable to be predicted related to production quality and its influencing factor variables, the variable to be predicted includes equipment production parameters to be predicted, and the influencing factor variables include at least one influencing factor affecting the equipment production parameters;

[0006] The fitted values ​​of the first historical multivariate time series are generated using a vector autoregressive moving average model.

[0007] Based on the time information of the first historical multivariate time series, a time vector of the first historical multivariate time series is generated;

[0008] Using a neural network model, the predicted value of the variable to be predicted is generated based on the fitted value of the first historical multivariate time series and the time vector of the first historical multivariate time series.

[0009] In some embodiments, the variable to be predicted includes equipment production parameters to be predicted, and the influencing factor variable includes at least one influencing factor that affects the equipment production parameters.

[0010] In some embodiments, the equipment production parameters include the amount of alumina, and the influencing factor variables include the content of a target chemical substance in the air, wherein the target chemical substance includes at least one of carbon monoxide, tin oxide, benzene, titanium dioxide, total nitrogen oxides, tungsten oxide, nitrogen dioxide, and indium oxide.

[0011] In some embodiments, the vector autoregressive moving average model includes an autoregressive model and a moving average model.

[0012] In some embodiments, before generating fitted values ​​for the first historical multivariate time series using a vector autoregressive moving average model, the method includes:

[0013] Determine the difference fraction of the vector autoregressive moving average model;

[0014] Determine the autoregressive order and the moving average order of the vector autoregressive moving average model;

[0015] The vector autoregressive moving average model is trained using a second historical multivariate time series, wherein the second historical multivariate time series is the multivariate time series used for training.

[0016] In some embodiments, before generating the predicted value of the variable to be predicted based on the fitted value of the first historical multivariate time series and the time vector of the first historical multivariate time series using a neural network model, the method further includes:

[0017] Obtain the fitted value of the second historical multivariate time series, wherein the second historical multivariate time series is the multivariate time series used for training, and the fitted value of the second historical multivariate time series is generated based on the second historical multivariate time series using the vector autoregressive moving average model;

[0018] Obtain the time vector of the second historical multivariate time series, which is generated based on the time information of the second historical multivariate time series.

[0019] The neural network model is trained based on the fitted values ​​of the second historical multivariate time series and the time vector of the second historical multivariate time series.

[0020] In some embodiments, generating the time vector based on the time information of the first historical multivariate time series includes:

[0021] Based on the time information of the first historical multivariate time series, the time vector is generated using Time2Vec.

[0022] Secondly, this disclosure provides a production quality prediction device for production equipment, comprising:

[0023] The acquisition module is used to acquire a first historical multivariate time series, which is a multivariate time series for prediction, and the multivariate time series includes the variable to be predicted and its influencing factor variables.

[0024] The first generation module is used to generate fitted values ​​for the first historical multivariate time series using a vector autoregressive moving average model.

[0025] The second generation module is used to generate a time vector of the first historical multivariate time series based on the time information of the first historical multivariate time series.

[0026] The prediction module is used to generate predicted values ​​for the variable to be predicted based on the fitted values ​​of the first historical multivariate time series and the time vector of the first historical multivariate time series using a neural network model.

[0027] Thirdly, this disclosure provides a computer device including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described in the foregoing aspects.

[0028] Fourthly, this disclosure provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the methods described in the above aspects.

[0029] Fifthly, this disclosure provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the methods described above.

[0030] This disclosure provides a prediction method, apparatus, device, storage medium, and computer program that generates fitted values ​​of a first historical multivariate time series using a vector autoregressive moving average model. The first historical multivariate time series includes a variable to be predicted related to production quality and its influencing factor variables. A time vector of the first historical multivariate time series is generated based on its time information. A neural network model is then used to predict the variable to be predicted based on the fitted values ​​and the time vector of the first historical multivariate time series. This method utilizes the vector autoregressive moving average model to capture the linear characteristics of the time series, captures periodic and trend changes through the step of generating the time vector, and simultaneously uses a neural network to handle nonlinear characteristics. This comprehensive approach captures the relationship between the variable to be predicted and its influencing factor variables, resulting in more accurate predictions.

[0031] 1. Technical Features: "Equipment production parameters include alumina content, and influencing factor variables include the content of target chemical substances in the air. Target chemical substances include at least one of carbon monoxide, tin oxide, benzene, titanium dioxide, total nitrogen oxides, tungsten oxide, nitrogen dioxide, and indium oxide." The technical effect is to solve the problem of inaccurate alumina content prediction. By introducing the content of target chemical substances in the air as influencing factor variables, the model can comprehensively capture the impact of chemical substances on alumina content, thereby improving the accuracy of prediction results.

[0032] 2. Technical Features: The "Vector Autoregressive Moving Average Model" includes both autoregressive and moving average models, solving the problems of lag effects and modeling inter-variable relationships in multivariate time series modeling. It effectively captures the mutual influences between multiple time series, improves prediction accuracy, and provides decision-makers with more information and insights.

[0033] 3. Technical Features: "Determine the difference of the vector autoregressive moving average model; determine the autoregressive order and moving average order of the vector autoregressive moving average model; obtain the fitted values ​​of the second historical multivariate time series, and train the vector autoregressive moving average model using the second historical multivariate time series, where the second historical multivariate time series is the multivariate time series used for training." This solves the problems of insufficient handling of multivariate time series stationarity and insufficient model parameter selection in the training of the vector autoregressive moving average model, improves the accuracy of the fitted values, and provides a reliable foundation for subsequent prediction.

[0034] 4. Technical Features: "Based on the time information of the first historical multivariate time series, time vectors are generated using Time2Vec." This method solves the problem of insufficient expression of time information in time series modeling. By generating time vectors through Time2Vec, the model's ability to capture time features is enhanced, and the prediction accuracy is improved.

[0035] 5. Technical feature: "Training a neural network model based on the fitted values ​​of the second historical multivariate time series and the time vector of the second historical multivariate time series" solves the problem of insufficient utilization of the linear features and time information of multivariate time series in the training of neural network models, and significantly improves the prediction accuracy and generalization ability of the model for complex time series. Attached Figure Description

[0036] The present disclosure will be described in more detail below based on embodiments and with reference to the accompanying drawings:

[0037] Figure 1 This is a flowchart illustrating a production quality prediction method for production equipment provided in an embodiment of this disclosure.

[0038] Figure 2 This is a schematic diagram illustrating the training process of the vector autoregressive moving average model in a production quality prediction method for production equipment provided in this embodiment of the present disclosure.

[0039] Figure 3 This is a schematic diagram of the training process of a neural network model in a production quality prediction method for production equipment provided in an embodiment of this disclosure.

[0040] Figure 4 A comparison chart of model prediction performance evaluation in a production quality prediction method for production equipment provided in this disclosure embodiment;

[0041] Figure 5 A schematic diagram of the T2V-LSTM model structure in a production quality prediction method for production equipment provided in an embodiment of this disclosure;

[0042] Figure 6 A schematic diagram of the T2V-GRU model structure in a production quality prediction method for production equipment provided in an embodiment of this disclosure;

[0043] Figure 7 A schematic diagram of the VARMA-T2V-TCN model structure and prediction process in a production quality prediction method for production equipment provided in this embodiment of the present disclosure;

[0044] Figure 8 This is a schematic diagram of a production quality prediction device for a production equipment provided in an embodiment of this disclosure.

[0045] In the accompanying drawings, the same parts are referred to by the same reference numerals, and the drawings are not drawn to scale. Detailed Implementation

[0046] To enable those skilled in the art to better understand the technical solutions of this disclosure, and to fully understand and implement the process of how this disclosure applies technical means to solve technical problems and achieve corresponding technical effects, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, not all embodiments. The embodiments of this disclosure and the various features within them can be combined with each other without conflict, and the resulting technical solutions are all within the protection scope of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort should fall within the protection scope of this disclosure.

[0047] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0048] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0049] Example 1

[0050] Figure 1 This is a schematic flowchart illustrating a production quality prediction method for production equipment provided in an embodiment of this disclosure. Figure 1 As shown, a method for predicting the production quality of production equipment includes:

[0051] S101, Obtain the first historical multivariate time series. The first historical multivariate time series is a multivariate time series used for prediction. The multivariate time series includes the variable to be predicted related to production quality and its influencing factor variable. The variable to be predicted includes the equipment production parameters to be predicted, and the influencing factor variable includes at least one influencing factor that affects the equipment production parameters.

[0052] The prediction method disclosed herein can be applied to prediction scenarios of industrial equipment production data, such as the prediction of alumina content, and can also be applied to other applicable prediction scenarios.

[0053] In this embodiment of the disclosure, the first historical multivariate time series is a multivariate time series used for prediction. This multivariate time series includes a variable to be predicted related to production quality and its influencing factor variables. The variable to be predicted is the variable that the prediction method of this embodiment will predict, and the influencing factor variables are variables that affect the variable to be predicted. The variable to be predicted includes equipment production parameters to be predicted, such as the amount of alumina.

[0054] In this embodiment of the disclosure, there may be one or more variables to be predicted. There may also be one or more influencing factor variables corresponding to the variables to be predicted. It is understood that when there are multiple variables to be predicted, one variable can also become an influencing factor variable of another variable.

[0055] Taking the amount of alumina as an example, the influencing factors can be the content of chemical substances in the air such as carbon monoxide, tin oxide, benzene, titanium dioxide, total nitrogen oxides, tungsten oxide, nitrogen dioxide and indium oxide. It can be the content of one of the chemical substances, or the content of multiple or all of them.

[0056] The first historical multivariate time series obtained in step S101 of this embodiment is used as data for prediction and is used to generate predicted values ​​of the variables to be predicted in subsequent steps.

[0057] S102 uses a vector autoregressive moving average model to generate fitted values ​​for the first historical multivariate time series.

[0058] The Vector Autoregressive Moving Average (VARMA) model is used to fit the linear correlation features between time series to generate fitted values ​​for the time series. In this embodiment of the disclosure, the VARMA model is used to fit the linear correlation features between a first historical multivariate time series to generate fitted values ​​for the first historical multivariate time series.

[0059] In one example, such as Figure 2 As shown, the vector autoregressive moving average model includes an autoregressive model and a moving average model. Before generating the fitted values ​​of the first historical multivariate time series using the vector autoregressive moving average model, the method includes:

[0060] S201, determine the difference of the vector autoregressive moving average model.

[0061] S202, determine the autoregressive order and moving average order of the vector autoregressive moving average model.

[0062] S203, using the second historical multivariate time series, trains a vector autoregressive moving average model, where the second historical multivariate time series is the multivariate time series used for training.

[0063] In step S102 of this embodiment, a fitted value of the first historical multivariate time series is generated using a vector autoregressive moving average model. This fitted value can be used as input to the encoder of the neural network model for multivariate advance prediction. By combining the linear modeling capability of the vector autoregressive moving average model with the nonlinear modeling capability of the neural network, the overall prediction accuracy can be improved.

[0064] In one example, the VARMA model consists of two parts: an autoregressive model (AR) and a moving average model (MA). Their lag values ​​are different. The AR model is represented by p, which represents the number of lagged observations in the autoregression; the MA model is represented by q, which represents the size of the moving average window. The parameters of the VARMA model can be defined using the Box-Jenkins method. The formula for the ARMA(p,q) model is shown in (1):

[0065]

[0066] y in the formula t The prediction result is represented in k×1 vector form, y t-1 c represents the value of the variable at time t-1, and c0 represents a constant in the form of a k×1 vector. The parameters of the autoregressive (AR) model are in k×k vector form, θ1, ..., θ2. q μ represents the parameters of the moving average (MA) model in k×k vector form. t μ t-1 Let μ represent the error term of the MA model, where μ t This represents independent, identically distributed white noise with zero mean and constant variance.

[0067] Similar to the Autoregressive Moving Average (ARMA) model, the VARMA model fits the linear correlation between time series values. The VARMA model equation increases with the number of variables in the time series. Assuming there are three time series y1, y2, and y3, to calculate y... 1，t (The value of y1 at time t), the equation of the VARMA(1,1) model with three variables is shown in formula (2), where the first subscript of the parameter represents the corresponding variable, such as subscript 1 representing the first variable among the three variables, subscript 2 representing the second variable among the three variables, and subscript 3 representing the third variable among the three variables.

[0068]

[0069] S103, Based on the time information of the first historical multivariate time series, generate the time vector of the first historical multivariate time series.

[0070] In step S103 of this embodiment, the time vector of the first historical multivariate time series is a representation of the time information (such as timestamps, dates, hours, etc.) in the first historical multivariate time series converted into a numerical vector that the model can process. Its purpose is to capture the periodicity and / or trend in the time information, enabling the time information to be effectively utilized by the neural network model.

[0071] In one example, time vectors are generated using Time2Vec based on the time information of a first historical multivariate time series. Time2Vec is a model-independent time vector representation method that aims to provide a representation of time in the form of vector embeddings. In cases involving events occurring synchronously and asynchronously, most neural network models typically assume that the input is synchronous. When time is known as a relevant feature, it is often used as another dimension of input, which often prevents neural network models from effectively utilizing time as a feature. Time2Vec relates to a time decomposition technique that encodes a time signal into a set of frequencies, and then uses learned frequencies to replace a fixed set of frequencies in the Fourier transform. Time2Vec transforms time itself and feeds this transformation into the model that uses the time information to perform tasks other than regression. In designing the time representation, Time2Vec identified three important properties: 1) capturing periodicity and aperiodicity, which distinguishes time from other time features that require better processing; 2) time rescaling invariance, used to eliminate the impact of time being measured in different units (days, hours, seconds, etc.) on the model's prediction process; and 3) universality, the Time2Vec representation method is easily used by different models and architectures.

[0072] The Time2Vec time representation is shown in formula (3):

[0073]

[0074] Where t2v(τ)[i] is the i-th element of t2v(τ), k is the Time2Vec dimension, and w i , It is a set of learnable parameters, where τ is the original time series. It is a periodic activation function. Considering that the time vectors for different tasks should have universality, the algorithm will use... Set to a sine function so that the selected algorithm can capture periodic behavior in the data. When When, for 1≤i≤k, w i , These are the frequency and phase shift of a sine function. Therefore, a sine function helps capture periodic behavior without feature engineering. Meanwhile, the linear term represents the progression of time and can be used to capture time-dependent non-periodic patterns in the input.

[0075] S104. Using a neural network model, based on the fitted values ​​of the first historical multivariate time series and the time vector of the first historical multivariate time series, the predicted values ​​of the variable to be predicted are generated.

[0076] In step S104 of this embodiment, a neural network model is used to generate predicted values ​​for the variable to be predicted based on the fitted values ​​and time vectors of the first historical multivariate time series. This neural network model is used to generate predicted values ​​for the variable to be predicted based on the fitted values ​​and time vectors of the first historical multivariate time series. Specifically, the fitted values ​​of the first historical multivariate time series generated using a VARMA model can be input into the neural network model (its autoencoder) for multivariate advance prediction. The neural network model can learn from two different but similar data sources. This neural network model can be any applicable model, such as a Long Short-Term Memory Network (LSTM) model, a Gated Recurrent Unit Network (GRU) model, or a Temporal Convolutional Network (TCN) model. This neural network model is trained based on the fitted values ​​and time vectors of the second historical multivariate time series, which is the multivariate time series used for training. During neural network model training, it is possible to determine whether to input other data based on the specific requirements of the network structure used by the neural network model, such as whether to input predicted value labels.

[0077] In one example, see Figure 3 Before generating predicted values ​​for the variable to be predicted based on the fitted values ​​of the first historical multivariate time series and the time vector of the first historical multivariate time series using a neural network model, the method also includes:

[0078] S301, Obtain the fitted value of the second historical multivariate time series, wherein the second historical multivariate time series is a multivariate time series used for training, and the fitted value of the second historical multivariate time series is generated based on the second historical multivariate time series using a vector autoregressive moving average model.

[0079] S302, Obtain the time vector of the second historical multivariate time series. The time vector of the second historical multivariate time series is generated based on the time information of the second historical multivariate time series.

[0080] S303, based on the fitted values ​​of the second historical multivariate time series and the time vector of the second historical multivariate time series, train the neural network model.

[0081] By using the fitted values ​​and time vectors of the second historical multivariate time series, a neural network model can be trained. This allows the neural network model to generate accurate thresholds for the variables to be predicted using the fitted values ​​and time vectors of the multivariate time series. Before training the neural network model using the fitted values ​​and time vectors of the second historical multivariate time series, hyperparameter optimization can be performed on the neural network model.

[0082] Example 2

[0083] Based on the above embodiments, in order to evaluate the difference between the predicted results and the actual values, this study uses three evaluation indicators—root mean square error (RMSE), mean absolute error (MAE), and mean absolute percentage error (MAPE)—to evaluate the prediction accuracy. Given a time series y of actual values ​​of length T. t and predicted value time series RMSE, MAE, and MAPE are calculated as shown in formulas (4), (5), and (6):

[0084]

[0085]

[0086]

[0087] The Time2Vec representation can be used in different neural network architectures. The output dimension of the Time2Vec layer in the algorithm is the specified hidden dimension (1≤i≤k), which is the sine curve learned from the network plus the linear representation of the input (i=0). Data processed using Time2Vec has been applied to LSTM, GRU, and TCN models. Using Time2Vec technology has improved the prediction accuracy of neural network models, achieving an accuracy improvement of approximately 10.3% compared to the traditional prediction model VARMA and approximately 17.5% compared to the ARIMA model.

[0088] Taking the prediction of alumina content as an example, considering other parameters as external factors affecting alumina content, the data includes eight other related parameters. The results of VARMA fitting are used as an enhanced data source and input into the T2V-NN model to predict alumina content. The results are shown in Table 1:

[0089] Table 1 Comparison of performance under deep learning models

[0090] Model MAE RMSE MAPE (%) LSTM 191.124 230.414 3.356 VARMA-LSTM 191.121 230.411 3.356 VARMA-T2V-LSTM 185.057 224.966 3.248 GRU 225.170 271.945 4.018 VARMA-GRU 224.415 271.083 4.004 VARMA-T2V-GRU 196.164 247.204 3.368 TCN 205.463 255.554 3.586 VARMA-TCN 201.759 239.576 3.511 VARMA-T2V-TCN 193.456 231.123 3.441

[0091] The results show that adding neural network model processing improved the prediction performance of all three T2V-NN neural network models. Baseline prediction was used to verify whether the prediction results consisted of simple repeating values. If the root mean square error of the verification model was higher than that of the baseline prediction, the prediction model should be repaired or abandoned.

[0092] The improved deep learning model was compared, and the improvement effect of the multivariate time series algorithm was verified using the air quality dataset from the UCI Machine Learning Repository. The dataset contains 9358 hourly average response instances from five metal oxide chemical sensor arrays embedded in an air quality chemical multi-sensor device. During training, temperature (°C), relative humidity (%), and absolute humidity were added as additional relevant variables, and hyperparameters were optimized for the neural network model structure. The accuracy of hourly average concentration predictions for CO (carbon monoxide), PT08.S1 (tin oxide), C6H6 (benzene), PT08.S2 (titanium dioxide), NOx (total nitrogen oxides), PT08.S3 (tungsten oxide, nominal NOx target), NO2 (nitrogen dioxide), PT08.S4 (tungsten oxide, nominal NO2 target), and PT08.S5 (indium oxide, nominal O3 target) was verified.

[0093] Figure 4 Baseline prediction is used to verify whether the prediction results consist of simple repeating values. If the model's performance is lower than the baseline, the prediction model should be repaired. The figure shows that parameters PT08.S3 and PT08.S4 are not suitable for use with the three model prediction methods (BASELINE has the lowest RMSE). For other parameters, in most cases, the prediction accuracy of LSTM, VARMA-LSTM, and VARMA-T2V-LSTM models gradually improves.

[0094] Example 3

[0095] Based on the above embodiments, this embodiment provides an application example.

[0096] The VARMA model consists of two parts: an autoregressive (AR) model and a moving average (MA) model, with different lag values. The AR model is denoted by p, representing the number of lagged observations in the autoregression; the MA model is denoted by q, representing the size of the moving average window. The Box Jenkins method is an efficient way to define the model parameter values ​​p and q, based on the stationarity of the sequence to ensure accuracy and reliability. The method consists of four steps: identification, estimation, diagnostic checks, and prediction. In the identification stage, to obtain more accurate results, multiple ARMA models with different parameters are selected in the algorithm, focusing on the autocorrelation function (ACF) and partial autocorrelation function (PACF) respectively. In the estimation stage, a nonlinear optimization process is used to estimate the parameters of the provisional model to minimize the overall error or maximize the likelihood function. In the diagnostic checks stage, the normality and sufficiency of the model residuals are checked. Finally, the selected model is used to predict future values. The formula for the ARMA(p,q) model is shown below:

[0097]

[0098] y in the formula t The prediction result is represented in k×1 vector form, y t-1 c represents the value of the variable at time t-1, and c0 represents a constant in the form of a k×1 vector. The parameters of the autoregressive (AR) model are in k×k vector form, θ1, ..., θ2. q μ represents the parameters of the moving average (MA) model in k×k vector form. t μ t-1 Let μ represent the error term of the MA model, where μ t This represents independent, identically distributed white noise with zero mean and constant variance.

[0099] Similar to ARMA, VARMA is also limited to fitting the linear correlation between time series values. The VARMA model equation increases with the number of variables in the time series. For example, given three time series y1, y2, and y3, to calculate y... 1，t The equations for the VARMA(1,1) model with three variables (y1 at time t) are shown below.

[0100]

[0101] Time2Vec is a model-independent time vector representation method that aims to provide a representation of time in the form of vector embeddings. In events involving synchronous and asynchronous occurrences, most neural network models typically assume synchronous inputs. When time is known as a relevant feature, it is often used as another dimension of input, which frequently prevents neural network models from effectively utilizing time as a feature.

[0102] Time2Vec relates to a time decomposition technique that encodes a time signal into a set of frequencies, and then uses learned frequencies to replace the fixed set of frequencies in the Fourier transform. Time2Vec transforms time itself and feeds this transformation into the model that uses the time information to perform tasks beyond regression. In designing its time representation, Time2Vec identified three important properties: 1) capturing periodicity and aperiodicity, distinguishing time from other time features that require better processing; 2) time rescaling invariance, used to eliminate the impact of time being measured in different units (days, hours, seconds, etc.) on the model's prediction process; and 3) universality, the Time2Vec representation is easily used by different models and architectures. The Time2Vec time representation formula is shown below:

[0103]

[0104] Where t2v(τ)[i] is the i-th element of t2v(τ), k is the Time2Vec dimension, and τ is the original time series. It is a periodic activation function, w i , It is a set of learnable parameters. Considering that the time vectors for different tasks should have universality, in the algorithm, [the following is used]: Set to a sine function so that the selected algorithm can capture periodic behavior in the data. When When, for 1≤i≤k, w i , These are the frequency and phase shift of a sine function. Therefore, a sine function helps capture periodic behavior without feature engineering. Meanwhile, the linear term represents the progression of time and can be used to capture time-dependent non-periodic patterns in the input.

[0105] Multivariate time series consist of multiple variables, each influenced not only by past values ​​but also by other variables. These variables exhibit linear relationships and fixed dependencies, thus classical statistical forecasting methods can be used to predict future values. However, the choice of lag time often carries human bias and subjective judgment. In real-world time series scenarios, traditional forecasting methods are limited by irregular time structures, missing values, heavy noise, and complex variable relationships. To address these limitations, neural networks can effectively eliminate these constraints. They possess good robustness, can automatically extract feature information from the original input data, and can be widely applied to time series forecasting problems.

[0106] The fitted values ​​generated by VARMA are input into a neural network autoencoder for multivariate advance prediction. The neural network can learn from two different but similar data sources. The VARMA-NN model combines the advantages of both to obtain linear and nonlinear relationships in sequences, while compensating for the subjectivity of feature extraction. It also leverages the characteristics of neural networks to improve the processing efficiency of long-span memory units and enhance prediction accuracy.

[0107] The T2V-LSTM model structure is as follows: Figure 5 As shown, the T2V-GRU model structure is as follows: Figure 6 As shown in Figure C. t-1 Indicates the state of the previous model block, a t Let σ represent the hidden layer state, and tanh represent the activation functions.

[0108] The structure and prediction process of VARMA-T2V-TCN are as follows: Figure 7 As shown in the diagram, first, the difference d of the VARMA model is determined to ensure the stationarity of the sequence. Next, the parameter values ​​of p and q of the VARMA model are determined. Based on the length k of the sequence to be predicted in the training set, the VARMA model is trained, and the fitting result is used as the input to the TCN model. The TCN model, after being adjusted by Time2Vec and hyperparameters, is then input, and the trained VARMA-T2V-TCN model is returned. Finally, the test set is predicted, and the results are verified. In the example diagram, the TCN model has 2 hidden layers, an inflation factor of d, and a kernel size of f = 3. In the algorithm, hyperparameters such as time block (bn), kernel size (f), and number of kernels (kn) are determined based on the input sequence.

[0109] During multi-step training, to avoid catastrophic forgetting, the network structure needs to be appropriately adjusted to provide better prediction accuracy. Deep learning networks are highly sensitive to hyperparameter settings; when hyperparameters are set incorrectly, the predicted output will produce high-frequency oscillations. Important hyperparameters during neural network model training include the number of hidden units in recurrent layers, the dropout value, and the learning rate; these hyperparameters can significantly affect model performance. Adjusting hyperparameters can optimize the learning process and yield better results for more complex neural network structures.

[0110] To ensure the accuracy of the VARMA model parameters and the stationarity of the series, the non-stationary series needs to be differencing, followed by determining the order of the autocorrelation model and the moving average model. If the parameters fail the ADF (Augmented Dickey-Fuller) test, it indicates that the correlated time series does not meet the stationarity requirement, and the model needs to be first-order differencing. The differencing results must pass the ADF test and the Johansen cointegration test to prove the stationarity of the data and that the parameters have a long-term, statistically significant relationship, preventing spurious regression. Subsequently, to find the optimal model for the data, the VARMA(p, q) and ARIMA(p, d, q) models are validated using the AIC information criterion, and the model parameters with the smallest return value are selected.

[0111] This disclosure discloses a time-series-based prediction method. To obtain nonlinear and hidden features in the time series of multivariate parameters in industrial equipment, a VARMA model is used to fit the multivariate parameter features. The Time2Vec vector embedding time form is combined as an enhanced data source for the neural network to automatically create feature engineering and generalize deep learning techniques. The results show that the model's performance is significantly improved after adjusting the network structure, proving its practicality in engineering applications.

[0112] Example 4

[0113] Based on the above embodiments, see Figure 8 A production quality prediction device for production equipment, comprising:

[0114] The acquisition module 801 is used to acquire a first historical multivariate time series. The first historical multivariate time series is a multivariate time series for prediction. The multivariate time series includes a variable to be predicted related to production quality and its influencing factor variables. The variable to be predicted includes the equipment production parameters to be predicted, and the influencing factor variables include at least one influencing factor that affects the equipment production parameters.

[0115] The first generation module 802 is used to generate fitted values ​​for the first historical multivariate time series using a vector autoregressive moving average model.

[0116] The second generation module 803 is used to generate a time vector of the first historical multivariate time series based on the time information of the first historical multivariate time series.

[0117] The prediction module 804 is used to generate predicted values ​​for the variable to be predicted by using a neural network model based on the fitted values ​​of the first historical multivariate time series and the time vector of the first historical multivariate time series.

[0118] In one example, the equipment production parameters include the amount of alumina, and the influencing factor variables include the content of target chemicals in the air, which include at least one of carbon monoxide, tin oxide, benzene, titanium dioxide, total nitrogen oxides, tungsten oxide, nitrogen dioxide, and indium oxide.

[0119] In one example, the vector autoregressive moving average model includes both an autoregressive model and a moving average model;

[0120] The device also includes a first trained model for:

[0121] Determine the difference fraction of the vector autoregressive moving average model;

[0122] Determine the autoregressive order and the moving average order of the vector autoregressive moving average model;

[0123] A vector autoregressive moving average model is trained using the second historical multivariate time series, where the second historical multivariate time series is the multivariate time series used for training.

[0124] In one example, a time vector is generated based on the time information of the first historical multivariate time series, including:

[0125] Based on the time information of the first historical multivariate time series, a time vector is generated using Time2Vec.

[0126] In one example, the device also includes a second trained model for:

[0127] Before generating predicted values ​​for the variable to be predicted based on the fitted values ​​of the first historical multivariate time series and the time vector of the first historical multivariate time series using a neural network model, the method also includes:

[0128] Obtain the fitted value of the second historical multivariate time series, where the second historical multivariate time series is the multivariate time series used for training, and the fitted value of the second historical multivariate time series is generated based on the second historical multivariate time series using a vector autoregressive moving average model.

[0129] Obtain the time vector of the second historical multivariate time series, which is generated based on the time information of the second historical multivariate time series.

[0130] A neural network model is trained based on the fitted values ​​of the second historical multivariate time series and the time vector of the second historical multivariate time series.

[0131] Example 5

[0132] Based on the above embodiments, this embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described in the above embodiments.

[0133] In some embodiments of this example, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the methods described in the above embodiments.

[0134] In some embodiments of this example, a computer program product is provided, including a computer program / instruction that, when executed by a processor, implements the steps of the methods described in the above embodiments.

[0135] The processor may include, but is not limited to, one or more processors or microprocessors. Each processor may be implemented as an Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), controller, microcontroller, microprocessor, or other electronic component, for executing the methods in the above embodiments.

[0136] Computer-readable storage media can be implemented by any type of volatile or non-volatile storage device or a combination thereof. Computer-readable storage media may include, but are not limited to, random access memory (RAM), read-only memory (ROM), flash memory, EPROM memory, EEPROM memory, registers, and computer storage media (e.g., hard disks, floppy disks, solid-state drives, removable disks, CD-ROMs, DVD-ROMs, Blu-ray discs, etc.).

[0137] Computer-readable storage media may also store at least one computer-executable program / instruction, such as computer-readable instructions. Computer-readable storage media include, but are not limited to, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Computer-readable storage media may include, for example, read-only memory (ROM), hard disk, flash memory, etc. For example, a non-transitory computer-readable storage medium may be connected to a computing device such as a computer, and then, when the computing device executes the computer-readable instructions stored on the computer-readable storage medium, the various methods described above can be performed.

[0138] In addition, the computer device may include (but is not limited to) a data bus, an input / output (I / O) bus, a display, and input / output devices (e.g., keyboard, mouse, speakers, etc.).

[0139] The processor can communicate with external devices via the I / O bus through wired or wireless networks.

[0140] In one embodiment, the at least one computer-executable instruction may also be compiled into or comprise a software product / computer program product, wherein one or more computer-executable instructions are executed by a processor to perform the steps of the various functions and / or methods in the embodiments described herein.

[0141] In the embodiments provided in this disclosure, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0142] It should be noted that, in this disclosure, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element limited by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0143] While the embodiments disclosed herein are as described above, the foregoing content is merely for the purpose of facilitating understanding of this disclosure and is not intended to limit this disclosure. Any person skilled in the art to which this disclosure pertains may make any modifications and changes in form and detail of the implementation without departing from the spirit and scope of this disclosure; however, the scope of patent protection of this disclosure shall still be determined by the scope defined in the appended claims.

Claims

1. A production quality prediction method of a production device, characterized by, include: Obtain a first historical multivariate time series, which is a multivariate time series for prediction. The multivariate time series includes a variable to be predicted related to production quality and its influencing factor variables. The variable to be predicted includes equipment production parameters to be predicted, and the influencing factor variables include at least one influencing factor that affects the equipment production parameters. The fitted values ​​of the first historical multivariate time series are generated using a vector autoregressive moving average model. Based on the time information of the first historical multivariate time series, a time vector of the first historical multivariate time series is generated; Using a neural network model, the predicted value of the variable to be predicted is generated based on the fitted value of the first historical multivariate time series and the time vector of the first historical multivariate time series.

2. The method of claim 1, wherein, The equipment production parameters include the amount of alumina, and the influencing factor variables include the content of target chemical substances in the air. The target chemical substances include at least one of carbon monoxide, tin oxide, benzene, titanium dioxide, total nitrogen oxides, tungsten oxide, nitrogen dioxide, and indium oxide.

3. The method of claim 1, wherein, The vector autoregressive moving average model includes an autoregressive model and a moving average model.

4. The method according to claim 3, characterized in that, Before generating the fitted values ​​of the first historical multivariate time series using the vector autoregressive moving average model, the method includes: Determine the difference fraction of the vector autoregressive moving average model; Determine the autoregressive order and the moving average order of the vector autoregressive moving average model; The vector autoregressive moving average model is trained using a second historical multivariate time series, wherein the second historical multivariate time series is the multivariate time series used for training.

5. The method of claim 1, wherein, The step of generating a time vector based on the time information of the first historical multivariate time series includes: Based on the time information of the first historical multivariate time series, the time vector is generated using Time2Vec.

6. The method according to any one of claims 1 to 5, characterized in that, Before generating the predicted value of the variable to be predicted based on the fitted value of the first historical multivariate time series and the time vector of the first historical multivariate time series using a neural network model, the method further includes: Obtain the fitted value of the second historical multivariate time series, wherein the second historical multivariate time series is the multivariate time series used for training, and the fitted value of the second historical multivariate time series is generated based on the second historical multivariate time series using the vector autoregressive moving average model; Obtain the time vector of the second historical multivariate time series, which is generated based on the time information of the second historical multivariate time series. The neural network model is trained based on the fitted values ​​of the second historical multivariate time series and the time vector of the second historical multivariate time series.

7. A production quality prediction device of a production apparatus, characterized by, include: The acquisition module is used to acquire a first historical multivariate time series, which is a multivariate time series for prediction. The multivariate time series includes a variable to be predicted related to production quality and its influencing factor variables. The variable to be predicted includes equipment production parameters to be predicted, and the influencing factor variables include at least one influencing factor that affects the equipment production parameters. The first generation module is used to generate fitted values ​​for the first historical multivariate time series using a vector autoregressive moving average model. The second generation module is used to generate a time vector of the first historical multivariate time series based on the time information of the first historical multivariate time series. The prediction module is used to generate predicted values ​​for the variable to be predicted based on the fitted values ​​of the first historical multivariate time series and the time vector of the first historical multivariate time series using a neural network model.

8. A computer device comprising a memory, a processor, and a computer program stored on the memory, wherein the computer program comprises instructions that, when executed by the processor, cause the processor to perform the method of any one of claims 1-7. The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product comprising computer programs / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method described in any one of claims 1 to 6.

Images