Industrial time series prediction method based on adaptive continuous learning

Abstract

The invention discloses an industrial time sequence prediction method based on adaptive continuous learning. The method comprises the following steps: firstly, dividing a non-stationary industrial time series data set to obtain a plurality of domains with the maximum distribution difference; different time domains are then modeled in sequence, and an improved empirical playback (DER + +) method is used to avoid catastrophic forgetting of previously accumulated knowledge. Meanwhile, a soft sample buffer area is introduced to promote memory and learning of key modes in the current field. And finally, the time-sensitive activation function TimeRelu enables the time convolutional network (TCN) to have a time evolution property, and the generalization ability of the prediction model is enhanced. According to the method, the continuous learning normal form is introduced into the time sequence prediction task, the limitations of huge resource overhead of traditional cumulative training, disastrous forgetting of an incremental learning mode and the like are overcome, and the method has theoretical and practical significance on industrial time sequence prediction.

Description

Technical Field

[0001] The present invention belongs to the technical field of industrial time series prediction, and in particular to an industrial time series prediction method based on adaptive continuous learning. Background Art

[0002] With the rapid development of modern industrial information technology, industrial processes generate massive amounts of real-time data. This data contains rich information related to production efficiency and product quality. Analyzing, modeling, and predicting industrial time series data can effectively uncover patterns in industrial production conditions and optimize production processes. Currently, deep learning has achieved high prediction accuracy in certain industrial applications due to the powerful feature extraction and nonlinear fitting capabilities of neural networks. However, a major challenge facing process industries is that manufacturing processes are dynamic, complex, and susceptible to external environmental interference. This can lead to data distribution drift, resulting in reduced predictive performance for models trained on historical data. Traditional approaches retrain models from scratch using all data, significantly increasing data storage, computing resources, and time costs. Alternatively, fine-tuning historical models using only data from new conditions is a lightweight and efficient incremental update approach, but it suffers from the problem of catastrophic forgetting of knowledge learned from historical data. Therefore, there is an urgent need to find a solution for effectively training neural networks and updating models on non-stationary industrial data. Summary of the Invention

[0003] The present invention addresses the problem that the distribution drift phenomenon that is prevalent in industrial time series causes existing methods to perform poorly. It provides an industrial time series prediction method based on adaptive continuous learning. The method first divides the non-stationary industrial time series dataset into several domains with the largest distribution differences. Then, different time domains are modeled sequentially, and the improved experience replay (DER++) method is used to avoid catastrophic forgetting of previously accumulated knowledge. At the same time, a soft sample buffer is introduced to promote the memory and learning of key patterns in the current domain. Finally, the time-sensitive activation function TimeRelu gives the temporal convolutional network (TCN) the property of time evolution, enhancing the generalization ability of the prediction model.

[0004] The technical solution adopted by the present invention to achieve the above-mentioned purpose is:

[0005] An industrial time series prediction method based on adaptive continuous learning is implemented by performing the following steps to obtain an industrial time series prediction model, train industrial time series data, predict key data of production and processing products, and visualize the data. The method specifically includes:

[0006] Step 1: Collect industrial time series data and perform preprocessing;

[0007] Step 2: Combined with prior knowledge, the time series distribution representation (TDC) is used to divide the industrial time series data into K most dissimilar subsequences;

[0008] Step 3: Process the subsequences obtained above by combining the variable values ​​at m historical moments and the predicted target value t seconds later into a sample data set, where m is the number of input historical time points and t is the predicted time point. Each subsample data set is then divided into a training data set, a validation data set, and a prediction data set in proportion, and then normalized.

[0009] Step 4: Establish a continuous learning framework and use the improved experience replay DER++ to constrain the loss optimization of the current domain; select consistent samples based on the prediction error and put them into the soft sample buffer for joint training with the next batch of samples;

[0010] Step 5: Use the universal time series representation framework TS2Vec, including the input projection layer, timestamp mask, and dilated convolution; set the model structure parameters; improve the activation function of the stacked dilated convolution module to TimeRelu, selectively activate the corresponding feature representation according to different time domains, so that the network weights of the temporal convolutional network (TCN) evolve dynamically over time, improving the model's generalization ability;

[0011] Step 6: Sequentially train the deep neural network using the training dataset of each subsequence to obtain the corresponding weights and biases. The obtained weights and biases are then introduced into the validation dataset of the subsequence. The prediction error of the validation dataset is then calculated, and the weights and biases that minimize the prediction error of the validation set are saved. The model is then tested on the test dataset of each subsequence and the results are saved. The AdamW optimizer is used, with the mean squared error as the loss function, and the prediction model parameters are adjusted by backpropagation.

[0012] Step 7: Substitute the weights and biases obtained from the K-1 segment subsequences before sequential training into the prediction model, use the K-th segment test data set to make predictions, and obtain the predicted value of the model on the unknown distribution.

[0013] The industrial time series data are attribute parameters of objects to be processed in the industrial production process and parameter process data of production equipment.

[0014] The pretreatment comprises the following steps:

[0015] Step 1-1, remove outliers;

[0016] Step 1-2: Filter the industrial time series.

[0017] The time series distribution characterization (TDC) is based on the maximum entropy principle and maximizes the use of shared knowledge in time series data by finding the time series periods that are most dissimilar to each other. It includes the following steps:

[0018] Divide the time series into n parts evenly, where each part is the minimum unit period;

[0019] Define the set {q1, ..., q K}, a given K value is given to iterate from the initial value to the given value, and in each iteration round, each time series period of the input sequence is further selected based on the greedy strategy; the further selection of each time series period of the input sequence is as follows: between the start point and the end point of the time series, one splitting point is selected from the candidate splitting points by maximizing the distribution distance to subdivide the data segment, and then a splitting point is further selected for the current smallest data segment until all K data segments are selected to represent the start point and end point of the time subsequence with the least similar distribution;

[0020] In the set, the K value that maximizes the average distribution distance of the K most dissimilar subsequences is the optimal K value and is output.

[0021] The improved experience replay DER++ method effectively alleviates the catastrophic forgetting problem of continuous learning by maintaining a replay buffer, including:

[0022] Step 4-1. Save the original network response:

[0023] First, in the process of optimizing the loss for the current domain, the network is encouraged to imitate its original response to past samples to preserve prior knowledge, that is, to optimize the following objectives:

[0024]

[0025] Among them, t c is the current task, is the optimal parameter set at the end of task t, and α is a hyperparameter that balances past tasks with the current task;

[0026] Due to the previous task D t , t∈{1,…,t c -1} training data is not available, so a replay buffer M is introduced. t To save past experience of task t and keep the logits output on the network training trajectory in a buffer To converge to a flatter minimum, the optimization objective is as follows:

[0027]

[0028] Step 4-2, save the true value of the sample:

[0029] The true value y of the training sample is retained in the buffer to reduce the impact of the replay sample of highly biased logits caused by sudden distribution changes when the task switches. Ultimately, the optimization goal is as follows:

[0030]

[0031] Among them, the task samples in the buffer M are sampled with the same probability, and α and β are hyperparameters that balance the constraints of the current task on the past network response and the true value of the sample;

[0032] Step 4-3: Select the current batch of samples with smaller loss and put them into the soft sample buffer for joint training with the next batch of samples.

[0033] The step 5 comprises:

[0034] Step 5-1. Determine the model input dimension: batchsize×m×n features , batchsize is the batch size set during model batch training, m is the number of input historical time points, n features is the number of features at each moment of input;

[0035] Step 5-2: Determine the structural parameters of the prediction model based on the model input dimensions;

[0036] Step 5-3: Concatenate the data point x with its associated time t to obtain the enhanced input feature vector [x, t];

[0037] Step 5-4: Define the time-related ReLU function as TimeRelu; it is used to extract the time features that can distinguish different tasks from the timestamp t separately, and selectively activate the corresponding feature representations, so that the network weights evolve dynamically over time;

[0038]

[0039] in, and Is a parameter A neural network is used to control the slope and threshold; if and TimeRelu degenerates into the traditional ReLU activation function;

[0040] Use these TimeRelu units to replace all ReLU units in the temporal convolutional network; each TimeRelu unit in the network has a separate set of parameters Parameterize.

[0041] The prediction model training and calculation process includes the following steps:

[0042] Step 6-1. Determine the prediction model training hyperparameters: learning rate λ, maximum number of iterations I MAX , the number of past samples in the buffer I, the capacity of the soft sample buffer J, and the hyperparameters α and β that balance the current task and the past tasks; randomly initialize the network parameters θ and set the initial number of iterations F = 0;

[0043] Step 6-2, sequential training task t, t∈1,…,T, for the sub-dataset Sampling a batch of samples During the training process, the previous batch of samples of the current task stored in the soft sample buffer is first taken out for joint training;

[0044] Step 6-3: Sample K past task samples from the replay buffer with equal probability to constrain the loss optimization process of the current task;

[0045] Step 6-4: replay the new buffer with the current batch of samples and update the soft sample buffer according to the error consistency of the current batch of samples;

[0046] Step 6-5: After training the current task t, return to the optimal network parameters θ obtained for the current task and replay the buffer Soft sample buffer Save the test errors on all tasks and repeat steps 6-3 to 6-5 for subsequent tasks.

[0047] The final prediction model is tested on an unknown distribution and the predicted values ​​are output to verify the generalization ability of the model trained by the adaptive continuous learning method.

[0048] An industrial time series prediction device based on adaptive continuous learning includes: data acquisition equipment and test equipment in the industrial production process, an interface front end, and a back end. The back end is provided with a memory and a processor. The processor stores a program. When the processor loads the program, the method steps described above are executed to obtain a trained and optimized prediction model, so that the model predicts the industrial time series data, obtains the predicted value of the corresponding industrial data, and visually displays it to the user interface front end of the industrial scene.

[0049] The data acquisition equipment and testing equipment in the industrial production process include: equipment for controlling production process parameters, sensors, control valves, and equipment for measuring attribute parameters for measuring product quality.

[0050] The present invention has the following beneficial effects and advantages:

[0051] An industrial time series forecasting method based on adaptive continuous learning provides a solution for effectively training neural networks and updating forecasting models on non-stationary industrial time series data, overcoming the shortcomings of cumulative training and incremental learning approaches. This method focuses on adaptive continuous learning of industrial process time series. It first partitions the non-stationary industrial time series dataset into several domains with the largest distribution differences. It then sequentially models different time domains, using a modified experience replay (DER++) method to prevent catastrophic forgetting of previously accumulated knowledge. Furthermore, a soft sample buffer is introduced to facilitate the memorization and learning of key patterns in the current domain. Finally, the time-sensitive activation function TimeRelu imbues the temporal convolutional network (TCN) with temporal evolutionary properties, enhancing the generalization capability of the forecasting model. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 Schematic diagram of an industrial example of the method of the present invention.

[0053] Figure 2 Flow chart of the method of the present invention.

[0054] Figure 3 This is a continuous learning framework diagram of the present invention.

[0055] Figure 4 This is a network structure diagram of the present invention.

[0056] Figure 5 Comparison chart of prediction errors of different methods on various tasks.

[0057] Figure 6 (a) and (b) are the prediction results of the final prediction model on the base task and the unknown task, respectively. DETAILED DESCRIPTION

[0058] To make the above-mentioned objects, features, and advantages of the present invention more readily apparent, the specific implementation methods of the present invention are described in detail below with reference to the accompanying drawings. The following description sets forth many specific details to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways than those described herein, and those skilled in the art may make similar modifications without departing from the scope of the invention. Therefore, the present invention is not limited to the specific implementation methods disclosed below.

[0059] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art of the art to which the present invention pertains. The terms used in the specification of the invention herein are for the purpose of describing specific embodiments only and are not intended to limit the present invention.

[0060] like Figure 1-4Shown are an industrial example schematic diagram, a method flow chart, a continuous learning framework diagram, and a network structure diagram of the present invention.

[0061] An industrial time series prediction method based on adaptive continuous learning focuses on the adaptive continuous learning of industrial process time series. It first combines prior knowledge and uses time series distribution representation (TDC) to divide the time series data into domains; then sequentially models different domains and uses the improved experience replay DER++ method to avoid catastrophic forgetting of previously accumulated knowledge; at the same time, a soft sample buffer is introduced to promote the memory and learning of key patterns in the current time domain; finally, in terms of network structure, the time-sensitive activation function TimeRelu enables the feature weights of the temporal convolutional network (TCN) to evolve dynamically over time, thereby improving the generalization ability of the prediction model. The programming language used in the program execution step of the present invention is not limited to MATLAB, Python, etc.

[0062] The specific steps of the present invention are as follows:

[0063] Step 1. Collect industrial time series. Taking the actual grinding and grading process as an example, the ore enters the ball mill through the conveyor belt, and a certain amount of grinding media and cyclone water are added at the same time. The formed slurry will be discharged into the pump pool through the discharge port of the ball mill. The slurry enters the equipment through the feed port of the hydrocyclone. Under the action of centrifugal force, large-particle minerals settle to the bottom of the cyclone and return to the ball mill in the form of sediment. Small-particle minerals are brought to the top of the cyclone and discharged through the overflow pipe for subsequent sorting process. The key parameters in this process mainly include 8 variables: ore storage time, cyclone sedimentation water, cyclone feed flow, cyclone feed concentration, cyclone feed pressure, ball mill current, cyclone pump box water feed flow and cyclone feed pump box liquid level. The key parameter to be predicted is the cyclone feed concentration; and the collected variable data are preprocessed;

[0064] Step 1-1: Remove outliers according to the set error range;

[0065] Step 1-2: Filter the industrial time series; the filtering method uses, for example: wavelet threshold denoising.

[0066] Step 2: Combined with prior knowledge, the time series distribution representation (TDC) is used to divide the non-stationary industrial time series data into K most dissimilar subsequences and assign domain labels.

[0067] Step 2-1: To efficiently calculate and avoid trivial solutions, first divide the time series into n parts on average, where each part is considered as the smallest unit period that cannot be further divided. Here, n = 10 is taken as an example.

[0068] Step 2-2: Randomly search for the value of K in the set {2, 3, 4, 5, 6, 7, 8, 9, 10}.

[0069] Step 2-3: Given K, select each period of the input sequence based on a greedy strategy. First, consider A and B as the start and end points of the time series, respectively. Select one split point (denoted as C) from the nine candidate split points by maximizing the distribution distance. Where d is a distance metric function, such as Euclidean or edit distance, and S AC and S CB They represent the period from A to C and the period from C to B respectively.

[0070] Steps 2-4: After determining C, consider using the same strategy to select another point D. Applying a similar strategy to different values ​​of K can obtain the starting and ending points of the K-segment most dissimilar subsequences.

[0071] In step 2-5, the K value that maximizes the average distribution distance of the K most dissimilar subsequences in the set {2, 3, 4, 5, 6, 7, 8, 9, 10} is the optimal K value, where K=4.

[0072] Step 3: Input the subsequences obtained above to obtain sample data sets for the four task domains. Then, divide the training data set, validation data set, and test data set for each domain into 70%, 10%, and 20% ratios. Then, perform the normalization operation. The formula is as follows:

[0073]

[0074] Among them, mean(x i ) and std(x i ) represent the variables x i The mean and variance of .

[0075] Step 4: Establish a continuous learning framework. Because DER++ is a replay-based method, it does not require storing old models or modifying the network structure. It only needs to maintain a replay buffer, effectively alleviating the catastrophic forgetting problem of continuous learning. Improved experience replay DER++ is used to constrain the loss optimization of the current domain, encouraging the network to imitate its original response to past samples. The soft sample buffer selects and stores consistent samples based on prediction error and jointly trains them with the next batch of samples to better learn the current temporal pattern.

[0076] include:

[0077] Step 4-1. Save the original network response. In order to retain previous knowledge, the network is encouraged to imitate its original response to past samples during the loss optimization process for the current domain, that is, to optimize the following objectives:

[0078]

[0079] Among them, t c is the current task, is the mean square error loss, Denotes the sub-dataset D from task t t The expectation of drawing a sample, D KL is the Kullback-Leibler divergence, which is used to measure the difference between two probability distributions. is the optimal parameter set at the end of task t, is the predicted value of the sample based on the past optimal model, f θ (x) is the predicted value of the sample on the current model? α is a hyperparameter that balances the past task and the current task. t , t∈{1,…,t c -1} training data is not available, so a replay buffer M is introduced. t To save past experience of task t and keep the logits output on the network training trajectory in a buffer To converge to a flatter minimum, the optimization objective is as follows:

[0080]

[0081] in, Represents the replay buffer M at the past task t t The expectation obtained by sampling the sample (x,z).

[0082] Step 4-2: Save the true value of the sample. The true value y of the training sample is retained in the buffer to reduce the impact of the replay sample of highly biased logits caused by sudden distribution changes when switching tasks. Ultimately, the optimization goal is as follows:

[0083]

[0084] Among them, the task samples in the replay buffer M are sampled with the same probability, and α and β are hyperparameters that balance the constraints of the current task on the past network response and the true value of the sample.

[0085] Step 4-3: Select the current batch of samples with smaller loss and put them into the soft sample buffer, and conduct joint training with the next batch of samples to better estimate the current task and promote the memory and learning of key patterns.

[0086] Step 5. Use the general time series representation framework TS2Vec to model the time series prediction task. It consists of three parts: an input projection layer, a timestamp mask, and a dilated convolution. The input projection layer is used to map the input feature dimension to the latent dimension. The timestamp mask then masks the latent vector after the input projection layer along the time axis, but it does not change the size of the time series. It improves the robustness of the learned representation by forcing each timestamp to reconstruct itself in a different context. Finally, many dilated convolution blocks are stacked to extract the features of the time series. For the stacked dilated convolution modules, their activation functions are improved. TimeRelu selectively activates the corresponding feature representation according to different time domains, so that the network weights evolve dynamically over time.

[0087] Step 5-1. Determine the model input dimension: batchsize×m×n features , batchsize is the batch size set during model batch training, m is the number of input historical time points, n features is the number of features at each moment of input;

[0088] Step 5-2: Determine the structural parameters of the prediction model based on the model input dimensions;

[0089] Step 5-3: Concatenate the data point x and its associated time t to obtain the enhanced input feature vector [x, t] to better capture the complex patterns of industrial time series data;

[0090] Step 5-4: Extract the time features that can distinguish different tasks from the timestamp t separately, and selectively activate the corresponding feature representations so that the network weights evolve dynamically over time. This time-related ReLU function is named TimeRelu, and the formula is as follows:

[0091]

[0092] Among them, τ t is the time feature extracted from timestamp t, is a neural network, used to control the slope of the activation function, and It is also a neural network used to control the threshold of the activation function. and Is a parameter A neural network is used to control the slope and threshold; if and TimeRelu degenerates into the traditional ReLU activation function; use these TimeRelu units to replace all ReLU units in the temporal convolutional network. Each TimeRelu unit in the network consists of a separate set of parameters. Parameterize.

[0093] Step 6: Sequentially train the deep neural network using the training dataset of each subsequence to obtain the corresponding weights and biases. The obtained weights and biases are then introduced into the validation dataset of the subsequence. The prediction error of the validation dataset is then calculated, and the weights and biases that minimize the prediction error of the validation set are saved. The model is then tested on the test dataset of each subsequence and the results are saved. The AdamW optimizer is used, with the mean squared error as the loss function, and the prediction model parameters are adjusted by backpropagation.

[0094] Step 6-1. Determine the prediction model training hyperparameters: learning rate λ, maximum number of iterations I MAX , the number of past samples in the buffer I, the capacity of the soft sample buffer J, and the hyperparameters α and β that balance the current task and the past tasks; randomly initialize the network parameters θ and set the initial number of iterations F = 0;

[0095] Step 6-2, sequential training task t, t∈1,…,T, for the sub-dataset Sampling a batch of samples During the training process, the previous batch of samples of the current task stored in the soft sample buffer is first taken out for joint training:

[0096] Get samples from the soft sample buffer:

[0097]

[0098] Merge the features of soft samples and current batch samples:

[0099] x all =x′+x t (7)

[0100] Merge the soft samples with the labels of the current batch of samples:

[0101] y all =y′+y t (8)

[0102] Among them, x t 、y t They are the features and labels of the current task samples, x′ and y′ are the features and labels of the soft samples, respectively. all 、y all are the features and labels after the soft sample and the current task sample are merged, represents the soft sample buffer, Indicates from The batch samples sampled in , “←” indicates assignment;

[0103] Step 6-3: Sample I past task samples from the buffer with equal probability to constrain the loss optimization process of the current task, as shown in the following formula:

[0104] Sample I playback samples from the replay buffer:

[0105]

[0106] The constraints imposed by the replay sample on the current model:

[0107]

[0108] Parameter update process with constraints:

[0109]

[0110] in, Represents the replay sample buffer, Indicates from The playback samples sampled in ,f θ Represents the current neural network model, θ is the parameter of the neural network; sample() is sampling, represents the two-norm, Represents the gradient derivation operation.

[0111] Step 6-4: update the replay buffer with the current batch of samples and update the soft sample buffer according to the error consistency of the current batch of samples;

[0112] Update the current batch of samples To replay buffer

[0113]

[0114] Calculate the loss error of the current batch of samples and sort them:

[0115]

[0116] Take the first J samples with smaller errors and put them into the soft sample buffer

[0117]

[0118] Update soft sample buffer

[0119]

[0120] Among them, UpdateMemory is the update buffer, loss is the loss error calculated for the sample, and sort is the sorting operation.

[0121] Step 6-5: After training the current task t, return to the optimal network parameters θ obtained for the current task and replay the buffer Soft sample buffer Save the test errors on all tasks and repeat steps 6-3 to 6-5 for subsequent tasks.

[0122] Step 7: The final prediction model is tested on the unknown distribution of the last subsequence, and the predicted value is output to verify the generalization ability of the model trained by the adaptive continuous learning method.

[0123] The results of the above method are as follows Figure 5 、 6 shown. Figure 5 Comparison of the prediction error of the final model trained with different methods on each task and the average error on all tasks. Figure 6 (a) is a comparison chart of the predicted values ​​and actual results of the models obtained by different methods on the basic task. Figure 6 (b) Comparison of the predicted values ​​and actual results of the models obtained by different methods for an unknown task. This shows that the adaptive continuous learning method can flexibly cope with fluctuations in operating conditions during the grinding and classification process, improving the model's adaptability to various operating conditions and effectively addressing the performance degradation of the prediction model caused by data drift in non-stationary environments. Furthermore, the superior performance under unknown conditions demonstrates the strong and robust generalization capability of the model trained using this method.

[0124] In summary, the present invention establishes an industrial time series prediction method based on adaptive continuous learning, which divides non-stationary industrial time series data into several domains with the largest distribution differences between each other through time series distribution representation TDC. Then, different time domains are modeled sequentially, and the improved experience replay (DER++) method is used to avoid catastrophic forgetting of previously accumulated knowledge. At the same time, a soft sample buffer is introduced to promote the memory and learning of key patterns in the current domain. Finally, the time-sensitive activation function TimeRelu enables the temporal convolutional network to have a time evolution property, thereby enhancing the generalization ability of the prediction model. The present invention introduces the continuous learning paradigm into the time series prediction task, overcomes the limitations of the huge resource overhead of traditional cumulative training and the catastrophic forgetting of the incremental learning method, and effectively solves the problem of decreased accuracy of the prediction model in a non-stationary industrial environment. It has theoretical and practical significance for the field of industrial time series prediction.

[0125] The present invention also provides an industrial time series prediction device based on adaptive continuous learning, comprising: data acquisition equipment and testing equipment for the industrial production process, an interface front-end, and a back-end. The back-end includes a memory and a processor, and the processor stores a program. When the processor loads the program, it executes the method steps described above to obtain a trained and optimized prediction model, which predicts industrial time series data, obtains predicted values ​​for the corresponding industrial data, and visually displays them to the user interface front-end of the industrial scenario. The data acquisition equipment and testing equipment for the industrial production process include: equipment for controlling production process parameters, sensors, control valves, and equipment for measuring attribute parameters that measure product quality.

[0126] The embodiments described above will help those skilled in the art further understand the present invention, but are not intended to limit the present invention in any way. It should be noted that a person skilled in the art may make various modifications and improvements without departing from the scope of the present invention. Such modifications and improvements are all within the scope of protection of the present invention.

Claims

1. An industrial time series forecasting method based on adaptive continuous learning, characterized by: Perform the following steps to obtain an industrial time series prediction model, train the model on industrial time series data, predict key data of the production and processing target products, and visualize the data. The method specifically includes: Step 1: Collect industrial time series data and perform preprocessing; Step 2: Combined with prior knowledge, the time series distribution representation (TDC) is used to divide the industrial time series data into K most dissimilar subsequences; Step 3: Process the subsequences obtained above by combining the variable values ​​at m historical moments and the predicted target value t seconds later into a sample data set, where m is the number of input historical time points and t is the predicted time point. Each subsample data set is then divided into a training data set, a validation data set, and a prediction data set in proportion, and then normalized. Step 4: Establish a continuous learning framework and use the improved experience replay DER++ to constrain the loss optimization of the current domain; select consistent samples based on the prediction error and put them into the soft sample buffer for joint training with the next batch of samples; Step 5: Use the universal time series representation framework TS2Vec, including the input projection layer, timestamp mask, and dilated convolution; set the model structure parameters; improve the activation function of the stacked dilated convolution module to TimeRelu, selectively activate the corresponding feature representation according to different time domains, so that the network weights of the temporal convolutional network (TCN) evolve dynamically over time, improving the model's generalization ability; Step 6: Sequentially train the deep neural network using the training dataset of each subsequence to obtain the corresponding weights and biases. The obtained weights and biases are then introduced into the validation dataset of the subsequence. The prediction error of the validation dataset is then calculated, and the weights and biases that minimize the prediction error of the validation set are saved. The model is then tested on the test dataset of each subsequence and the results are saved. The AdamW optimizer is used, with the mean squared error as the loss function, and the prediction model parameters are adjusted by backpropagation. Step 7: Substitute the weights and biases obtained from the K-1 segment subsequences before sequential training into the prediction model, use the K-th segment test data set to make predictions, and obtain the predicted value of the model on the unknown distribution.

2. The industrial time series prediction method based on adaptive continuous learning according to claim 1 is characterized in that: The industrial time series data are attribute parameters of objects to be processed in the industrial production process and parameter process data of production equipment.

3. The industrial time series prediction method based on adaptive continuous learning according to claim 1 is characterized in that: The pretreatment comprises the following steps: Step 1-1, remove outliers; Step 1-2: Filter the industrial time series.

4. The industrial time series prediction method based on adaptive continuous learning according to claim 1 is characterized in that: The time series distribution characterization (TDC) is based on the maximum entropy principle and maximizes the use of shared knowledge in time series data by finding the time series periods that are most dissimilar to each other. It includes the following steps: Divide the time series into n parts evenly, where each part is the minimum unit period; Define the set {q1, ..., q K }, a given K value is given to iterate from the initial value to the given value, and in each iteration round, each time series period of the input sequence is further selected based on the greedy strategy; the further selection of each time series period of the input sequence is as follows: between the start point and the end point of the time series, one splitting point is selected from the candidate splitting points by maximizing the distribution distance to subdivide the data segment, and then a splitting point is further selected for the current smallest data segment until all K data segments are selected to represent the start point and end point of the time subsequence with the least similar distribution; In the set, the K value that maximizes the average distribution distance of the K most dissimilar subsequences is the optimal K value and is output.

5. The industrial time series prediction method based on adaptive continuous learning according to claim 1 is characterized in that: The improved experience replay DER++ method effectively alleviates the catastrophic forgetting problem of continuous learning by maintaining a replay buffer, including: Step 4-1. Save the original network response: First, in the process of optimizing the loss for the current domain, the network is encouraged to imitate its original response to past samples to preserve prior knowledge, that is, to optimize the following objectives: Among them, t c is the current task, is the optimal parameter set at the end of task t, and α is a hyperparameter that balances past tasks with the current task; Due to the previous task D t , t∈{1,…,t c -1} training data is not available, so a replay buffer M is introduced. t To save past experience of task t and keep the logits output on the network training trajectory in a buffer To converge to a flatter minimum, the optimization objective is as follows: Step 4-2, save the true value of the sample: The true value y of the training sample is retained in the buffer to reduce the impact of the replay sample of highly biased logits caused by sudden distribution changes when the task switches. Ultimately, the optimization goal is as follows: Among them, the task samples in the buffer M are sampled with the same probability, and α and β are hyperparameters that balance the constraints of the current task on the past network response and the true value of the sample; Step 4-3: Select the current batch of samples with smaller loss and put them into the soft sample buffer for joint training with the next batch of samples.

6. The industrial time series prediction method based on adaptive continuous learning according to claim 1, It is characterized in that The step 5 comprises: Step 5-1. Determine the model input dimension: batchsize×m×n features , batchsize is the batch size set during model batch training, m is the number of input historical time points, n features is the number of features at each moment of input; Step 5-2: Determine the structural parameters of the prediction model based on the model input dimensions; Step 5-3: Concatenate the data point x with its associated time t to obtain the enhanced input feature vector [x, t]; Step 5-4: Define the time-related ReLU function as TimeRelu; it is used to extract the time features that can distinguish different tasks from the timestamp t separately, and selectively activate the corresponding feature representations, so that the network weights evolve dynamically over time; in, and Is a parameter A neural network is used to control the slope and threshold; if and TimeRelu degenerates into the traditional ReLU activation function; Use these TimeRelu units to replace all ReLU units in the temporal convolutional network; each TimeRelu unit in the network has a separate set of parameters Parameterize.

7. The industrial time series prediction method based on adaptive continuous learning according to claim 1 is characterized in that: The prediction model training and calculation process includes the following steps: Step 6-1. Determine the prediction model training hyperparameters: learning rate λ, maximum number of iterations I MAX , the number of past samples in the buffer I, the capacity of the soft sample buffer J, and the hyperparameters α and β that balance the current task and the past tasks; randomly initialize the network parameters θ and set the initial number of iterations F = 0; Step 6-2, sequential training task t, t∈1,…,T, for the sub-dataset Sampling a batch of samples During the training process, the previous batch of samples of the current task stored in the soft sample buffer is first taken out for joint training; Step 6-3: Sample I past task samples from the replay buffer with equal probability to constrain the loss optimization process of the current task; Step 6-4: replay the new buffer with the current batch of samples and update the soft sample buffer according to the error consistency of the current batch of samples; Step 6-5: After training the current task t, return to the optimal network parameters θ obtained for the current task and replay the buffer Soft sample buffer Save the test errors on all tasks and repeat steps 6-3 to 6-5 for subsequent tasks.

8. The industrial time series prediction method based on adaptive continuous learning according to claim 1 is characterized in that: The final prediction model is tested on an unknown distribution and the predicted values ​​are output to verify the generalization ability of the model trained by the adaptive continuous learning method.

9. An industrial time series prediction device based on adaptive continuous learning, characterized in that: include: Data acquisition equipment and test equipment, interface front-end and back-end in the industrial production process, the back-end is provided with a memory and a processor, and the processor stores a program. When the processor loads the program, the method steps described in any one of claims 1 to 8 are executed to obtain a trained and optimized prediction model, so that the model predicts industrial time series data, obtains the predicted value of the corresponding industrial data, and visually displays it to the user interface front-end of the industrial scene.

10. The industrial time series forecasting method based on adaptive continuous learning according to claim 1, characterized in that: The data acquisition equipment and testing equipment in the industrial production process include: equipment for controlling production process parameters, sensors, control valves, and equipment for measuring attribute parameters for measuring product quality.

Images