A Time-Series Regression Prediction Method and System with Uncertainty Estimation

Abstract

The present invention provides a time series regression prediction method and system with uncertainty estimation. The method is based on a deep learning model with an attention mechanism, and can perform time series regression prediction and uncertainty modeling. The system includes: a data acquisition and preprocessing module, a neural network module, a training module, and a testing module. The time series regression prediction method with uncertainty estimation according to the embodiments of the present invention constructs a time series prediction and uncertainty model, extracts features of the input time series based on the attention mechanism, and simultaneously decodes the output time series and its prediction uncertainty, which can effectively improve the practical reference value of the time series prediction result, and has higher execution efficiency and lower time and space costs compared with the existing uncertainty estimation methods. The present invention is also applied to the discharge modeling of the 0-dimensional diagnostic physical quantities of the tokamak, making the modeling result more reliable and practically valuable.

Description

Technical Field

[0001] The present invention belongs to the fields of deep learning and uncertainty measurement, and particularly relates to a time series regression prediction method and system with uncertainty estimation. Background Art

[0002] Deep neural networks are a powerful machine learning tool that can be used to handle time series prediction problems. However, deep neural networks often face many challenges in practical applications, such as insufficient sample size, skewness of dataset distribution, unknown inputs, etc. These problems can lead to uncertainty in the model, which will have an adverse impact on the prediction results of deep neural networks and reduce the accuracy and reliability of their predictions. For example, in the financial field, uncertainty may cause the model to be unable to accurately predict stock price or exchange rate fluctuations, thus bringing losses to investors; in the transportation field, uncertainty may lead to inaccurate traffic predictions, thus bringing inconvenience to traffic planners and passengers; in the field of controlled nuclear fusion, uncertainty may lead to inaccurate discharge predictions, thus affecting the safe and stable operation of the device and even possibly damaging the device.

[0003] Therefore, the uncertainty measurement of deep neural networks is very important. How to effectively measure and manage the uncertainty in deep learning models is a research hotspot in the current deep learning field.

[0004] Searching existing patents reveals that there are almost no uncertainty measurement methods for time series. Most of the existing literature uses methods such as Monte Carlo dropout and model ensembles for uncertainty estimation.

[0005] The disadvantages of existing uncertainty measurement methods mainly include the following aspects:

[0006] 1. High computational cost: These methods require running multiple models or sampling, so the computational cost is relatively high, especially when high-precision predictions or large amounts of data are needed;

[0007] 2. Poor interpretability: The uncertainty measurements generated by these methods are usually difficult to interpret. For application scenarios using these measurements, it is difficult to clearly explain the source of prediction uncertainty, which may not be very suitable for some critical application scenarios, such as controlled nuclear fusion, medical, finance, etc.

[0008] In summary, uncertainty measurement is a very important research field in deep learning. The research results can help us better understand the performance and behavior of deep learning models, thereby improving their effectiveness and reliability in practical applications. However, efficient and low-cost modeling for uncertainty estimation of time series remains a challenge. Summary of the Invention

[0009] To solve the problem that deep neural networks have uncertainties and to overcome the problems of large computational overhead and potential impact on the accuracy of prediction results in existing uncertainty measurement methods, the present invention provides a time series regression prediction method with uncertainty estimation, which uses an attention mechanism to realize the modeling of time series and its prediction uncertainty. At the same time, the model is trained using Tokamak discharge experiment data to realize high-fidelity and fast diagnosis of signal time series and its prediction uncertainty modeling, and to realize the verification of 0-dimensional diagnostic physical quantities in Tokamak experiment proposals.

[0010] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0011] A time series regression prediction method with uncertainty estimation is constructed based on a deep learning model with an attention mechanism, which specifically includes the following steps:

[0012] S1. Data acquisition and preprocessing: Obtain time series data, including input time series and output time series, and perform resampling and standardization preprocessing operations on the data to establish a data set for model training and testing;

[0013] S2. Construct an attention-based time series prediction and uncertainty estimation model: Input the input time series into the attention-based time series prediction and uncertainty estimation model, which will extract features from the input time series and map them to the latent space, and obtain the output time series and its uncertainty through a time series output module and an uncertainty estimation module respectively;

[0014] S3. Train the model: First, use the target output time series and the predicted output of the time series output module to calculate the loss according to the loss function L1, and use the error backpropagation algorithm to optimize the model parameters; then, based on the existing model parameters, calculate the loss between the actual deviation and the predicted output of the uncertainty estimation module according to the loss function L2, and use the error backpropagation algorithm to optimize the parameters of the uncertainty estimation network in the model, and finally obtain the optimal time series prediction regression model with uncertainty estimation;

[0015] S4. Test and verify the effectiveness of the model: Input the test set data into the trained time series prediction and uncertainty model, and output the model prediction time series and its prediction uncertainty.

[0016] Furthermore, the attention-based time series prediction and uncertainty model constructed in step 2 includes a position encoder, an input encoder, a time series output module, a direct uncertainty estimation module, and a linear output layer.

[0017] Position encoder: Add temporal information to the data to help the model learn the relative and absolute position information of the data;

[0018] Input Encoder: Extract features from the input time series, compress it into a semantic vector of a specified length, and map it into the latent space;

[0019] Time Series Output Module: Decode according to the feature vector mapped into the latent space by the input encoder to obtain the global features of the output time series, and obtain the final expression of the output time series through a linear fully connected layer;

[0020] Direct Uncertainty Estimation Module: Decode according to the feature vector mapped into the latent space by the input encoder to obtain the global features of the time series uncertainty, and obtain the final expression of the uncertainty time series through a linear fully connected layer.

[0021] Linear Output Layer: Perform a simple linear mapping on the outputs of the time series output module and the direct uncertainty estimation module to the target output dimension.

[0022] Furthermore, the loss functions L1 and L2 use a masking mechanism to calculate the effective mean square error loss and weighted mean square error loss according to the effective length of the time series.

[0023] Furthermore, the position encoder encodes the position information of the original tensor using a periodic function to obtain a time series tensor, and combines the time series tensor with the original input tensor, enabling the model to have the ability to learn time series information.

[0024] Furthermore, the direct uncertainty estimation module includes an uncertainty estimation decoder and a linear output layer, where the uncertainty estimation decoder is used to decode the global features of the time series uncertainty according to the mapping of the input features in the latent space; the linear output layer maps it into the uncertainty output vector space of the sample by adjusting the weights of the uncertainty global features.

[0025] Furthermore, the uncertainty estimation decoder includes a multi-head attention mechanism and a fully connected neural network. The multi-head attention mechanism establishes multiple attentions through a linear layer. Each attention focuses on different parts of the input information, and then they are concatenated, which can enhance the expression ability of the model; the fully connected neural network is a series connection of multiple linear layers and the activation function Relu. Through the composite mapping of simple linear and non-linear processing units, relatively complex non-linear processing capabilities can be obtained.

[0026] On the other hand, the present invention applies for a time series regression prediction system with uncertainty estimation, constructs a time series prediction and uncertainty model based on the attention mechanism, and specifically includes the following:

[0027] Data acquisition and preprocessing module: used to acquire time series data, including input time series and output time series, and perform resampling and normalization preprocessing operations on the data to establish a data set for model training and testing;

[0028] Neural network module: The neural network includes a position encoder, an input encoder, a time series output module, a direct uncertainty estimation module, and a linear output layer; it is used to obtain the output time series and its uncertainty according to the input time series;

[0029] Training module: used to train the neural network using the training data set of the task to obtain a trained neural network;

[0030] Testing module: used to input the input time series data of the test set into the trained neural network model to obtain the predicted time series and its uncertainty.

[0031] In an embodiment of the present invention, an electronic device is provided, which includes a readable storage medium, a central processing unit, and a graphics processing unit. The computer program that can be run on the central processing unit and the graphics processing unit is stored on the readable storage medium, and when the central processing unit and the graphics processing unit execute the computer program, the steps of the time series prediction method are implemented.

[0032] In an embodiment of the present invention, a readable storage medium is provided, on which a computer program is stored, and when the computer program is run by a processor, the steps of the time series regression prediction model method with uncertainty estimation are executed.

[0033] The beneficial effects of the present invention are:

[0034] 1. This deep learning model based on the attention mechanism constructs a time series regression prediction model with uncertainty estimation. By constructing a time series prediction and uncertainty model, based on the attention mechanism, through feature extraction of the input time series, it innovatively decodes the output time series and its prediction uncertainty at the same time, which can effectively improve the practical reference value of the time series prediction result, and has higher execution efficiency and lower time and space costs compared with the existing uncertainty estimation methods.

[0035] 2. This deep learning model based on the attention mechanism constructs a time series regression prediction model with uncertainty estimation, which is innovatively applied to the discharge modeling of the 0-dimensional diagnostic physical quantities of tokamaks, making the modeling results more reliable and more practical. Description of the Drawings

[0036] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only for the purpose of showing the preferred embodiments and are not considered as a limitation to the present invention.

[0037] Figure 1 It is a schematic flowchart of a time series regression prediction method with uncertainty estimation according to an embodiment of the present invention;

[0038] Figure 2 It is a basic flowchart for obtaining time series data according to an embodiment of the present invention;

[0039] Figure 3 It is an architecture diagram of a neural network model for time series prediction and uncertainty estimation based on attention according to an embodiment of the present invention;

[0040] Figure 4 It is a basic flowchart for training a neural network to obtain a trained neural network according to an embodiment of the present invention;

[0041] Figure 5 It is a schematic diagram of the modeling effect of a time series prediction and uncertainty estimation model based on attention according to an embodiment of the present invention;

[0042] Figure 6 It is a schematic structural diagram of a time series prediction device according to an embodiment of the present invention;

[0043] Figure 7 It is a schematic structural diagram of an electronic device according to an embodiment of the present invention. Detailed implementation manners

[0044] In order to make the objectives, technical solutions and advantages of the present invention more clear and understandable, the present invention will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0045] The present invention uses a time series prediction and uncertainty estimation model based on the attention mechanism to model time series regression problems, enabling the model to output both time series prediction results and their uncertainties simultaneously. Therefore, the main contribution of the present invention is the construction of a time series prediction and uncertainty estimation model based on the attention mechanism. In addition, in the embodiments, discharge modeling and uncertainty estimation modeling are performed on the 0-dimensional diagnostic physical quantities of the tokamak.

[0046] Such as Figure 1As shown in the figure, the present invention provides a time series regression prediction method with uncertainty, comprising the following steps:

[0047] S1. Data acquisition and preprocessing: Acquire time series data, including input time series and output time series, and perform resampling and standardization preprocessing operations on the data to establish a data set for model training and testing;

[0048] S2. Construct an attention-based time series prediction and uncertainty estimation model: Input the input time series into the attention-based time series prediction and uncertainty estimation model, which extracts features from the input time series and maps them to the latent space, and obtains the output time series and its uncertainty through the time series output module and the uncertainty estimation module respectively;

[0049] S3. Train the model: First, use the target output time series and the predicted output of the time series output module to calculate the loss according to the loss function L1, and use the error backpropagation algorithm to optimize the model parameters; then, based on the existing model parameters, calculate the loss between the actual deviation and the predicted output of the uncertainty estimation module according to the loss function L2, and use the error backpropagation algorithm to optimize the parameters of the uncertainty estimation network in the model, and finally obtain the optimal time series prediction regression model with uncertainty estimation;

[0050] S4. Test and verify the effectiveness of the model: Input the test set data into the trained time series prediction and uncertainty model, and output the predicted time series and its predicted uncertainty of the model.

[0051] Specifically, as Figure 2 shown, the data acquisition and preprocessing process described in step S1 includes the following steps:

[0052] S11. Acquire the original time series data. Taking the tokamak dynamics system as an example, the original time series data is read from the MDSplus database and stored in different HDF5 files according to the experimental order for subsequent use;

[0053] S12. Resample the original time series data using a fixed sampling rate to obtain the resampled time series, reducing the computational cost while retaining sufficient data information;

[0054] S13. Standardize the resampled time series data using the Z-Score standardization method, so that the data conforms to the standard normal distribution after processing, that is, the mean is 0 and the standard deviation is 1;

[0055] S14. Divide the preprocessed time series data according to the ratio of 4:4:2 to obtain two training sets and one test set.

[0056] Specifically, as Figure 3 shown, the time series regression prediction model with uncertainty estimation includes a position encoder, an input encoder, a time series output module, a direct uncertainty estimation module, and a linear output layer.

[0057] The position encoder encodes the position information of the original tensor using a periodic function to obtain a time series tensor, and combines the time series tensor with the original input tensor, enabling the model to learn the relative position information and absolute position information of the time series.

[0058] The input encoder extracts features from the input time series, compresses it into a semantic vector of a specified length, and maps it into the latent space.

[0059] The time series output module decodes according to the feature vector mapped into the latent space by the input encoder to obtain the global features of the output diagnostic signal, and obtains the final expression of the diagnostic signal time series through a linear fully connected layer.

[0060] The direct uncertainty estimation module includes an uncertainty estimation decoder and a regression output layer.

[0061] Among them, the uncertainty estimation decoder based on the attention mechanism consists of a multi-head attention mechanism and a fully connected neural network, and is used to decode the global features of the time series uncertainty according to the mapping of the input features in the latent space. The multi-head attention mechanism refers to splitting the input vector into multiple heads, performing attention calculations on each head, and finally concatenating the results of all heads to obtain the final output vector. Through this method, task-related information is extracted, increasing the model's attention to information in different aspects, and improving the generalization ability and effect of the model. Each head uses the following function for calculation,

[0062]

[0063] where Q, K, and V are the inputs of the multi-head attention mechanism, and Q, K, and V are the same in the multi-head self-attention mechanism; Softmax is an activation function that can normalize a numerical vector into a probability distribution vector, and the sum of all probabilities is 1; T represents the transpose operation; d model is the dimension of the position vector, which is the same as the hidden state dimension value of the entire model; h represents the number of heads in the multi-head attention mechanism, i ∈ [1, h]; W i Q 、W i K 、W i V are the weight matrices of Q, K, and V respectively.

[0064] The fully connected neural network uses the following function to perform relatively complex non-linear processing on the output of the attention mechanism.

[0065]

[0066] Among them, x is the input of the fully connected neural network; Relu is the rectified linear unit in the activation function; W1 and W2 are the weight parameters of two linear layers in the fully connected neural network; T represents the transpose operation; b1 and b2 are the bias of two linear layers in the fully connected neural network.

[0067] The linear output layer is a simple one-dimensional linear layer. By adjusting the weights of the uncertainty global features, it maps them into the uncertainty output vector space of the samples, performs simple linear mapping, and aligns the model output tensor and the 0-dimensional diagnostic data tensor.

[0068] Specifically, as Figure 4 shown, the model training process in step S3 includes the following steps:

[0069] S31. Randomly initialize the weight and bias parameters of the neural network;

[0070] S32. Initialize the neural network hyperparameters and the stochastic gradient descent optimizer. The neural network hyperparameters include batch size, learning rate, and number of iterations;

[0071] S33. Use the target output time series and the predicted output of the time series output module, calculate the loss according to the loss function L1, and use the error backpropagation algorithm to optimize the model parameters;

[0072] S34. Based on the existing model parameters, calculate the loss between the actual deviation and the predicted output of the uncertainty estimation module according to the loss function L2, and use the error backpropagation algorithm to optimize the parameters of the uncertainty estimation network in the model;

[0073] The loss functions L1 and L2 use the masking mechanism to calculate the valid mean squared error loss VMSE and the weighted mean squared error loss VWMSE according to the effective length of the time series;

[0074] The calculation formula of the valid mean squared error loss function VMSE is as follows:

[0075]

[0076] Among them, n V is the effective length of the time series; W V is a matrix containing only 0 and 1, used to extract the effective part of the mean squared error for the time series

[0077] ^; y is the real experimental data; y is the model predicted output.

[0078] The calculation formula of the effective weighted mean square error loss VWMSE is as follows:

[0079]

[0080] Where n V_in is the number of time slices that are valid and within the prediction uncertainty coverage; n V_out is the number of time slices that are valid and outside the prediction uncertainty coverage; y V_in is the true experimental data within the prediction uncertainty coverage; is the model prediction output within the prediction uncertainty coverage; y V_out is the true experimental data within the prediction uncertainty coverage; is the model prediction output within the prediction uncertainty coverage; λ is a parameter used to balance the importance of the coverage rate and the uncertainty width; μ is the predefined target uncertainty coverage rate; PUCP is the prediction uncertainty coverage rate, and the calculation formula of the PUCP is as follows:

[0081]

[0082] Where N is the length of the time series, a i is a binary value, and the calculation formula is as follows:

[0083]

[0084] Where y i is the target value, is the point prediction value, and uncertainty is the uncertainty.

[0085] S35. Finally, an optimal time series prediction regression model with uncertainty estimation is obtained.

[0086] The time series prediction method provided in this embodiment can be applied to multiple tasks. Below, we will discuss it by taking the Tokamak discharge modeling task as an example.

[0087] The discharge modeling and uncertainty estimation modeling for the 0 - dimensional diagnostic physical quantities of the tokamak are implemented as follows: The input data for the discharge modeling and uncertainty estimation modeling of the 0 - dimensional diagnostic physical quantities of the tokamak is a time series of 92 control signals, including plasma current feed - forward, poloidal magnetic field coil current, toroidal magnetic field, power of the lower hybrid current drive and heating system, neutral beam injection system, ion cyclotron resonance heating system, electron cyclotron resonance heating / current drive system, gas puffing system, supersonic molecular beam injection, pellet injection system, plasma shape feed - forward; The modeling objective of the discharge modeling and uncertainty estimation modeling for the 0 - dimensional diagnostic physical quantities of the tokamak is 11 diagnostic signals, including the actual plasma current I p , the average electron density n of the plasma at the magnetic axis of the tokamak e , the plasma stored energy W mhd , the toroidal voltage V of the tokamak loop , the normalized magnetic pressure β of the tokamak n , the toroidal magnetic pressure β t , the poloidal magnetic pressure β p , the elongation ratio k, the internal inductance li, the safety factor q0, and the safety factor q on the 95% flux surface 95 . As Figure 2 shown, the sizes of the stacked layers are: D0, D1, and D2 are all 6, and the size d model of the hidden layer is 512.

[0088] On the premise that the set target uncertainty coverage μ is 0.9 and the parameter λ is 4, the results of the discharge modeling and uncertainty estimation modeling for the 0 - dimensional diagnostic physical quantities of the tokamak are as Figure 5 shown. Shot #73873 is selected as the test data, and the discharge time of this shot exceeds 70s, and the sequence length exceeds 7×10 4 . The specific process is as follows: First, set the input quantity of the tokamak control system. In this embodiment, the control system source file corresponding to shot #73873 is directly called, and then the actual actuator input signal of shot #73873 is used as the input of the system. Next, convert the data into the Tensor type and load it into the GPU, and load the trained deep - learning model into the GPU. Use the trained deep - learning model to calculate the input data to obtain the modeling results of 11 0 - dimensional diagnostic physical quantities and their predicted uncertainties, and finally visualize the data.

[0089] In the Tokamak device "EAST" in Hefei, with a total of 932 shots as the test set, the performance of the model on the overall test set was tested. Under the premise that the target uncertainty coverage μ of the present invention is set to 0.9 and λ is set to 4, the average predicted uncertainty coverage rate PUCP of the 0-dimensional diagnostic physical quantity discharge modeling reaches 90.891%, meeting the expectations. The good performance on the test set indicates that the modeling results of the present invention are accurate and reliable and have practical value.

[0090] On the other hand, as Figure 6 shown, a time series regression prediction system with uncertainty estimation according to the present invention, the device includes the following modules:

[0091] Data acquisition and preprocessing module: used to acquire time series data, including input time series and output time series, and perform resampling and standardization preprocessing operations on the data to establish a data set for model training and testing;

[0092] Neural network module: The neural network includes a position encoder, an input encoder, a time series output module, a direct uncertainty estimation module, and a linear output layer; used to obtain the output time series and its uncertainty according to the input time series;

[0093] Training module: used to train the neural network using the training data set of the task to obtain a trained neural network;

[0094] Testing module: used to input the input time series data of the test set into the trained neural network model to obtain the predicted time series and its uncertainty.

[0095] As Figure 7 shown, the present invention also provides an electronic device, including a readable storage medium, a core processor, and a graphics processor. A computer program stored on the readable storage medium and executable on the core processor and the graphics processor, when the core processor and the graphics processor execute the computer program, will implement the steps of a time series regression prediction method with uncertainty estimation described in this embodiment.

[0096] Specifically, the readable storage medium is a general storage medium, such as a mobile disk, a hard disk, an optical disc, etc. A computer program is stored on the storage medium. When the computer program is executed by a processor, it realizes each process of the above-mentioned time series prediction method embodiment with uncertainty estimation and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0097] Although the preferred embodiments of the embodiments of the present application have been described, those skilled in the art can make additional changes and modifications to these embodiments once they learn the basic creative concepts. Therefore, the appended claims are intended to be construed as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

Claims

1. A time series regression prediction method with uncertainty estimation, characterized in that, Construct a time series prediction and uncertainty model based on the attention mechanism, which specifically includes the following steps: S1. Data acquisition and preprocessing: Obtain time series data, including input time series and output time series, and perform resampling, normalization, and preprocessing operations for labeling discharge category labels on the data to establish a data set for model training and testing; S2. Construct an attention-based time series prediction and uncertainty estimation model: Input the input time series into the attention-based time series prediction and uncertainty estimation model, which will extract features from the input time series and map them to the latent space, and obtain the output time series and its uncertainty through the time series output module and the uncertainty estimation module respectively; S3. Train the model: First, use the target output time series and the predicted output of the time series output module to calculate the loss according to the loss function L1, and use the error backpropagation algorithm to optimize the model parameters; then, based on the existing model parameters, calculate the loss between the actual deviation and the predicted output of the uncertainty estimation module according to the loss function L2, and use the error backpropagation algorithm to optimize the parameters of the uncertainty estimation network in the model. Finally, obtain the optimal time series prediction regression model with uncertainty estimation; S4. Test and verify the effectiveness of the model: Input the test set data into the trained time series prediction and uncertainty model, and output the predicted time series and its predicted uncertainty of the model; The time series prediction and uncertainty model constructed based on the attention mechanism in S2 includes: The position encoder will encode the position information of the original tensor using a periodic function to obtain a time series tensor, and combine the time series tensor with the original input tensor, enabling the model to have the ability to learn the relative and absolute position information of the time series; The input encoder extracts features from the input time series, compresses it into a semantic vector of a specified length, and maps it into the latent space; The time series output module decodes according to the feature vector mapped into the latent space by the input encoder to obtain the global features of the output time series, and obtains the final expression of the output time series through a linear fully connected layer; The direct uncertainty estimation module decodes according to the feature vector mapped into the latent space by the input encoder to obtain the global features of the time series uncertainty, and obtains the final expression of the uncertainty time series through a linear fully connected layer; The linear output layer performs a simple linear mapping on the outputs of the time series output module and the direct uncertainty estimation module to the target output dimension; The direct uncertainty estimation module includes an uncertainty estimation decoder and a linear output layer, where the uncertainty estimation decoder is used to decode the global features of the time series uncertainty according to the mapping of the input features in the latent space; the linear output layer maps it to the uncertainty output vector space of the sample by adjusting the weights of the uncertainty global features; The uncertainty estimation decoder consists of a multi-head attention mechanism and a fully-connected neural network. The multi-head attention mechanism uses linear layers to establish multiple attentions, each of which focuses on different parts of the input information and then concatenates them, which can enhance the expression ability of the model. The fully-connected neural network is a series connection of multiple linear layers and the activation function Relu. Through the composite mapping of simple linear and non-linear processing units, relatively complex non-linear processing capabilities can be obtained.

2. The time series regression prediction method with uncertainty estimation according to claim 1, characterized in that The data acquisition and preprocessing process described in S1 includes the following steps: S11. Obtain the original time series data, which is read from the MDSplus database and stored in different HDF5 files according to the experimental order for subsequent use. S12. Resample the original time series data using a fixed sampling rate to obtain the resampled time series, reducing the computational cost while retaining sufficient data information. S13. Standardize the resampled time series data using the Z-Score normalization method, so that the data conforms to the standard normal distribution, that is, the mean is 0 and the standard deviation is 1 after processing. S14. According to the preprocessed time series data above, divide it into two training sets and one test set according to the ratio of 4:4:

2.

3. The time series regression prediction method with uncertainty estimation according to claim 1, characterized in that, In S3, the loss functions L1 and L2 use a masking mechanism to calculate the effective mean square error loss and the weighted mean square error loss according to the effective length of the time series.

4. The time series regression prediction method with uncertainty estimation according to claim 1, wherein The position encoder encodes the position information of the original tensor using a periodic function to obtain a time series tensor, and combines the time series tensor with the original input tensor, enabling the model to have the ability to learn time series information.

5. The time series regression prediction method with uncertainty estimation according to claim 1, characterized in that The position encoder uses sine and cosine functions to add time series information to the original vector to help the model learn the relative and absolute position information of the data.

6. The time series regression prediction method with uncertainty estimation according to claim 1, characterized in that The uncertainty estimation decoder based on the attention mechanism consists of a multi-head attention mechanism and a fully-connected neural network. The multi-head attention mechanism refers to splitting the input vector into multiple heads, calculating the attention for each head, and finally concatenating the results of all heads to obtain the final output vector. Through this method, task-related information is extracted, increasing the model's attention to different aspects of information, and improving the generalization ability and effect of the model. Each head uses the following function for calculation. ， Among them, , , are the inputs of the multi-head attention mechanism. In the multi-head self-attention mechanism, , , are the same; is an activation function that normalizes a numerical vector into a probability distribution vector, and the sum of all probabilities is 1; represents a transpose operation; is the dimension of the position vector, which is the same as the dimension value of the hidden state of the entire model; represents the number of heads in the multi-head attention mechanism, ; , , are respectively , , 's weight matrices.

7. The time series regression prediction method with uncertainty estimation according to claim 6, characterized in that The fully-connected neural network uses the following function to perform relatively complex non-linear processing on the output of the attention mechanism. ， Among them, is the input of the fully connected neural network; is the rectified linear unit in the activation function; , are the weight parameters of two linear layers in the fully connected neural network; represents the transpose operation; , are the biases of two linear layers in the fully connected neural network.

8. The time series regression prediction method with uncertainty estimation according to claim 1, characterized in that The linear output layer is a simple one-dimensional linear layer, which performs a simple linear mapping to align the model output tensor and the 0-dimensional diagnostic data tensor.

9. The time series regression prediction method with uncertainty estimation according to claim 1, characterized in that, The model training process described in step S3 includes the following steps: S31. Randomly initialize the weight and bias parameters of the neural network. S32. Initialize the neural network hyperparameters and the stochastic gradient descent optimizer. The neural network hyperparameters include batch size, learning rate, and number of iterations. S33. Use the target output time series and the predicted output of the time series output module, calculate the loss according to the loss function L1, and use the error backpropagation algorithm to optimize the model parameters; S34. Based on the existing model parameters, calculate the loss between the actual deviation and the predicted output of the uncertainty estimation module according to the loss function L2, and use the error backpropagation algorithm to optimize the parameters of the uncertainty estimation network in the model; S35. Finally, obtain the optimal time series prediction regression model with uncertainty estimation.

10. The time series regression prediction method with uncertainty estimation according to claim 3, wherein The loss functions L1 and L2 use a masking mechanism to calculate the effective mean square error loss VMSE and the weighted mean square error loss VWMSE according to the effective length of the time series. Specifically, it includes: The calculation formula of the effective mean square error loss function VMSE is as follows: Among them, is the effective length of the time series; is a matrix containing only 0 and 1, used to extract the effective part for the mean square error of the time series; is the real experimental data; is the model prediction output; The calculation formula of the effective weighted mean square error loss VWMSE is as follows: Among them, is the number of time slices that are valid and within the prediction uncertainty coverage range; is the number of time slices that are valid and outside the prediction uncertainty coverage range; is the true experimental data within the prediction uncertainty coverage range; is the model prediction output within the prediction uncertainty coverage range; is the true experimental data within the prediction uncertainty coverage range; is the model prediction output within the prediction uncertainty coverage range; is a parameter used to balance the importance of both the coverage rate and the uncertainty width; is the predefined target uncertainty coverage rate; is the prediction uncertainty coverage rate.

11. The time series regression prediction method with uncertainty estimation according to claim 10, characterized in that The described The calculation formula is as follows: where N is the length of the time series, is a binary value, and its calculation formula is as follows: Among them, is the target value, is the point prediction value, is the uncertainty.

12. A time series regression prediction system with uncertainty estimation, which is used to implement the steps of the time series regression prediction method with uncertainty estimation according to any one of claims 1-11, characterized in that, The system includes the following modules: Data acquisition and preprocessing module: used to acquire time series data, including input time series and output time series, and perform resampling and standardization preprocessing operations on the data to establish a data set for model training and testing; Neural network module: The neural network includes a position encoder, an input encoder, a time series output module, a direct uncertainty estimation module, and a linear output layer; Used to obtain the output time series and its uncertainty according to the input time series; Training module: used to train the neural network using the training data set of the task to obtain a trained neural network; Testing module: used to input the input time series data of the test set into the trained neural network model to obtain the predicted time series and its uncertainty.

13. An electronic device, characterized in that, It includes a readable storage medium, a central processing unit, and a graphics processing unit; wherein, the readable storage medium is used to store one or more computer programs, and when the processor executes the computer programs, it implements the steps of the time series regression prediction method with uncertainty estimation according to any one of claims 1 to 11.

14. A readable storage medium, characterized in that, A computer program is stored on the readable storage medium, and when the computer program is run by the processor, it executes the steps of the time series regression prediction method with uncertainty estimation according to any one of claims 1 to 11.

Images