Financial data prediction implementation method and device based on diffusion model, and storage medium
By dividing time series data into historical windows and prediction windows, integrating features and adding noise training diffusion models, the problems of insufficient utilization of historical data and inconsistent processes in the existing technology are solved, and higher-precision financial data prediction is achieved.
Patent Information
- Application Number
- CN202510243033.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-28
- Publication Date
- 2025-07-04
AI Technical Summary
The existing financial data prediction methods based on diffusion models fail to make full use of historical data, resulting in limited prediction accuracy and inconsistent forward diffusion and backward denoising processes, affecting prediction performance.
By dividing the time series data into historical window sequences and prediction window sequences, fusing feature extraction and adding noise, using the diffusion step count minimization loss function to train the target diffusion model, ensuring the consistency of the forward diffusion and backward denoising processes, and fully mining the key features in historical data.
It improves the accuracy and consistency of model predictions, and significantly improves the accuracy and robustness of financial data predictions.
Smart Images
Figure CN120258980A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of diffusion models, and in particular, to a method and device for realizing financial data prediction based on a diffusion model, and a storage medium. Background Art
[0002] In recent years, diffusion models have shown excellent performance in generation tasks in various fields. Financial transaction data prediction is an important task that identifies potential patterns in time series data and uses them to infer future data points to solve many problems in life. The prediction results obtained through such analysis can be widely applied to different fields, including finance, transportation, stock markets, and energy.
[0003] Existing prediction methods based on diffusion models can be roughly divided into two categories: one is the autoregressive method, that is, generating a prediction result based on the last observation result; the other is the non-autoregressive method, that is, first learning the distribution of the time series, and then generating a prediction result through conditional sampling. Although the autoregressive method performs well in short-term prediction, its long-term prediction results are inaccurate due to the existence of cumulative errors. The non-autoregressive method can well solve the long-term prediction problem. However, most existing diffusion-based non-autoregressive methods face one (or more) of the following problems, making it impossible to achieve the expected prediction effect.
[0004] First, since the conditional strategy was initially designed for images, its sampling results are more suitable for time series data. For example, the TimeGrad model makes predictions in an autoregressive manner, while the CSDI model uses non-autoregressive generation and self-supervised masking to guide the denoising process. Second, most training methods perform conditional guidance by introducing inductive bias during the denoising process, which may not be able to obtain sufficient prediction information from the conditions. The TimeDiff model adopts a non-autoregressive strategy for conditional sampling, while the Mr-Diff model uses seasonal trend decomposition to decompose the time series into multiple resolutions to guide the denoising process. Finally, these methods use an unconditional forward process to perturb the time series, and conditionally generate the future sequence based on historical data during the denoising process. This inconsistency between the forward process and the backward process may limit the prediction performance of the diffusion model. Therefore, practice shows that the existing financial data prediction methods based on diffusion models in the prior art fail to make full use of historical data, resulting in limited prediction accuracy. Summary of the Invention
[0005] The present invention provides a method and device for realizing financial data prediction based on a diffusion model, and a storage medium, which are used to make full use of historical data and improve the prediction accuracy of the diffusion model.
[0006] To solve the above technical problems, a first aspect of the present invention discloses a method for realizing financial data prediction based on a diffusion model, the method comprising:
[0007] Obtain time series data corresponding to a target financial transaction, where the time series data is target financial transaction data collected over a continuous time period;
[0008] Divide the time series data into multiple windows, and each window is divided into a historical window sequence and a prediction window sequence;
[0009] For each of the windows, fuse the historical window sequence and the prediction window sequence according to a preset sequence fusion method to obtain a fused sequence; extract the integrated features of the fused sequence according to a preset feature extraction strategy;
[0010] Randomly sample a batch of diffusion steps that follow a uniform distribution from the time series data, add noise and the integrated features to each of the prediction window sequences according to the diffusion steps to generate a noise-added prediction window sequence; and minimize a loss function according to the noise-added prediction window sequence and the diffusion steps;
[0011] Train an initial diffusion model according to the time series data and the loss function to obtain a target diffusion model trained to convergence;
[0012] The target diffusion model is used to perform data prediction on the target financial transaction.
[0013] As an optional implementation manner, in the first aspect of the present invention, the extracting the integrated features of the fused sequence according to a preset feature extraction strategy includes:
[0014] Extract the integrated features of the fused sequence based on a preset feature extraction adapter according to a preset feature extraction strategy;
[0015] Wherein, the feature extraction adapter includes a plurality of residual layers, and each residual layer includes two encoders, two structured state space sequences and a decoder, wherein:
[0016] The two encoders are used to extract high-level feature representations from the fused sequence;
[0017] The two structured state spaces are used to capture the temporal dynamic characteristics in the high-level feature representations;
[0018] The decoder is used to decode an integrated sequence according to the high-level feature representation and the temporal dynamic characteristics corresponding to the high-level feature representation.
[0019] As an alternative implementation, in the first aspect of the present invention, adding noise and the integrated features to each of the prediction windows according to the diffusion steps to generate a noise-added prediction window sequence includes:
[0020] Adding noise and the integrated features to each of the prediction window sequences according to the diffusion steps, and generating a noise-added prediction window sequence based on the following formula:
[0021]
[0022] In the above formula, is the noise-added prediction window sequence, is the prediction window sequence, H is the window length of the prediction window sequence, t is the diffusion step, where α s ∈[0, 1] is the noise addition amount weight, ε is a noise matrix conforming to a normal distribution, λ t is the integrated feature weight, where, C t is the integrated feature.
[0023] As an alternative implementation, in the first aspect of the present invention, the loss function L t (θ) is where:
[0024] is the time series sample predicted by the diffusion model according to the noise-added prediction window sequence and the diffusion step t.
[0025] As an alternative implementation, in the first aspect of the present invention, the specific manner in which the target diffusion model performs data prediction on the target financial transaction includes:
[0026] Obtaining the transaction data of the target financial transaction within a preset time period as the target historical window sequence, and determining a corresponding Gaussian distribution noise vector according to the target historical window sequence;
[0027] Estimating an approximate prediction window sequence from the target historical window sequence;
[0028] Fusing the target historical window sequence and the approximate prediction window sequence according to the preset sequence fusion method to obtain a target fusion sequence; extracting the target integrated feature of the target fusion sequence according to the preset feature extraction strategy;
[0029] The target diffusion model denoises the Gaussian distribution noise vector based on the target integrated feature to obtain a target prediction sequence, and the target prediction sequence is used to represent the data prediction result of the target financial transaction.
[0030] As an alternative implementation, in the first aspect of the present invention, estimating an approximate prediction window sequence from the target historical window sequence includes:
[0031] Mapping the target historical window sequence to the hidden state h n in a cyclic manner to obtain the target hidden state h L ;
[0032] wherein, the mapping manner of the hidden state h n is as follows:
[0033] h n = LSTM en (h n-1 , x n )
[0034] wherein, x n is the nth value of the target historical window sequence , h n is the hidden state, n = {1, 2,..., L}, L is the window length of the target historical window sequence, and LSTM en represents a long short-term memory network unit for encoding;
[0035] Estimate the approximate prediction window sequence according to the following formula:
[0036]
[0037] In the above formula, is the mth value of the approximate prediction window sequence , m = {1, 2,..., H}, W represents the weight of a preset fully connected layer, b represents the bias of a preset fully connected layer, and h' m is determined based on the following formula:
[0038]
[0039] In the above formula, h'0 = h L , LSTM de represents a long short-term memory network unit for decoding.
[0040] As an alternative implementation, in the first aspect of the present invention, the target diffusion model denoises the approximate prediction window sequence based on the target integrated features to obtain a target prediction sequence, including:
[0041] Set the number of denoising steps to T, and set the initial sequence to be the Tth value of the Gaussian distribution noise vector
[0042] Iteratively denoise the Gaussian distribution noise vector according to the following formula:
[0043]
[0044] where is the data after denoising for T - t + 1 times, is the time series sample predicted by the target diffusion model according to the Gaussian distribution noise vector and the diffusion step t, where α s ∈ [0, 1] is the noise addition amount weight, λ t is the integrated feature weight, where is the target integrated feature, β t = 1 - α t , and ε is a noise matrix conforming to a normal distribution.
[0045] The second aspect of the present invention discloses a financial data prediction implementation device based on a diffusion model, and the device includes:
[0046] A sampling module, configured to obtain time series data corresponding to a target financial transaction, where the time series data is target financial transaction data collected over a continuous time period;
[0047] A partitioning module, configured to divide the time series data into multiple windows, and each window is divided into a historical window sequence and a prediction window sequence;
[0048] A feature extraction adapter, configured to, for each of the windows, fuse the historical window sequence and the prediction window sequence according to a preset sequence fusion device to obtain a fused sequence; extract the integrated feature of the fused sequence according to a preset feature extraction strategy;
[0049] A diffusion module, configured to randomly sample a batch of diffusion steps obeying a uniform distribution from the time series data, add noise and the integrated feature to each of the prediction window sequences according to the diffusion steps to generate a noise - added prediction window sequence; and minimize a loss function according to the noise - added prediction window sequence and the diffusion steps;
[0050] A training module, configured to train an initial diffusion model according to the time series data and the loss function to obtain a target diffusion model trained to convergence; the target diffusion model is used to implement data prediction for the target financial transaction.
[0051] As an alternative implementation, in the second aspect of the present invention, the feature extraction adapter includes a plurality of residual layers, and each of the residual layers includes two encoders, two structured state space sequences, and a decoder;
[0052] Moreover, the specific manner in which the feature extraction adapter extracts the integrated features of the fusion sequence according to a preset feature extraction strategy includes:
[0053] The two encoders are used to extract high-level feature representations from the fusion sequence;
[0054] The two structured state spaces are used to capture the temporal dynamic characteristics in the high-level feature representations;
[0055] The decoder is used to decode an integrated sequence based on the high-level feature representation and the temporal dynamic characteristics corresponding to the high-level feature representation.
[0056] As an alternative implementation, in the second aspect of the present invention, the specific manner in which the diffusion module adds noise and the integrated features to each prediction window according to the number of diffusion steps to generate a sequence of noise-added prediction windows includes:
[0057] Adding noise and the integrated features to each prediction window sequence according to the number of diffusion steps, and generating a sequence of noise-added prediction windows based on the following formula:
[0058]
[0059] In the above formula, is the sequence of noise-added prediction windows, is the prediction window sequence, H is the window length of the prediction window sequence, t is the number of diffusion steps, where α s ∈[0, 1] is the noise addition amount weight, ε is a noise matrix conforming to a normal distribution, λ t is the integrated feature weight, where, C t is the integrated feature.
[0060] As an alternative implementation, in the second aspect of the present invention, the loss function L t (θ) is where:
[0061] is the time series sample predicted by the diffusion model according to the sequence of noise-added prediction windows and the number of diffusion steps t.
[0062] As an alternative implementation, in the second aspect of the present invention, the apparatus further includes a past-future mapping module. When predicting data for the target financial transaction:
[0063] The sampling module is further configured to obtain transaction data of the target financial transaction within a preset time period as a target historical window sequence, and determine a corresponding Gaussian distribution noise vector according to the target historical window sequence;
[0064] The past-future mapping module is configured to estimate an approximate prediction window sequence from the target historical window sequence;
[0065] The feature extraction adapter is further configured to fuse the target historical window sequence and the approximate prediction window sequence according to the preset sequence fusion device to obtain a target fusion sequence; extract target integrated features of the target fusion sequence according to the preset feature extraction strategy;
[0066] The target diffusion model is configured to denoise the Gaussian distribution noise vector based on the target integrated features to obtain a target prediction sequence, and the target prediction sequence is used to represent the data prediction result of the target financial transaction.
[0067] As an alternative implementation, in the second aspect of the present invention, the specific manner in which the past-future mapping module estimates an approximate prediction window sequence from the target historical window sequence includes:
[0068] Map the target historical window sequence to the hidden state h n in a cyclic manner to obtain the target hidden state h L ;
[0069] where the mapping method of the hidden state h n is as follows:
[0070] h n = LSTM en (h n-1 , x n )
[0071] where X n is the nth value of the target historical window sequence , h n is the hidden state, n = {1, 2,..., L}, L is the window length of the target historical window sequence, and LSTM en represents a long short-term memory network unit for encoding;
[0072] Estimate the approximate prediction window sequence according to the following formula:
[0073]
[0074] In the above formula, is the m-th value of the approximate prediction window sequence , where m = {1, 2,..., H}, W represents the weight of the preset fully connected layer, b represents the bias of the preset fully connected layer, and h′ m is confirmed based on the following formula:
[0075]
[0076] In the above formula, h′0 = h L , LSTM de represents a long short-term memory network unit for decoding.
[0077] As an alternative implementation, in the second aspect of the present invention, the specific manner in which the target diffusion model denoises the approximate prediction window sequence based on the target integrated feature to obtain the target prediction sequence includes:
[0078] Set the number of denoising steps to T, and set the initial sequence as the T-th value of the Gaussian distribution noise vector
[0079] Iteratively denoise the Gaussian distribution noise vector according to the following formula:
[0080]
[0081] where, is the data after T - t + 1 times of denoising, is the time series sample predicted by the target diffusion model according to the Gaussian distribution noise vector and the diffusion step t, where α s ∈[0, 1] is the noise addition amount weight, λ t is the integrated feature weight, where, is the target integrated feature, β t = 1 - α t , and ε is a noise matrix conforming to a normal distribution.
[0082] The third aspect of the present invention discloses another financial data prediction implementation system based on a diffusion model, and the system includes:
[0083] A memory storing executable program code;
[0084] A processor coupled to the memory;
[0085] The processor calls the executable program code stored in the memory and executes the method for realizing financial data prediction based on a diffusion model disclosed in the first aspect of the present invention.
[0086] The fourth aspect of the present invention discloses a computer storage medium storing computer instructions, which are used to execute the method for realizing financial data prediction based on a diffusion model disclosed in the first aspect of the present invention when the computer instructions are called.
[0087] Compared with the prior art, in the construction process of the target diffusion model in the present invention, the integrated features are extracted from the fusion sequence obtained by fusing the historical window sequence and the prediction window sequence, so that the key features in the historical data can be fully mined; in addition, the forward diffusion and backward denoising processes in the embodiments of the present invention are consistent, thereby improving the prediction accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS
[0088] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.
[0089] Figure 1 It is a schematic flowchart of a method for realizing financial data prediction based on a diffusion model disclosed in an embodiment of the present invention;
[0090] Figure 2 It is a schematic flowchart of a method for training a diffusion model for financial data prediction disclosed in an embodiment of the present invention;
[0091] Figure 3 It is a schematic flowchart of another method for realizing financial data prediction based on a diffusion model disclosed in an embodiment of the present invention;
[0092] Figure 4 It is a schematic structural diagram of a device for realizing financial data prediction based on a diffusion model disclosed in an embodiment of the present invention;
[0093] Figure 5 It is a schematic structural diagram of another device for realizing financial data prediction based on a diffusion model disclosed in an embodiment of the present invention;
[0094] Figure 6 It is a schematic structural diagram of a system for realizing financial data prediction based on a diffusion model disclosed in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0095] To enable those skilled in the art to better understand the solution of the present invention, the following will clearly and completely describe the technical solution in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.
[0096] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, device, product or terminal including a series of steps or units is not limited to the listed steps or units, but may optionally further include steps or units not listed, or may optionally further include other steps or units inherent to these processes, methods, products or terminals.
[0097] Referring to "embodiment" herein means that the specific features, structures or characteristics described in connection with the embodiment may be included in at least one embodiment of the present invention. The phrase appears in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described herein may be combined with other embodiments.
[0098] The present invention discloses a method, device and storage medium for realizing financial data prediction based on a diffusion model, which are used to make full use of historical data and improve the prediction accuracy of the diffusion model.
[0099] Embodiment 1
[0100] Please refer to Figure 1 , Figure 1 which is a schematic flowchart of a method for realizing financial data prediction based on a diffusion model disclosed in an embodiment of the present invention. Among them, Figure 1 the described method for realizing financial data prediction based on a diffusion model can be implemented in a device for realizing financial data prediction based on a diffusion model. The device for realizing financial data prediction based on a diffusion model can be integrated in a cloud server or a local server. As Figure 1 shown, the method for realizing financial data prediction based on a diffusion model may include the following operations:
[0101] Step 101: Obtain time series data corresponding to a target financial transaction, where the time series data is target financial transaction data collected over a continuous time period.
[0102] In the embodiments of the present invention, there can also be multiple types of time series data corresponding to the target financial transaction. In this case, the dimension of the time series data is larger. For example, d types of financial transaction data samples collected over a continuous time period are obtained as the time series data X. r . Optionally, the time series data X r can be normalized to obtain the data X.
[0103] Step 102: Divide the time series data into multiple windows, and each window is divided into a historical window sequence and a prediction window sequence.
[0104] In the embodiments of the present invention, in order to make full use of the historical data of the time series data, each window is divided into a historical window sequence and a prediction window sequence. For example, a part of the data X is segmented into a historical window sequence and a prediction window sequence where L is the length of the historical window, H is the length of the prediction window, and d is the dimension of the time series. When d = 1, the time series is considered univariate, and when d>1, it is considered multivariate.
[0105] Step 103: For each window, fuse the historical window sequence and the prediction window sequence according to a preset sequence fusion method to obtain a fused sequence; extract the integrated features of the fused sequence according to a preset feature extraction strategy.
[0106] Different from the prior art, the integrated features in the embodiments of the present invention are extracted from the fused sequence obtained by fusing the historical window sequence and the prediction window sequence. These integrated features are used to participate in the subsequent noise removal process, so the consistency of the forward diffusion and backward denoising processes can be ensured, and the accuracy of model prediction is improved.
[0107] In an optional embodiment, extracting the integrated features of the fused sequence according to a preset feature extraction strategy may include:
[0108] Extracting the integrated features of the fused sequence according to a preset feature extraction strategy based on a preset feature extraction adapter;
[0109] where the feature extraction adapter may include multiple residual layers, and each residual layer may include two encoders, two structured state space sequences, and a decoder, where:
[0110] The two encoders are used to extract high-level feature representations from the fused sequence;
[0111] The two structured state spaces are used to capture the temporal dynamic characteristics in the high-level feature representations;
[0112] The decoder is used to decode the integrated sequence according to the high-level feature representation and the temporal dynamic characteristics corresponding to the high-level feature representation.
[0113] In this optional embodiment, by introducing a residual connection, the input can be directly passed to the output, while allowing the model to learn the residual (difference) between the input and the output, thereby improving the training efficiency and performance of the model. Additionally, the residual layer realizes the complete process of feature extraction, dynamic modeling, and output generation through the combination of an encoder, a structured state space sequence, and a decoder, while ensuring the training stability of the network using the residual connection.
[0114] Step 104: Randomly sample a batch of diffusion steps that follow a uniform distribution from the time series data, add noise and integrated features to each prediction window sequence according to the diffusion steps to generate a noisy prediction window sequence; and minimize the loss function based on the noisy prediction window sequence and the diffusion steps.
[0115] In the embodiment of the present invention, the integrated feature is the core feature of the time series data. Using the principle of the diffusion model, the integrated feature is combined with the noise and added to the prediction window to implement the noise addition process in the construction of the diffusion model.
[0116] Step 105: Train the initial diffusion model according to the time series data and the loss function to obtain a target diffusion model trained to convergence.
[0117] In the embodiment of the present invention, the target diffusion model is used for data prediction of target financial transactions.
[0118] In the construction process of the target diffusion model in the embodiment of the present invention, the integrated feature is extracted from the fusion sequence obtained by fusing the historical window sequence and the prediction window sequence, so that the key features in the historical data can be fully mined. Additionally, the forward diffusion and backward denoising processes in the embodiment of the present invention are consistent, thereby improving the prediction accuracy of the model.
[0119] In an optional embodiment, adding noise and integrated features to each prediction window according to the diffusion steps to generate a noisy prediction window sequence may include:
[0120] Adding noise and integrated features to each prediction window sequence according to the diffusion steps, and generating a noisy prediction window sequence based on the following formula:
[0121]
[0122] In the above formula, is the noisy prediction window sequence, is the prediction window sequence, H is the window length of the prediction window sequence, t is the diffusion step, where α s ∈ [0, 1] is the noise addition amount weight, ε is a noise matrix that conforms to a normal distribution, and λ t is the integrated feature weight, where, C t is an integrated feature.
[0123] In yet another alternative embodiment, the loss function L t (θ) can be where:
[0124] is the time series sample predicted by the diffusion model based on the noise-added prediction window sequence and the diffusion step t.
[0125] In yet another alternative embodiment, the specific manner in which the target diffusion model performs data prediction on the target financial transaction may include:
[0126] Obtain the transaction data of the target financial transaction within a preset time period as the target historical window sequence, and determine a corresponding Gaussian distribution noise vector according to the target historical window sequence;
[0127] Estimate an approximate prediction window sequence from the target historical window sequence;
[0128] Fuse the target historical window sequence and the approximate prediction window sequence according to a preset sequence fusion method to obtain a target fusion sequence; extract the target integrated feature of the target fusion sequence according to a preset feature extraction strategy;
[0129] The target diffusion model denoises the Gaussian distribution noise vector based on the target integrated feature to obtain a target prediction sequence, and the target prediction sequence is used to represent the data prediction result of the target financial transaction.
[0130] In this alternative embodiment, an approximate prediction window sequence is estimated according to the historical window sequence, which matches the practice of dividing each time series data into a historical window and a prediction window during the model training stage. Subsequently, the operations of fusing the target historical window sequence and the approximate prediction window sequence and extracting the target feature set enable the prediction process of this alternative embodiment to fully consider the fusion features of historical data and future data, significantly improving the prediction performance of the model.
[0131] In yet another alternative embodiment, estimating an approximate prediction window sequence from the target historical window sequence may include:
[0132] Map the target historical window sequence to the hidden state h n in a cyclic manner to obtain the target hidden state h L ;
[0133] where the mapping method of the hidden state h n is as follows:
[0134] h n = LSTM en (h n-1 , x n )
[0135] where x n is the n-th value of the target historical window sequence and h n is the hidden state, n = {1, 2,..., L}, L is the window length of the target historical window sequence, and LSTM en represents a long short-term memory network cell for encoding;
[0136] The approximate prediction window sequence is estimated according to the following formula:
[0137]
[0138] In the above formula, is the m-th value of the approximate prediction window sequence where m = {1, 2,..., H}, W represents the weight of a preset fully connected layer, b represents the bias of a preset fully connected layer, and h' m is determined based on the following formula:
[0139]
[0140] In the above formula, h'0 = h L , LSTM de represents a long short-term memory network cell for decoding.
[0141] Furthermore, the target diffusion model denoises the approximate prediction window sequence based on the target integrated features to obtain the target prediction sequence, which may include:
[0142] Set the number of denoising steps to T, and set the initial sequence to be the T-th value of a Gaussian distribution noise vector
[0143] Iteratively denoise the Gaussian distribution noise vector according to the following formula:
[0144]
[0145] where, is the data after T - t + 1 times of denoising, is the time series sample predicted by the target diffusion model according to the Gaussian distribution noise vector and the diffusion step t, where α s ∈ [0, 1] is the noise addition amount weight, λt is the integrated feature weight, where is the target integrated feature, β t = 1 - α t , and ε is a noise matrix conforming to a normal distribution.
[0146] In yet another alternative embodiment, in step 101, the time series data is divided into multiple windows, and each window is divided into a historical window sequence and a prediction window sequence, which may include:
[0147] Fit the time series data to a target curve, and sample a series of target points on the target curve according to a preset sampling frequency. For example, a target point can be selected every fixed distance;
[0148] Calculate the slope of the tangent line at the position of the target curve where each target point is located to obtain a set of slopes corresponding to the time series data;
[0149] Traverse the set of slopes, and exclude the outliers in the set of slopes. The outliers mainly include extreme values whose absolute values exceed a set threshold. During the traversal, whenever the slopes of m consecutive sets of slopes in the set of slopes can be recognized as having a changing trend, then the target points corresponding to these m slopes are determined as a window. Additionally, if the slopes of m consecutive sets of slopes in the set of slopes cannot be recognized as having a changing trend but m exceeds a preset node length, the target points corresponding to these m slopes are also determined as a window. This provides more reliable and reasonable training data for subsequent model training.
[0150] Furthermore, each window is divided into a historical window sequence and a prediction window sequence, which can be randomly divided, or may include:
[0151] Determine the subset of slopes corresponding to each window. For each subset of slopes, select the first k slopes as the historical window slopes, where the changing trend of the first k slopes is closest to the changing trend of the entire subset of slopes.
[0152] Embodiment 2
[0153] The embodiment of the present invention discloses a specific implementation method for financial data prediction based on a diffusion model, and the description of this method is as follows:
[0154] Refer to Figure 2 , and the training method of the prediction model in the implementation method for financial data prediction based on a diffusion model may include the following steps:
[0155] St1, Obtain d types of financial transaction data samples collected over a continuous time period as time series data X r , normalize the time series data X r to obtain data X, and divide part of the data X into a historical window sequence and the prediction window sequence where L is the length of the historical window, H is the length of the prediction window, and d is the dimension of the time series. When d = 1, the time series is considered univariate, and when d > 1, it is considered multivariate.
[0156] Construct and initialize the base model, which includes a feature extraction adapter module, a past-future mapping module, and a diffusion module; the feature extraction adapter module is used to capture key time features for prediction from the combined time series of the historical window and the prediction window; the diffusion module is used to reconstruct the prediction sequence.
[0157] St2. Extract the integrated feature C through the feature extraction adapter t , C t represents the output of the feature extraction adapter when the input is and t, represents the fused sequence formed by combining the historical window sequence and the prediction window sequence .
[0158] Among them, the integrated feature C extracted by the feature extraction adapter t is:[[]]
[0159]
[0160] where represents the fused sequence formed by combining the historical window sequence and the prediction window sequence ; each residual layer of the feature extraction adapter network consists of two encoders, two structured state space sequences (S4s), and a decoder.
[0161] St3. Randomly sample a batch of diffusion steps that follow a uniform distribution, add noise to each prediction window sequence through the diffusion module, and minimize the variable L t (θ), and the variable L t (θ) is used to let the denoising network predict the time series sample
[0162] The noise-added prediction window sequence is:[[]]
[0163]
[0164] where α s ∈ [0,1] represents the weight of the noise addition amount, ε ∈ R d×H is a noise matrix that conforms to a normal distribution. λ tis the weight of the integrated feature. In the present invention, λ t is set to
[0165] In an optional embodiment, the steps for the diffusion module to reconstruct the prediction sequence are as follows:
[0166] The diffusion module sets the diffusion step number T;
[0167] Add noise to the prediction window sequence to generate a noise-added sequence
[0168] Perform step-by-step denoising on the noise-added sequence to generate a prediction sequence
[0169]
[0170] wherein, is the data after T - t + 1 times of denoising, is the predicted time series sample estimated by the denoising network of the diffusion module through inputting and the diffusion step t, β t = 1 - α t , ε ∈ R d×H is a noise matrix conforming to a normal distribution.
[0171] The calculation formula of the variable L t (θ) is:
[0172]
[0173] wherein, is the predicted time series sample estimated by the denoising network of the diffusion module through inputting and the diffusion step t.
[0174] St4. Combine the training data X and the variable L t (θ) to iteratively update the parameters of the diffusion module. After the diffusion module converges, combine the diffusion module with the feature extraction adapter module to form a prediction model.
[0175] St5. When the prediction model works, the feature extraction adapter module is used to capture the key time features for prediction from the comprehensive time series of the historical window and the prediction window; the diffusion module is used to reconstruct the prediction sequence.
[0176] Furthermore, the financial transaction data prediction method in the financial data prediction implementation method based on the diffusion model may include:
[0177] Refer to Figure 3, the prediction method in this embodiment may include the following steps:
[0178] Sta. Obtain the above-trained prediction model; obtain the financial transaction data over a specified time period as the historical window sequence And initialize a Gaussian distribution noise vector As the prediction window sequence;
[0179] Stb. Estimate an approximate prediction window from the historical window sequence through the past-future mapping module From
[0180] Stc. The feature extraction adapter extracts the integrated feature C t , C t Represents the output of the feature extraction adapter when the input is And t, Represents the historical window sequence And the approximate prediction window sequence The merged fusion sequence formed;
[0181] Std. Denoise the prediction window sequence through the diffusion module To obtain the prediction sequence
[0182] The advantages of the embodiments of the present invention are as follows:
[0183] (1) The embodiments of the present invention propose a method for training a financial transaction data prediction model, design a diffusion model, and ensure the consistency of the forward diffusion and backward denoising processes through the feature extraction adapter and the past-future mapping module, improving the prediction accuracy. In the prediction stage of the embodiments of the present invention, the fusion features of historical data and future data are considered, significantly improving the prediction performance of the model.
[0184] (2) A financial transaction data prediction method proposed by the embodiments of the present invention first trains a prediction model based on a diffusion model, adds a feature extraction adapter and a past-future mapping module during training, and captures key features in the time series through these modules, ensuring the accuracy and robustness of the prediction, thereby alleviating the inconsistency problem of the model during the training process.
[0185] In the embodiments of the present invention, in order to verify the performance of the financial transaction data prediction model provided by the invention, combined with different data sets, the financial transaction data prediction model provided by the present invention is compared with various other existing models.
[0186] In this embodiment, the datasets CSI300 and Exchange are used respectively. Among them, the dataset CSI300 records the financial data of the CSI 300 Index; the dataset Exchange records the exchange rate changes between different countries.
[0187] In this embodiment, multiple existing models are selected, and TimeDTR is compared with these benchmark methods. For the sake of easy distinction, the financial transaction data prediction model provided by the present invention is denoted as the TimeDTR model, and the TimeDTR model is trained according to the training method provided by the present invention.
[0188] Table 1: Performance comparison on the dataset CSI300
[0189]
[0190] In this embodiment, when the CSI300 dataset is used, the values of ARR, ASR, CR, and IR are shown in Table 1. The performance of the portfolio and its risk management ability are evaluated by the annualized rate of return (ARR), the annualized Sharpe ratio (ASR), the Calmar ratio (CR), and the information ratio (IR), which can objectively evaluate the performance of the model. ARR represents the annualized rate of return and measures the average return of the investment; ASR is the annualized Sharpe ratio, which reflects the risk-adjusted return; CR is the Calmar ratio, which measures the ratio of the maximum drawdown to the annualized return; IR is the information ratio, which represents the ratio of the excess return to the tracking error. Among them, an upward arrow indicates that the higher the value, the better the model performance.
[0191] Table 2: Performance comparison on the dataset Exchange
[0192]
[0193] In this embodiment, when the Exchange dataset is used, the values of MAE and MSE are shown in Table 2. The prediction accuracy is evaluated by the mean squared error (MSE) and the mean absolute error (MAE), which can objectively evaluate the performance of the model. MAE represents the mean absolute error between the predicted value and the true value, reflecting the accuracy of the prediction result; MSE represents the mean squared error between the predicted value and the true value, reflecting the stability and robustness of the prediction result. Among them, a downward arrow indicates that the lower the value, the better the model performance.
[0194] It can be seen from the results that the overall performance of TimeDTR is the best. Whether it is the dataset CSI300 or the dataset Exchange, the average ranking of TimeDTR among all baselines has reached the first.
[0195] Embodiment Three
[0196] Please refer to Figure 4 ,Figure 4 This is a schematic structural diagram of a financial data prediction implementation device based on a diffusion model disclosed in an embodiment of the present invention. As Figure 4 shown, the financial data prediction implementation device based on the diffusion model may include:
[0197] A sampling module 201, configured to obtain time series data corresponding to a target financial transaction, where the time series data is target financial transaction data collected over a continuous time period;
[0198] A partitioning module 202, configured to divide the time series data into multiple windows, and each window is divided into a historical window sequence and a prediction window sequence;
[0199] A feature extraction adapter 203, configured to, for each window, fuse the historical window sequence and the prediction window sequence according to a preset sequence fusion device to obtain a fused sequence; extract integrated features of the fused sequence according to a preset feature extraction strategy;
[0200] A diffusion module 204, configured to randomly sample a batch of diffusion steps that follow a uniform distribution from the time series data, add noise and integrated features to each prediction window sequence according to the diffusion steps to generate a noise-added prediction window sequence; and minimize a loss function according to the noise-added prediction window sequence and the diffusion steps;
[0201] A training module 205, configured to train an initial diffusion model according to the time series data and the loss function to obtain a target diffusion model trained to convergence; the target diffusion model is used to implement data prediction for the target financial transaction.
[0202] In an optional embodiment, the feature extraction adapter 203 may include multiple residual layers, and each residual layer may include two encoders, two structured state space sequences, and a decoder;
[0203] Moreover, the specific manner in which the feature extraction adapter extracts integrated features of the fused sequence according to a preset feature extraction strategy may include:
[0204] The two encoders are configured to extract high-level feature representations from the fused sequence;
[0205] The two structured state spaces are configured to capture temporal dynamic characteristics in the high-level feature representations;
[0206] The decoder is configured to decode to obtain an integrated sequence according to the high-level feature representation and the temporal dynamic characteristics corresponding to the high-level feature representation.
[0207] In another optional embodiment, the specific manner in which the diffusion module 204 adds noise and integrated features to each prediction window according to the diffusion steps to generate a noise-added prediction window sequence may include:
[0208] Add noise and integrated features to each predicted window sequence according to the diffusion steps, and generate a noisy predicted window sequence based on the following formula:
[0209]
[0210] In the above formula, is the noisy predicted window sequence, is the predicted window sequence, H is the window length of the predicted window sequence, t is the diffusion step, where α s ∈ [0, 1] is the noise addition amount weight, ε is a noise matrix conforming to the normal distribution, λ t is the integrated feature weight, where, C t is the integrated feature.
[0211] In yet another alternative embodiment, the loss function L t (θ) is where:
[0212] is the time series sample predicted by the diffusion model according to the noisy predicted window sequence and the diffusion step t.
[0213] In yet another alternative embodiment, as shown in Figure 5 , the device may further include a past-future mapping module 206. When making data predictions for the target financial transaction:
[0214] The sampling module 201 is further configured to obtain the transaction data of the target financial transaction within a preset time period as the target historical window sequence, and determine a corresponding Gaussian distribution noise vector according to the target historical window sequence;
[0215] The past-future mapping module 206 is configured to estimate an approximate predicted window sequence from the target historical window sequence;
[0216] The feature extraction adapter 203 is further configured to fuse the target historical window sequence and the approximate predicted window sequence according to a preset sequence fusion device to obtain a target fusion sequence; extract the target integrated feature of the target fusion sequence according to a preset feature extraction strategy;
[0217] The target diffusion model is configured to denoise the Gaussian distribution noise vector based on the target integrated feature to obtain a target prediction sequence, and the target prediction sequence is used to represent the data prediction result of the target financial transaction.
[0218] In yet another alternative embodiment, the specific manner in which the past-future mapping module 206 estimates an approximate predicted window sequence from the target historical window sequence may include:
[0219] Map the target historical window sequence to the hidden state h in a cyclic manner n to obtain the target hidden state h L ;
[0220] Among them, the mapping method of the hidden state h n is as follows:
[0221] h n = LSTM en (h n-1 , x n )
[0222] Among them, X n is the nth value of the target historical window sequence , h n is the hidden state, n = {1, 2,..., L}, L is the window length of the target historical window sequence, and LSTM en represents a long short-term memory network unit for encoding;
[0223] Estimate the approximate prediction window sequence according to the following formula:
[0224]
[0225] In the above formula, is the mth value of the approximate prediction window sequence , m = {1, 2,..., H}, W represents the weight of the preset fully connected layer, b represents the bias of the preset fully connected layer, and h' m is confirmed based on the following formula:
[0226]
[0227] In the above formula, h'0 = h L , LSTM de represents a long short-term memory network unit for decoding.
[0228] In yet another alternative embodiment, the specific manner in which the target diffusion model denoises the approximate prediction window sequence based on the target integrated features to obtain the target prediction sequence may include:
[0229] Set the number of denoising steps to T, and set the initial sequence to the Tth value of the Gaussian distribution noise vector
[0230] Iteratively denoise the Gaussian distribution noise vector according to the following formula:
[0231]
[0232] Among them, is the data after denoising for T - t + 1 times, is the time - series sample predicted by the target diffusion model according to the Gaussian - distributed noise vector and the diffusion step t, where α s ∈[0, 1] is the weight of the noise addition amount, and λ t is the integrated feature weight. Among them, is the target integrated feature, and β t = 1 - α t , and ε is the noise matrix conforming to the normal distribution.
[0233] Example 4
[0234] Please refer to Figure 6 , Figure 6 which is a schematic structural diagram of a financial data prediction implementation system based on a diffusion model disclosed in an embodiment of the present invention. As Figure 6 shown, the financial data prediction implementation system based on the diffusion model may include:
[0235] A memory 301 storing executable program code;
[0236] A processor 302 coupled to the memory 301;
[0237] The processor 302 calls the executable program code stored in the memory 301 and executes the steps in the financial data prediction implementation method based on the diffusion model described in Embodiment 1 or Embodiment 2 of the present invention.
[0238] Example 5
[0239] An embodiment of the present invention discloses a computer storage medium. When the computer instructions stored in this computer storage medium are called, they are used to execute the steps in the financial data prediction implementation method based on the diffusion model described in Embodiment 1 or Embodiment 2 of the present invention.
[0240] Example 6
[0241] An embodiment of the present invention discloses a computer program product. This computer program product includes a non - transitory computer - readable storage medium storing a computer program, and this computer program is operable to cause a computer to execute the steps in the financial data prediction implementation method based on the diffusion model described in Embodiment 1 or Embodiment 2.
[0242] The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed to multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative effort.
[0243] Through the above specific descriptions of the embodiments, those skilled in the art can clearly understand that each implementation mode can be realized by means of software plus a necessary general hardware platform, and of course, it can also be realized by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, and the storage medium includes read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc memories, magnetic disk memories, tape memories, or any other computer-readable medium capable of carrying or storing data.
[0244] Finally, it should be noted that: The method, device, and storage medium for realizing financial data prediction based on a diffusion model disclosed in the embodiments of the present invention only disclose the preferred embodiments of the present invention, and are only used to illustrate the technical solutions of the present invention, rather than to limit them; Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for realizing financial data prediction based on a diffusion model, characterized in that, The method includes: Obtaining time series data corresponding to a target financial transaction, where the time series data is target financial transaction data collected over a continuous time period; Dividing the time series data into multiple windows, and each window is divided into a historical window sequence and a prediction window sequence; For each of the windows, fusing the historical window sequence and the prediction window sequence according to a preset sequence fusion method to obtain a fused sequence; extracting integrated features of the fused sequence according to a preset feature extraction strategy; Randomly sampling a batch of diffusion steps that follow a uniform distribution from the time series data, adding noise and the integrated features to each of the prediction window sequences according to the diffusion steps to generate a noise-added prediction window sequence; and minimizing a loss function according to the noise-added prediction window sequence and the diffusion steps; Training an initial diffusion model according to the time series data and the loss function to obtain a target diffusion model trained to convergence; The target diffusion model is used for data prediction of the target financial transaction.
2. The implementation method for predicting financial data based on a diffusion model according to claim 1, wherein The extracting integrated features of the fused sequence according to a preset feature extraction strategy includes: Extracting integrated features of the fused sequence based on a preset feature extraction adapter according to a preset feature extraction strategy; Wherein, the feature extraction adapter includes a plurality of residual layers, and each residual layer includes two encoders, two structured state space sequences, and a decoder, where: The two encoders are used to extract high-level feature representations from the fused sequence; The two structured state spaces are used to capture temporal dynamic characteristics in the high-level feature representations; The decoder is used to decode the high-level feature representations and the temporal dynamic characteristics corresponding to the high-level feature representations to obtain an integrated sequence.
3. The implementation method for financial data prediction based on the diffusion model according to claim 1, characterized in that The adding noise and the integrated features to each of the prediction windows according to the diffusion steps to generate a noise-added prediction window sequence includes: Adding noise and the integrated features to each of the prediction window sequences according to the diffusion steps, and generating a noise-added prediction window sequence based on the following formula: In the above formula, is the noise-added prediction window sequence, is the prediction window sequence, H is the window length of the prediction window sequence, t is the diffusion step number, where α s ∈[0,1] is the noise addition amount weight, ε is a noise matrix conforming to the normal distribution, λ t is the integrated feature weight, where C t is the integrated feature.
4. The method for realizing financial data prediction based on a diffusion model according to claim 3, characterized in that, The loss function L t (θ) is Where: The time series sample predicted by the diffusion model according to the noise-added prediction window sequence and the diffusion step number t.
5. The method for realizing financial data prediction based on a diffusion model according to claim 1, characterized in that, The specific manner in which the target diffusion model performs data prediction on the target financial transaction includes: Obtaining transaction data of the target financial transaction within a preset time period as a target historical window sequence, and determining a corresponding Gaussian distribution noise vector according to the target historical window sequence; Estimating an approximate prediction window sequence from the target historical window sequence; Fusing the target historical window sequence and the approximate prediction window sequence according to the preset sequence fusion method to obtain a target fused sequence; extracting target integrated features of the target fused sequence according to the preset feature extraction strategy; The target diffusion model denoises the Gaussian distribution noise vector based on the target integrated features to obtain a target prediction sequence, and the target prediction sequence is used to represent the data prediction result of the target financial transaction.
6. The method for implementing financial data prediction based on a diffusion model according to claim 5, wherein The estimating an approximate prediction window sequence from the target historical window sequence includes: Map the target historical window sequence to the hidden state h in a cyclic manner n to obtain the target hidden state h L ; Among them, the hidden state h n is mapped as follows: h n = LSTM en (h n-1 , x n ) where x n is the n-th value of the target historical window sequence , h n is the hidden state, n = {1, 2, ..., L}, L is the window length of the target historical window sequence, and LSTM en represents a long short-term memory network unit for encoding; Estimating an approximate prediction window sequence according to the following formula: In the above formula, is the m-th value of the approximate prediction window sequence , where m = {1, 2,..., H}, W represents the weight of the preset fully connected layer, b represents the bias of the preset fully connected layer, and h′ m is determined based on the following formula: In the above formula, h′0 = h L , LSTM de represents a long short-term memory network unit for decoding.
7. The method for realizing financial data prediction based on a diffusion model according to claim 5, characterized in that, The target diffusion model denoises the approximate prediction window sequence based on the target integrated features to obtain a target prediction sequence, including: Set the denoising step number to T and set the initial sequence as the T-th value of the Gaussian distribution noise vector Iteratively denoising the Gaussian distribution noise vector according to the following formula: Among them, is the data after denoising for T - t + 1 times, is the time series sample predicted by the target diffusion model according to the Gaussian distribution noise vector and the diffusion step t, where α s ∈[0, 1] is the noise addition amount weight, λ t is the integrated feature weight, where is the target integrated feature, β t = 1 - α t , and ε is the noise matrix conforming to the normal distribution.
8. An apparatus for realizing financial data prediction based on a diffusion model, characterized in that, The device includes: A sampling module, configured to obtain time series data corresponding to a target financial transaction, where the time series data is target financial transaction data collected over a continuous time period; A partitioning module, configured to divide the time series data into multiple windows, and each window is divided into a historical window sequence and a prediction window sequence; A feature extraction adapter, configured to, for each of the windows, fuse the historical window sequence and the prediction window sequence according to a preset sequence fusion device to obtain a fused sequence; extract the integrated features of the fused sequence according to a preset feature extraction strategy; A diffusion module, configured to randomly sample a batch of diffusion steps that follow a uniform distribution from the time series data, add noise and the integrated features to each of the prediction window sequences according to the diffusion steps to generate a noise-added prediction window sequence; and minimize a loss function according to the noise-added prediction window sequence and the diffusion steps; A training module, configured to train an initial diffusion model according to the time series data and the loss function to obtain a target diffusion model trained to convergence; the target diffusion model is used to implement data prediction for the target financial transaction.
9. A financial data prediction implementation system based on a diffusion model, characterized in that, The system includes: a memory storing executable program code; a processor coupled to the memory; the processor calls the executable program code stored in the memory to execute the method for implementing financial data prediction based on a diffusion model according to any one of claims 1-7.
10. A computer storage medium, characterized in that, The computer storage medium stores computer instructions, which are used to execute the method for implementing financial data prediction based on a diffusion model according to any one of claims 1-7 when the computer instructions are called.