A time series data generation method and device based on a diffusion probability model

Abstract

The application discloses a time series data generation method and device based on a diffusion probability model, selects at least one feature column from original time series data as a basic feature column, calculates a relationship feature column of the remaining feature columns relative to the basic feature column, splices the original time series data with all the relationship feature columns, then performs linear interpolation expansion along a time axis to generate quasi-continuous time series data, samples training samples from the quasi-continuous time series data, adds noise to the training samples, and trains a diffusion probability model. Finally, random noise is sampled from standard Gaussian noise and input into the trained diffusion probability model to obtain generated time series data. The application improves the constraint satisfaction rate and trend similarity of generated data, and generates high-quality synthetic time series data meeting industry constraint conditions.

Description

Technical Field

[0001] This application belongs to the field of computer artificial intelligence technology, and in particular relates to a method and apparatus for generating time series data based on a diffusion probability model. Background Technology

[0002] Time series data possesses its own attribute rules across different industry sectors, exhibiting high complexity and strict business logic constraints. Multidimensional time series typically contain multiple interrelated feature columns, with inviolable business logic constraints between these columns. For example, in some sectors, different feature columns must satisfy the following condition: feature column a ≤ feature column b ≤ feature column c; or in specific sectors, the variation between feature column a and feature column b must be controlled within a certain threshold. These constraints are fundamental to the validity of time series data; violations result in logically flawed "pseudo-data" unusable for subsequent downstream tasks. Furthermore, time series in different sectors are influenced by various external or internal factors, exhibiting highly abrupt short-term trends. Time series may experience dramatic jumps within extremely short periods, making it difficult for traditional generative models based on smoothing assumptions to accurately capture their dynamic characteristics. On another front, time series in many sectors generally exhibit non-stationarity, with the mean and variance evolving over time, further complicating generative modeling.

[0003] Currently, time series data is generally scarce in practical applications across various industries. Due to multiple factors such as observation period limitations, data acquisition costs, difficulty in scene coverage, and privacy compliance constraints, it is difficult to acquire large quantities of complete, continuous, sufficiently sampled, and reliable time series data. Insufficient data volume has become a common challenge for time series analysis and modeling research in many fields. To meet this demand, many researchers have begun to use neural network models to generate time series data; however, existing models generally suffer from low-quality time series generation, loss of feature column constraint relationships, and insufficient fitting accuracy. Summary of the Invention

[0004] The purpose of this application is to provide a time series data generation method based on a diffusion probability model, which aims to solve the technical problems of low quality of multi-dimensional time series generation, loss of feature column constraint relationships, and insufficient fitting accuracy in the prior art.

[0005] To achieve the above objectives, the technical solution of this application is as follows:

[0006] A method for generating time series data based on a diffusion probability model includes:

[0007] Obtain the original time series data, select at least one feature column from the original time series data as the basic feature column, calculate the relationship feature columns of the remaining feature columns relative to the basic feature column, and concatenate the original time series data with all the relationship feature columns to obtain the enhanced time series data;

[0008] The enhanced time series data is linearly interpolated and extended along the time axis to generate near-continuous time series data.

[0009] Training samples are obtained by sampling quasi-continuous time series data, and noise is added to the training samples to train the diffusion probability model;

[0010] Random noise is sampled from standard Gaussian noise and input into a trained diffusion probability model to obtain the first time series output by the diffusion probability model. Then, downsampling and out-of-bounds pruning are performed. Finally, the generated time series data is obtained through the inverse operation of the basic feature columns and the relational feature columns.

[0011] Preferably, when the enhanced time series data is linearly interpolated and expanded along the time axis, and when expanding by M times, M-1 interpolation points are inserted between two adjacent nodes for each feature column.

[0012] Preferably, the diffusion probability model performs the following operations:

[0013] The input features are processed by an encoder and a decoder to obtain the decoded features;

[0014] The decoding features are processed through a trend extraction branch and a frequency domain extraction branch. The trend extraction branch obtains trend features by fitting a linear function, while the frequency domain extraction branch extracts frequency domain features by Fourier transform.

[0015] The trend features and frequency domain features are fused by a convolutional layer to obtain the output of the diffusion probability model.

[0016] Preferably, the total loss function used when training the diffusion probability model includes reconstruction loss and out-of-bounds loss.

[0017] Preferably, the reconstruction loss is calculated using the following formula:

[0018]

[0019] in, For samples without noise, This indicates the input samples with added noise corresponding to the model. Output , The reconstruction loss to be calculated.

[0020] Preferably, the out-of-bounds loss includes the upper bound out-of-bounds loss. and losses from crossing the lower boundary The formula for the loss due to exceeding the upper bound is:

[0021]

[0022]

[0023] in, Indicates model output The Middle The first node in the nth node The numerical values ​​corresponding to the relation feature columns; Indicates the first The first node in the nth node The loss is calculated based on the upper bound of the values ​​corresponding to the feature columns of a relation when they exceed the bounds; n represents the number of feature columns. Indicates the length of the expanded time dimension;

[0024] The formula for the lower bound out-of-bounds loss is:

[0025]

[0026]

[0027] in, Indicates the first The first node in the nth node The loss for the lower bound of the numerical values ​​corresponding to the relation feature columns.

[0028] This application also proposes a time series data generation device based on a diffusion probability model, including a processor and a memory storing a number of computer instructions, which, when executed by the processor, implement the steps of the above method.

[0029] This application proposes a method and apparatus for generating time series data based on a diffusion probability model. It explicitly extracts constraint relationships between feature columns through relational operations to generate relational feature columns. Linear interpolation is used to expand discrete sequences into near-continuous curves to reduce fitting difficulty. An out-of-bounds loss function penalty mechanism is introduced during diffusion model training to ensure constraint satisfaction. Finally, proportional downsampling and feature column reconstruction generate time series data that conforms to business logic. This application significantly improves the constraint satisfaction rate and trend similarity of the generated data, solving the problems of chaotic feature column relationships and low fitting accuracy in existing technologies. It can be widely applied to scenarios such as data synthesis and privacy-preserving data publishing, generating high-quality synthetic time series data that meets industry constraints. Attached Figure Description

[0030] Figure 1 This is a flowchart of the time series data generation method based on the diffusion probability model in this application. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0032] One embodiment of this application, such as Figure 1 As shown, a method for generating time series data based on a diffusion probability model is provided, including:

[0033] Step S1: Obtain the original time series data, select at least one feature column from the original time series data as the basic feature column, calculate the relationship feature columns of the remaining feature columns relative to the basic feature column, and concatenate the original time series data with all the relationship feature columns to obtain the enhanced time series data.

[0034] Time series data sources cover a wide range of fields, such as energy, electricity, and finance. One commonly used energy dataset recognized by researchers is the "Appliances Energy Prediction Data Set" from the UC Irvine Machine Learning Repository (UCI Machine Learning Repository). This is a publicly available dataset for research on building energy consumption prediction, containing over 19,000 data points and 28 feature columns. In the electricity sector, ETTh (Electricity Transformer Temperature) is a crucial dataset, providing hourly data on transformer temperatures from July 2016 to July 2018. It covers transformer load and oil temperature information, providing a vital data foundation for analyzing transformer operation and researching related issues, as well as model training and evaluation. In the financial sector, asset trading information from different markets is also an important research dataset. Financial time series data typically includes four price indicators (open, high, low, and close) as well as non-price information such as trading volume and turnover. This information is held by various exchanges, which compile and publicly release key daily trading information. In this embodiment, the original time series data is obtained through the above channels, denoted as... , express The number of data nodes in the system, each node has 1 feature column.

[0035] After obtaining the raw time series data, at least one feature column is selected as the basic feature column. The basic feature column must have a logical relationship or indirect logical relationship with the other feature columns, which are non-basic feature columns. Then, the relational feature columns of the other feature columns relative to the basic feature column are calculated. The relational feature columns reflect the business logic constraints between the basic feature column and the other feature columns.

[0036] Specifically, if there is only one basic feature column, for each non-basic feature column... The formula for calculating its relationship with the basic feature columns is as follows:

[0037] ;

[0038] in This is a function for calculating the characteristic columns of a relation. The function can be selected based on the characteristics of the specific data source. In a specific embodiment, The calculation formula is:

[0039] ;

[0040] For feature columns that require multiple basic feature columns to be constrained together The calculation formula is:

[0041] ;

[0042] The above A function to calculate the relational characteristic columns. , These represent different basic feature columns. In a specific embodiment, The calculation formula is:

[0043] .

[0044] This embodiment also normalizes the relational feature columns and maps them to the [−1,1] interval. Finally, the original time series data is concatenated with all relational feature columns to obtain the enhanced time series data. ,in Indicates the number of relational feature columns.

[0045] Existing technologies attempt to enable neural networks to implicitly learn the relationships between feature columns, but the constraints of time series data are essentially mathematical inequalities. This application innovatively transforms "constraint relationships" into "learnable relational feature columns," allowing the model to directly learn the relative changes between the target feature columns and the basic feature columns, rather than their absolute values. After normalizing the relational feature columns to the [−1,1] interval, the constraints are transformed into simple interval boundaries. The model learns the normalized relational feature columns, which have clear physical meaning and well-defined constraint boundaries.

[0046] Step S2: Linearly interpolate and augment the enhanced time series data along the time axis to generate quasi-continuous time series data.

[0047] In this embodiment, the expansion factor M is first determined based on the GPU memory capacity and computing efficiency, with a typical value range of 4 to 24.

[0048] For each feature column j, it is expanded M times along the time axis, for adjacent original nodes and Calculate the interpolation points:

[0049]

[0050] It should be noted that for the last original node of the sequence Since there are no subsequent nodes available for interpolation, forward difference expansion is used:

[0051]

[0052] This ultimately yields a class of continuous sequences:

[0053]

[0054] The matrix is ​​represented as:

[0055]

[0056] After linear interpolation augmentation, the data Transformed into Time series exhibit discrete point characteristics within short periods, lacking intermediate state information between adjacent time points. This application uses linear interpolation to fill in intermediate points between adjacent data points, transforming the discrete sequence into a quasi-continuous curve. Continuous functions can be approximated with arbitrary precision by polynomials, and smooth curves are easier for neural networks to learn than discrete point sets. Simultaneously, linear interpolation suppresses high-frequency noise, allowing the model to focus on underlying trends rather than random fluctuations. The augmented sequence makes the loss function surface more regular, the gradient descent direction more stable, and the optimization process more easily convergent.

[0057] Step S3: Sample training samples from quasi-continuous time series data, add noise to the training samples, and train the diffusion probability model.

[0058] This step involves continuous-series data. Samples were obtained by random sampling. By combining a randomly selected time step t, the input to the diffusion probability model is obtained through adding noise. The diffusion probability model in this application performs the following operations:

[0059] Step S3.1: The input features are processed by the encoder and decoder to obtain the decoded features.

[0060] This embodiment input The decoded features are obtained after passing through the encoder and decoder. Both encoders and decoders have their own structures, which are combinations of multilayer perceptrons and attention layers.

[0061] Step S3.2: The decoded features are processed through a trend extraction branch and a frequency domain extraction branch. The trend extraction branch obtains trend features by fitting a linear function, and the frequency domain extraction branch extracts frequency domain features by Fourier transform.

[0062] Output data The data is simultaneously fed to both the trend extraction branch and the frequency domain extraction branch. In the trend extraction branch, a linear function is used to fit the trend characteristics of the time series, and the output data is then processed. In the frequency domain extraction branch, the time domain data is transformed into frequency domain data through Fourier transform. Since each time domain data consists of multiple frequency domains, the frequency domain extraction branch needs to find the top k frequency domains with the greatest influence among the multiple frequency domain data and filter out the frequency domains with less influence, thereby simplifying the subsequent learning process.

[0063]

[0064]

[0065]

[0066] Frequency domain extraction branch final output data .

[0067] Step S3.3: The trend features and frequency domain features are fused through a convolutional layer to obtain the output of the diffusion probability model.

[0068]

[0069] Finally, the output data is passed through a convolutional layer. and The final data fused into the model .

[0070] Step S4: Sample random noise from standard Gaussian noise and input it into the trained diffusion probability model to obtain the first time series output by the diffusion probability model. Then, perform downsampling and out-of-bounds pruning. Finally, obtain the generated time series data through the inverse operation of the basic feature columns and the relational feature columns.

[0071] Specifically, after training the diffusion probability model, the trained model is used to generate data. This involves using standard Gaussian noise... Sampling yields random noise ,Will The input is fed into a pre-trained diffusion probability model, and noise is gradually reduced through the inverse learning process to generate the first time series output. This embodiment obtains samples from Gaussian noise. Its data shape is consistent with that during model training, except that the content of its nodes is random noise. And for... The columns designated as basic feature columns and relational feature columns have been specified beforehand. Therefore, the output... It also inherits the structure of these data, but the content of the nodes has been transformed from the original random noise into data similar to that in the real world.

[0072] Then, the first time series is downsampled to restore the expanded time dimension to its original length: one node is extracted every M points, preserving the original time resolution. This process makes... Transform into Proportional sampling preserves the linear correlation of the sequences, ensuring that the reduced sequence still maintains a high similarity to the target. At this point... compared to There is no change in the structure of the feature columns; the original basic feature columns and relational feature columns remain in their initial state. Only the length of the data in the time dimension has been reduced.

[0073] Then, perform out-of-bounds pruning on the downsampled relation feature columns:

[0074]

[0075] It is a length of This operation ensures a certain feature column. All relational feature columns are strictly located within the interval [−1,1], providing valid input for subsequent reconstruction. After out-of-bounds pruning, we obtained data where all nodes are located in the interval [−1, 1]. .

[0076] Finally, from Extract the basic feature column and Relationship characteristics after border crossing trimming The original target feature columns are calculated in reverse. The formula for restoring a single basic feature column is as follows:

[0077]

[0078] in yes The inverse operation.

[0079] The formula for restoring multiple basic feature columns is as follows:

[0080]

[0081]

[0082]

[0083] Concatenate all the restored feature columns to form the final time series data sample:

[0084]

[0085] After feature restoration and splicing, the data Transform into the desired time series data .

[0086] In another embodiment of this application, the total loss function used when training the diffusion probability model includes reconstruction loss and out-of-bounds loss, as shown in the following formula:

[0087]

[0088] in, For the total loss, To reconstruct the loss, For losses due to crossing the boundary, and These are the weighting coefficients.

[0089] The formula for calculating reconstruction loss is as follows:

[0090]

[0091] in, The training target, in this embodiment, is the sample without noise. ; This indicates the input samples with added noise corresponding to the model. Output ; To calculate the reconstruction loss, For mathematical expectation, This represents the square of the L1 norm. The reconstruction loss designed in this embodiment can directly compare the similarity between the model output and the target sequence.

[0092] The formula for calculating the out-of-bounds loss in this embodiment is as follows:

[0093]

[0094] in, Losses due to exceeding the upper boundary. Losses due to exceeding the lower boundary.

[0095] Loss due to exceeding upper boundary:

[0096]

[0097]

[0098] In the formula, Representative model output The Middle The first node in the nth node The numerical values ​​corresponding to the relation feature columns; That is the first The first node in the nth node The loss is calculated based on the upper bound of the values ​​corresponding to the feature columns of a relation when they exceed the bounds; n represents the number of feature columns. Represents the length of the expanded time dimension; upper bound over-bound loss. It is the upper bound loss of all nodes. The result of homogenization.

[0099] Loss due to crossing the lower boundary:

[0100]

[0101]

[0102] Similarly, the loss due to exceeding the lower bound. This means the lower bound loss of all nodes. The result of the averaging. In this embodiment, only the out-of-bounds portion is penalized; the gradient is zero within the legal interval, which does not interfere with normal fitting. The penalty strength is linearly related to the degree of out-of-bounds violation, guiding the model to make gradual corrections.

[0103] Another embodiment of this application provides a time series data generation apparatus based on a diffusion probability model, including a processor and a memory storing a plurality of computer instructions, which, when executed by the processor, implement the steps of the above method.

[0104] Specific limitations regarding the time series data generation device based on the diffusion probability model can be found in the limitations of the time series data generation method based on the diffusion probability model mentioned above, and will not be repeated here. The aforementioned time series data generation device based on the diffusion probability model can be implemented entirely or partially through software, hardware, or a combination thereof. It can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations described above.

[0105] The memory and processor are electrically connected directly or indirectly to enable data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines. The memory stores a computer program that can run on the processor, which implements the time series data generation method based on the diffusion probability model in this embodiment of the invention by running the computer program stored in the memory.

[0106] The memory may be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory stores the program, and the processor executes the program upon receiving an execution instruction.

[0107] The processor may be an integrated circuit chip with data processing capabilities. The aforementioned processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor.

[0108] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for generating time series data based on a diffusion probability model, characterized in that, The time series data generation method based on the diffusion probability model includes: Obtain the original time series data, select at least one feature column from the original time series data as the basic feature column, calculate the relationship feature columns of the remaining feature columns relative to the basic feature column, and concatenate the original time series data with all the relationship feature columns to obtain the enhanced time series data; The enhanced time series data is linearly interpolated and extended along the time axis to generate near-continuous time series data. Training samples are obtained by sampling quasi-continuous time series data, and noise is added to the training samples to train the diffusion probability model; Random noise is sampled from standard Gaussian noise and input into a trained diffusion probability model to obtain the first time series output by the diffusion probability model. Then, downsampling and out-of-bounds pruning are performed. Finally, the generated time series data is obtained through the inverse operation of the basic feature columns and the relational feature columns.

2. The time series data generation method based on the diffusion probability model as described in claim 1, characterized in that, The process involves linearly interpolating and augmenting the enhanced time series data along the time axis. When augmenting by M times, M-1 interpolation points are inserted between two adjacent nodes for each feature column.

3. The time series data generation method based on the diffusion probability model as described in claim 1, characterized in that, The diffusion probability model performs the following operations: The input features are processed by an encoder and a decoder to obtain the decoded features; The decoding features are processed through a trend extraction branch and a frequency domain extraction branch. The trend extraction branch obtains trend features by fitting a linear function, while the frequency domain extraction branch extracts frequency domain features by Fourier transform. The trend features and frequency domain features are fused by a convolutional layer to obtain the output of the diffusion probability model.

4. The time series data generation method based on the diffusion probability model as described in claim 1, characterized in that, When training the diffusion probability model, the total loss function used includes reconstruction loss and out-of-bounds loss.

5. The time series data generation method based on the diffusion probability model as described in claim 4, characterized in that, The formula for calculating the reconstruction loss is as follows: in, For samples without noise, This indicates the input samples with added noise corresponding to the model. Output , The reconstruction loss to be calculated.

6. The time series data generation method based on the diffusion probability model as described in claim 4, characterized in that, The boundary crossing loss includes the upper boundary crossing loss. and losses from crossing the lower boundary The formula for the loss due to exceeding the upper bound is: in, Indicates model output The Middle The first node in the nth node The numerical values ​​corresponding to the relation feature columns; Indicates the first The first node in the nth node The loss is calculated based on the upper bound of the values ​​corresponding to the feature columns of a relation when they exceed the bounds; n represents the number of feature columns. Indicates the length of the expanded time dimension; The formula for the lower bound out-of-bounds loss is: in, Indicates the first The first node in the nth node The loss for the lower bound of the numerical values ​​corresponding to the relation feature columns.

7. A time series data generation device based on a diffusion probability model, comprising a processor and a memory storing a plurality of computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 6.

Images