Time sequence prediction system and method based on expert denoising

Abstract

The invention relates to the technical field of time series prediction, and discloses a time series prediction system and method based on expert denoising, and the method comprises the steps: reading an online database, obtaining historical time series data, and building a conditional diffusion model frame which comprises a time context embedding model and a training model, and inputting the historical time series data into a conditional diffusion model, and obtaining historical data characterization. According to the time series prediction system and method based on expert denoising provided by the invention, through establishing a conditional diffusion model framework, after inputting the historical time series data, a complex time series mode is effectively captured, and time-rich characterization is extracted; meanwhile, time characterization is combined with time context embedding, a multi-expert de-noising device is adopted to carry out segmented noise adding and de-noising, and each time of de-noising is refined, so that the model capacity can be remarkably improved while the high reasoning speed can be kept, and then the prediction precision under different noise conditions is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of time series prediction, and in particular to a time series prediction system and method based on expert denoising. Background Art

[0002] Time series forecasting refers to using the acquired data to arrange them in chronological order, analyzing their direction and degree of change, and thus speculating on the levels that may be achieved in several future periods. The basic idea of ​​time series forecasting is to treat the time series as a sample of a random variable and use probability and statistical methods to minimize the impact of accidental factors.

[0003] As a commonly used time series prediction method, the time series prediction method based on the diffusion model still has some limitations. For example, it lacks an effective mechanism and it is difficult to extract information-rich representations from historical data, which limits the model's ability to understand complex time series patterns. In addition, the calculation complexity is high, making it difficult to increase the capacity of the model while maintaining an efficient reasoning speed. At the same time, there is a lack of targeted processing for data with different noise levels, resulting in less than ideal prediction accuracy in some cases. Summary of the invention

[0004] In view of the deficiencies of the prior art, the present invention provides a time series prediction system and method based on expert denoising, which solves the technical problems of targeted processing of different noise levels and unsatisfactory prediction accuracy in the prior art.

[0005] In view of the above problems, the present invention provides a time series prediction system and method based on expert denoising.

[0006] In the first aspect, the present invention provides a time series prediction method based on expert denoising, the method comprising: reading an online database to obtain historical time series data, the historical time series data comprising real-world time series data, sinusoidal complex time series and exponential complex time series, establishing a conditional diffusion model framework, the conditional diffusion model framework comprising: a time context embedding model and a training model, inputting the historical time series data into the conditional diffusion model to obtain historical data representation.

[0007] Preferably, the conditional diffusion model framework also includes: a projection layer, which uses linear transformation to map data points in low-dimensional space to high-dimensional space; position encoding, which is used to inject timing information of time series data points; adaptive layer normalization, which is used to reduce internal covariate shift and adapt to diverse data distribution; a state space model (SSM) block, which uses the Mamba-2 architecture and can be stacked to adapt to data complexity; a reparameterization technique, which transforms the sampling method of random variables so that the gradient is back-propagated through the sampling process; a linear layer, which is used to predict the time series mean and time series logarithmic variance of the latent distribution; outputting the latent representation through the conditional diffusion model, obtaining the output information, and using it as a historical data representation.

[0008] Preferably, the historical data representation is input into a time context embedding model, and the time context embedding model combines the historical data representation in a weighted manner and is used to encode time information to obtain a first comprehensive indicator.

[0009] Preferably, an online database is read to obtain a universal denoiser group and a pre-trained model, the universal denoiser group is initialized using the pre-trained model, and an initialized universal denoiser group is obtained.

[0010] Preferably, a low-segment expert denoiser is obtained, and the method includes: reading an online database to obtain low noise, fine-tuning a denoiser in an initialized general denoiser group according to the low noise, obtaining a low-segment expert denoiser, inputting the low noise into the low-segment expert denoiser, using the low-segment expert denoiser to denoise the low noise, and obtaining a first low-noise sequence indicator in combination with a first comprehensive indicator.

[0011] Preferably, a mid-segment expert denoiser is obtained, and the method includes: obtaining medium noise after the low noise is denoised by the low-segment expert denoiser, fine-tuning a denoiser in the initialized general denoiser group according to the medium noise, obtaining a mid-segment expert denoiser, inputting the medium noise into the mid-segment expert denoiser, denoising the medium noise using the mid-segment expert denoiser, and obtaining a first medium noise sequence indicator in combination with a first comprehensive indicator.

[0012] Preferably, a high-segment expert denoiser is obtained, and the method includes: obtaining high noise after the medium noise is denoised by the medium-segment expert denoiser, fine-tuning a denoiser in the initialized general denoiser group according to the high noise to obtain the high-segment expert denoiser, inputting the high noise into the high-segment expert denoiser, and obtaining a high noise sequence index in combination with the first comprehensive index.

[0013] Preferably, the prediction result is obtained, and the method includes: inputting the high-noise sequence indicator into the training model, denoising and gradually restoring the signal, and combining the first comprehensive indicator to obtain the second medium-noise sequence indicator through training, inputting the second medium-noise sequence indicator into the training model, denoising and gradually restoring the signal, and combining the first comprehensive indicator to obtain the second low-noise sequence indicator through training, inputting the second low-noise sequence indicator into the training model, denoising and gradually restoring the original signal, and combining the first comprehensive indicator to obtain the second comprehensive indicator through training, and using the second comprehensive indicator as the time series prediction result.

[0014] In a second aspect, a time series prediction system based on expert denoising is provided, the system comprising: a first reading module, the reading module is used to read an online database to obtain historical time series data, the historical time series data includes real-world time series data, sinusoidal complex time series and exponential complex time series; an extraction module, the extraction module, based on a conditional diffusion model and historical time series data, captures complex time series patterns, extracts rich representations of time information, and uses them as historical data representations; an encoding module, the time encoding module comprises a time context embedding model, the time context embedding model combines historical data representations with time context embedding to obtain a first comprehensive indicator;

[0015] A second reading module, the second reading module is used to read the online database to obtain the low noise and general denoiser groups; a prediction module, the prediction module is used for time series prediction, wherein a general denoiser in the general denoiser group is fine-tuned to obtain a low-segment expert denoiser, and after adding the low noise to the low-segment expert denoiser, a first low noise sequence index and a medium noise are obtained in combination with the first comprehensive index;

[0016] Among them, after a universal denoiser in the universal denoiser group is fine-tuned according to the medium noise, a mid-range expert denoiser is obtained, and after the mid-range expert denoiser is added to the medium noise, the first medium noise sequence index and the high noise are obtained by combining the first comprehensive index; wherein, after a universal denoiser in the universal denoiser group is fine-tuned according to the high noise, a high-range expert denoiser is obtained, and after the high-range expert denoiser is input into the high noise, the high noise sequence index is obtained by combining the first comprehensive index; the high-range expert denoiser is input into the training model, the high noise is denoised, and the signal is gradually restored, and then the medium noise and the second medium noise index are obtained by combining the first comprehensive index; the mid-range expert denoiser is input into the training model, the medium noise is denoised, and the signal is gradually restored, and then the low noise and the second low noise index are obtained by combining the first comprehensive index; the low-range expert denoiser is input into the training model, the low noise is denoised, and the original signal is gradually restored, and then the second comprehensive index is obtained by combining the first comprehensive index, and the second comprehensive index is used as the prediction result.

[0017] One or more technical solutions provided in the present invention have at least the following technical effects or advantages:

[0018] The present invention provides a time series prediction system and method based on expert denoising. By establishing a conditional diffusion model framework, after inputting historical time series data, the system can effectively capture complex time series patterns and extract rich time representations. At the same time, the time representations are embedded in the time context, and multiple expert denoisers are used to perform segmented denoising and denoising. Each denoising is refined and subdivided, which can maintain a high-speed inference speed while significantly improving the model capacity, thereby improving the prediction accuracy under different noise conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 The present invention provides a flow chart of a time series prediction method based on expert denoising;

[0020] Figure 2 The present invention provides a schematic diagram of the structure of a time series prediction system based on expert denoising.

[0021] Explanation of the accompanying drawings: 1. First reading module; 2. Extraction module; 3. Encoding module; 4. Second reading module; 5. Prediction module. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] Embodiment 1:

[0024] like Figure 1 As shown, an embodiment of the present invention provides a time series prediction method based on expert denoising, the method comprising:

[0025] Step A100: read the online database to obtain historical time series data, where the historical time series data includes real-world time series data and sinusoidal and exponential complex time series;

[0026] In the present invention, a time series prediction method based on expert denoising provided by an embodiment of the present invention is applied to a time series prediction system based on expert denoising. In order to ensure more accurate time series prediction in the later stage, it is necessary to obtain historical time series data by reading an online database. The historical time series data may include economic data, meteorological data, sales data, population data, traffic data, energy data, etc. The historical time series data can be obtained through network data crawling, data trading platforms, official statistical agencies, professional databases, international data archives, etc., which provides sufficient data support for the later extraction of historical data representation.

[0027] Step A200: Establishing a conditional diffusion model framework, the conditional diffusion model framework includes: a time context embedding model and a training model, inputting the historical time series data into the conditional diffusion model to obtain historical data representation;

[0028] In order to effectively extract historical data representation and perform time series prediction in the present invention, it is necessary to establish a conditional diffusion model framework to be compatible with different model architectures. The framework includes a time context embedding model and a training model, wherein the conditional diffusion model framework also includes a projection layer, a position encoding, an adaptive layer normalization, a state space model (SSM) block, a reparameterization technique, and a linear layer. The linear layer is used to predict the mean and logarithmic variance of the latent distribution, output the latent representation through the conditional diffusion model, obtain the output information, and use it as the historical data representation;

[0029] Furthermore, step A200 of the present invention further includes:

[0030] Step A210: Through the projection layer, the projection layer converts the input data into a high-dimensional space by calling the linear layer of the torch library, that is, calling the torch.nn.linear function to convert the input historical time series data into a high-dimensional space in order to learn complex time series patterns;

[0031] Step A220: Position coding, wherein the position coding generates a unique position vector for each position in the sequence, obtains the time series information of the time series data point, injects the time series information of the time series data point, and then performs position coding to save the time sequence information, that is, generates a unique position vector for each time step input in the time series;

[0032] The specific formula is:

[0033] PE_(pos,2i)=sin(pos / 10000^{2i / d_{model}})

[0034] PE_(pos,2i+1)=cos(pos / 10000^{2i / d_{model}})

[0035]

[0036] Step A230: Adaptive layer normalization. For the historical time series data and the time series information of the time series data points after dimensionality increase, the offset of the internal covariates can be reduced by adaptive layer normalization, so as to better adapt to the diverse data distribution, thereby improving the performance and generalization ability of the model. At the same time, adaptive layer normalization can make the model training more stable.

[0037] Reduce oscillation during training and speed up model convergence;

[0038] Furthermore, after processing the data at each layer, the data is standardized into data with a mean of 0 and a variance of 1 to facilitate subsequent processing. The specific algorithm is to first subtract the mean of the original data and then divide it by the standard deviation of the original data.

[0039] Step A240: A state space model (SSM) block, based on the original SSM model block, can be used to establish a more connected general framework by using the Mamba-2 architecture, that is, the connection of each framework in the conditional diffusion model. Compared with Mamba-1, it can support a larger state dimension. In the present invention, different frameworks can be stacked to adapt to the complexity of the data;

[0040] Specifically, the state space model (SSM) block utilizes the stackability of the Mamba-2 architecture, that is, the output of the previous Mamba-2 architecture is used as the input of the next Mamba-2 architecture, and can be stacked to adapt to data complexity;

[0041] Step A250: Reparameterization technique can transform the parameters and operations of the conditional diffusion model in the present invention, so that the model has different forms during training and reasoning, but the final results remain consistent, thereby improving the stability and efficiency of the training model, and then improving the accuracy of time series prediction;

[0042] Specifically, by changing the sampling method of random variables, the gradient can be directly back-propagated through the sampling process, which improves the computational efficiency and accelerates the convergence speed. It can be used to achieve efficient gradient back-propagation in the variational inference process.

[0043] Step A260: The linear layer can predict the mean and logarithmic variance of the latent distribution and output the latent historical data representation through the conditional diffusion model. The specific formula is as follows:

[0044]

[0045] Step A300: Input the latent historical data representation output by the conditional diffusion model into the time context embedding model. The time context embedding model combines the historical data representation in a weighted manner and is used to encode the time information to obtain the first comprehensive index, and provide time guidance for the noise addition and denoising training in the later training model, thereby improving the accuracy of time series prediction.

[0046] Step 400: read an online database, obtain a universal denoiser group and a pre-trained model, initialize the universal denoiser using the pre-trained model, and obtain an initialized universal denoiser group;

[0047] Among them, online databases can include OpenCV libraries or cloud services such as Amazon AWS and Microsoft Azure;

[0048] The pre-trained model is a pre-trained model trained on a large-scale data set. The universal denoiser group is initialized through the pre-trained model to obtain an initialized universal denoiser group.

[0049] Step 500: read the online database to obtain low noise;

[0050] Among them, online databases may include LAION-5B, ImageNet, etc.

[0051] Specifically, the low noise obtained is Gaussian noise with a mean of 0 and a variance of 1;

[0052] Furthermore, step A500 of the present invention further includes:

[0053] Step A510: By using the pre-trained model, a universal denoiser in the initialized universal denoiser group is fine-tuned for the obtained low noise to obtain a low-segment expert denoiser, the low noise is input into the low-segment expert denoiser, the low noise is denoised by the low-segment expert denoiser, and the first low noise sequence indicator is obtained through training in combination with the first comprehensive indicator. The low-segment expert denoiser is responsible for the input and output of the low noise, so that the signal restored by the low-segment expert denoiser training is more accurate, and the result of time series prediction is more accurate.

[0054] Step 600: The low noise is denoised by the low-segment expert denoiser to obtain medium noise;

[0055] Furthermore, step A600 of the present invention further includes:

[0056] Step A610: Through the pre-training model, a denoiser in the initialized general denoiser group is fine-tuned for the obtained medium noise to obtain a mid-segment expert denoiser, the medium noise is input into the mid-segment expert denoiser, the mid-segment expert denoiser is used to denoise the medium noise, and combined with the first comprehensive indicator, the first medium noise sequence indicator is obtained through training. The mid-segment expert denoiser is responsible for the input and output of the medium noise, so that the signal restored by the mid-segment expert denoiser training is more accurate, and the result of time series prediction is more accurate.

[0057] Step 700: the medium noise is denoised by the mid-segment expert denoiser to obtain high noise;

[0058] Furthermore, step A700 of the present invention further includes:

[0059] Step A710: Through the pre-training model, a denoiser in the initialized general denoiser group is fine-tuned for the acquired high noise to obtain a high-segment expert denoiser, the high noise is input into the high-segment expert denoiser, and the high noise sequence index is obtained through training in combination with the first comprehensive index. The high-segment expert denoiser is responsible for the input and output of the high noise, so that the signal restored by the high-segment expert denoiser is more accurate, and the result of time series prediction is more accurate.

[0060] Step A800: the high noise sequence indicator is input into the training model, denoised and gradually restored the signal, and combined with the first comprehensive indicator, the second noise sequence indicator is trained to obtain the second low noise sequence indicator, the second low noise sequence indicator is input into the training model, denoised and gradually restored the signal, and combined with the first comprehensive indicator, the second comprehensive indicator is trained to obtain the second low noise sequence indicator, the second low noise sequence indicator is input into the training model, denoised and gradually restored the original signal, and combined with the first comprehensive indicator, the second comprehensive indicator is obtained, and the second comprehensive indicator is used as the time series prediction result;

[0061] By establishing a low-segment expert denoiser, a mid-segment expert denoiser and a high-segment expert denoiser, and by using multiple expert denoisers to work together, the gradual denoising and denoising of a section of noise are jointly performed, thereby reducing the amount of calculation of each expert denoiser, thereby making the calculation prediction of the low-segment expert denoiser, the mid-segment expert denoiser and the high-segment expert denoiser more accurate;

[0062] Furthermore, by using multiple expert denoisers, the present invention can efficiently train, infer and predict, while improving the computing capacity of the model.

[0063] Embodiment 2:

[0064] Based on the same inventive concept of a time series prediction method based on expert denoising in the above embodiment 1, Figure 2As shown, the present invention provides a time series prediction system based on expert denoising, the system comprising:

[0065] A first reading module 1, the reading module is used to read the online database to obtain historical time series data, the historical time series data includes real-world time series data, sinusoidal complex time series and exponential complex time series;

[0066] Extraction module 2, the extraction module, based on the conditional diffusion model framework and historical time series data, captures complex time series patterns, extracts rich representations of time information, and uses them as historical data representations;

[0067] Coding module 3, the time coding module includes a time context embedding model, the time context embedding model combines historical data representation and time context embedding to obtain a first comprehensive indicator;

[0068] A second reading module 4, the second reading module is used to read the online database and the online database to obtain the low noise and universal denoiser groups;

[0069] Prediction module 5, the prediction module is used for real-world time series prediction;

[0070] Further, a universal denoiser in the universal denoiser group obtains a low-segment expert denoiser for the obtained low noise after fine-tuning the pre-trained model, and after adding the low-segment expert denoiser to the low noise, the first low-noise sequence index and the medium noise are obtained through training in combination with the first comprehensive index;

[0071] Further, a universal denoiser in the universal denoiser group obtains a mid-range expert denoiser for the obtained medium noise after fine-tuning the pre-trained model, and after adding the mid-range expert denoiser to the medium noise, the first medium noise sequence index and high noise are obtained through training in combination with the first comprehensive index;

[0072] Furthermore, a general denoiser in the general denoiser group, for the acquired high noise, is fine-tuned through a pre-trained model to acquire a high-segment expert denoiser, and after inputting the high noise into the high-segment expert denoiser, the high noise is combined with the first comprehensive indicator to train and acquire a high noise sequence indicator;

[0073] Furthermore, the high-segment expert denoiser is input into the training model to denoise the high noise, and after gradually restoring the signal, the first comprehensive indicator is combined to obtain the medium noise and the second medium noise indicators. The medium-segment expert denoiser is input into the training model to denoise the medium noise, and after gradually restoring the signal, the first comprehensive indicator is combined to obtain the low noise and the second low noise indicators. The low-segment expert denoiser is input into the training model to denoise the low noise, and after gradually restoring the original signal, the first comprehensive indicator is combined to obtain the second comprehensive indicator, and the second comprehensive indicator is used as the prediction result.

[0074] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A time series prediction method based on expert denoising, characterized by: The method comprises: Reading an online database to obtain historical time series data, wherein the historical time series data includes real-world time series data, sinusoidal complex time series, and exponential complex time series; A conditional diffusion model framework is established, wherein the conditional diffusion model framework includes: a time context embedding model and a training model, and the historical time series data is input into the conditional diffusion model to obtain historical data representation.

2. The method according to claim 1, characterized in that: The conditional diffusion model framework also includes: A projection layer, wherein the projection layer uses a linear transformation to map data points in a low-dimensional space to a high-dimensional space; Position coding, the position coding is used to inject timing information of time series data points; Adaptive layer normalization, which is used to reduce internal covariate shift and adapt to diverse data distribution; A state space model (SSM) block, which utilizes the Mamba-2 architecture and is stackable to accommodate data complexity; The reparameterization technique transforms the sampling method of random variables so that the gradient is back-propagated through the sampling process; A linear layer, which is used to predict the time series mean and logarithmic variance of the latent distribution; The latent representation is output through the conditional diffusion model, the output information is obtained, and it is used as the historical data representation.

3. The method according to claim 2, characterized in that: The historical data representation is input into the time context embedding model, and the time context embedding model combines the historical data representation in a weighted manner and is used to encode time information to obtain a first comprehensive indicator.

4. The method according to claim 3, characterized in that: An online database is read to obtain a universal denoiser group and a pre-trained model, and the universal denoiser group is initialized using the pre-trained model to obtain an initialized universal denoiser group.

5. The method according to claim 4, characterized in that: Obtaining a low-segment expert denoiser, the method comprising: An online database is read to obtain low noise. According to the low noise, a denoiser in the initialization general denoiser group is fine-tuned to obtain a low-segment expert denoiser. The low noise is input into the low-segment expert denoiser, and the low noise is denoised using the low-segment expert denoiser. Combined with the first comprehensive index, a first low noise sequence index is obtained.

6. The method according to claim 5, characterized in that: Obtain a mid-segment expert denoiser, the method comprising: obtaining a medium noise after the low noise is denoised by the low-segment expert denoiser, fine-tuning a denoiser in an initialization general denoiser group according to the medium noise, obtaining a mid-segment expert denoiser, inputting the medium noise into the mid-segment expert denoiser, denoising the medium noise using the mid-segment expert denoiser, and obtaining a first medium noise sequence indicator in combination with a first comprehensive indicator.

7. The method according to claim 6, characterized in that: Obtaining a high-end expert denoiser, the method comprising: After the medium noise is denoised by the medium-segment expert denoiser, high noise is obtained. According to the high noise, a denoiser in the initialization general denoiser group is fine-tuned to obtain a high-segment expert denoiser. The high noise is input into the high-segment expert denoiser, and combined with the first comprehensive indicator, a high noise sequence indicator is obtained.

8. The method according to claim 7, characterized in that: Obtaining prediction results, the method includes: The high-noise sequence indicator is input into the training model, denoised and gradually restored the signal, and combined with the first comprehensive indicator, the second medium-noise sequence indicator is trained to obtain the second low-noise sequence indicator, the second low-noise sequence indicator is input into the training model, denoised and gradually restored the signal, and combined with the first comprehensive indicator, the second comprehensive indicator is trained to obtain the second comprehensive indicator, and the second comprehensive indicator is used as the time series prediction result.

9. A time series prediction system based on expert denoising, characterized by: The system comprises: A first reading module, the reading module is used to read the online database to obtain historical time series data, the historical time series data includes sinusoidal complex time series and exponential complex time series; An extraction module, wherein the extraction module captures complex time series patterns based on a conditional diffusion model and historical time series data, extracts rich representations of time information, and uses the representations as historical data; An encoding module, wherein the time encoding module comprises a time context embedding model, wherein the time context embedding model combines historical data representation with time context embedding to obtain a first comprehensive indicator; A second reading module, the second reading module is used to read the online database to obtain a low-noise and universal denoiser group; A prediction module, wherein the prediction module is used for time series prediction, wherein a general denoiser in the general denoiser group is fine-tuned to obtain a low-segment expert denoiser, and after adding low noise to the low-segment expert denoiser, a first low-noise sequence index and a medium noise are obtained by combining the first comprehensive index; Among them, a universal denoiser in the universal denoiser group is fine-tuned according to the medium noise to obtain a mid-range expert denoiser, and after adding the medium noise to the mid-range expert denoiser, a first medium noise sequence index and high noise are obtained by combining the first comprehensive index; Among them, a general denoiser in the general denoiser group is fine-tuned according to the high noise to obtain a high-segment expert denoiser, and after the high noise is input into the high-segment expert denoiser, the high noise sequence index is obtained by combining the first comprehensive index; The high-segment expert denoiser is input into the training model to denoise the high noise, and after gradually restoring the signal, the first comprehensive indicator is combined to obtain the medium noise and the second medium noise indicators. The medium-segment expert denoiser is input into the training model to denoise the medium noise, and after gradually restoring the signal, the first comprehensive indicator is combined to obtain the low noise and the second low noise indicators. The low-segment expert denoiser is input into the training model to denoise the low noise, and after gradually restoring the original signal, the first comprehensive indicator is combined to obtain the second comprehensive indicator, and the second comprehensive indicator is used as the prediction result.

Images