Single-variable time sequence probability prediction method and device and storage medium

Abstract

The invention discloses a univariate time sequence probability prediction method and device and a storage medium, and belongs to the technical field of industrial large models, and the method comprises the steps: carrying out the sliding window segmentation of a univariate time sequence, and obtaining an original data window sequence; performing physical constraint frequency domain enhancement operation on the original data window sequence to obtain an enhanced window sequence; extracting a time domain feature and a frequency domain feature of the enhanced window sequence and splicing the time domain feature and the frequency domain feature into a joint feature vector; inputting the joint feature vector into a pre-constructed Lags-Lama decoder architecture, outputting a hidden state vector, inputting the hidden state vector into a parameterized generalized extreme value distribution GEV output layer, and generating a probability prediction result of a future time point; according to the method, the problems of poor zero sample generalization ability, insufficient long sequence dependence modeling and data distribution offset in univariate time sequence prediction are solved.

Description

Technical Field

[0001] The present invention relates to a single variable time series probability prediction method, device and storage medium, and belongs to the technical field of industrial large models. Background Art

[0002] Traditional time series models (such as ARIMA and Prophet) need to be retrained for each data set; deep learning methods (such as DeepAR and N-BEATS) perform significantly worse on cross-domain data; existing tokenization methods (such as sliding windows) ignore the frequency domain characteristics of time series, resulting in insufficient learning of periodic patterns; absolute position coding fails in long sequence predictions, relative position coding (such as T5) has difficulty modeling cross-period dependencies, and conventional enhancement methods (such as noise injection) destroy the physical laws of time series, affecting the reliability of probabilistic predictions.

[0003] Existing methods do not address the following core contradictions: Pre-trained models have difficulty balancing cross-domain generalization and local temporal feature capture; Existing architectures lack explicit modeling of the frequency domain characteristics of time series data; Traditional positional encoding cannot adapt to variable sequence lengths in zero-shot scenarios. Summary of the Invention

[0004] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a univariate time series probability prediction method, device and storage medium to solve the problems of poor zero-sample generalization ability, insufficient long sequence dependency modeling and data distribution offset in univariate time series prediction.

[0005] To achieve the above object, the present invention is implemented by adopting the following technical solutions: In a first aspect, the present invention provides a univariate time series probability forecasting method, comprising: Perform sliding window segmentation on the univariate time series to obtain the original data window sequence; Performing a physical constraint frequency domain enhancement operation on the original data window sequence to obtain an enhanced window sequence; Extract the time domain features and frequency domain features of the enhanced window sequence and concatenate them into a joint feature vector; The joint feature vector is fed into the pre-built Lag-Llama decoder architecture, which outputs the hidden state vector. The hidden state vector is input into the parameterized generalized extreme value distribution GEV output layer to generate the probability prediction results of future time points.

[0006] Furthermore, performing a physical constraint frequency domain enhancement operation on the original data window sequence includes: Select the original data of two adjacent windows and jointly calculate FFT, randomly mask the specified high-frequency components, and then perform inverse transformation to generate enhanced samples; Alternatively, for the FFT spectrum of data from similar devices, key frequency band components are mixed according to Bernoulli distribution.

[0007] Furthermore, the step of extracting the time domain features and frequency domain features of the enhanced window sequence and concatenating them into a joint feature vector includes: Normalize the window data and calculate the statistics to obtain the time domain characteristics of the window sequence; Perform FFT on the window data, retain the amplitudes of the first K main frequency components, and filter the preset noise frequency band to obtain the frequency domain characteristics of the window sequence; The time domain features and frequency domain features of the window sequence are concatenated into a joint feature vector.

[0008] Furthermore, the time domain feature includes at least one of mean, variance, and kurtosis.

[0009] Furthermore, the Lag-Llama decoder architecture adopts a Transformer decoder, which dynamically selects time delay features based on PACF analysis through an automatic lag selection module, and applies an improved RoPE position encoding, whose rotation angle calculation introduces a time decay factor β^(t / T), where β∈(0,1), t is the current position, and T is the total length of the sequence.

[0010] Furthermore, the method also includes: optimizing the distribution parameters of the GEV output layer through the continuous hierarchical probability score CRPS loss function, and adding a distribution smoothing regularization term.

[0011] In a second aspect, the present invention provides a univariate time series probability prediction device, comprising: The segmentation module is used to perform sliding window segmentation on the univariate time series to obtain the original data window sequence; An enhancement module is used to perform a physical constraint frequency domain enhancement operation on the original data window sequence to obtain an enhanced window sequence; A splicing module is used to extract the time domain features and frequency domain features of the enhanced window sequence and splice them into a joint feature vector; The decoding module is used to input the joint feature vector into the pre-built Lag-Llama decoder architecture and output the hidden state vector; The output module is used to input the hidden state vector into the parameterized generalized extreme value distribution (GEV) output layer to generate probability prediction results for future time points.

[0012] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of any of the aforementioned methods when executed by a processor.

[0013] In a fourth aspect, the present invention provides a computer device, comprising: Memory, used to store computer programs / instructions; A processor is configured to execute the computer program / instructions to implement the steps of any of the aforementioned methods.

[0014] In a fifth aspect, the present invention provides a computer program product, comprising a computer program / instruction, which implements the steps of any of the aforementioned methods when executed by a processor.

[0015] Compared with the prior art, the present invention has the following beneficial effects: The present invention provides a univariate time series probability prediction method, device and storage medium. By performing a physical constraint frequency domain enhancement operation on an original data window sequence, the time domain features and frequency domain features of the enhanced window sequence are extracted and spliced ​​into a joint feature vector. The joint feature vector is input into a pre-built Lag-Llama decoder architecture, a latent state vector is output, and the latent state vector is input into a parameterized generalized extreme value distribution (GEV) output layer to generate a probability prediction result for a future time point. The method solves the problems of poor zero-sample generalization ability, insufficient long sequence dependency modeling and data distribution offset in univariate time series prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a flowchart of a univariate time series probability prediction method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0017] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention.

[0018] Example 1: This example introduces a single-variable time series probability prediction method, including: Perform sliding window segmentation on the univariate time series to obtain the original data window sequence; Performing a physical constraint frequency domain enhancement operation on the original data window sequence to obtain an enhanced window sequence; Extract the time domain features and frequency domain features of the enhanced window sequence and concatenate them into a joint feature vector; The joint feature vector is fed into the pre-built Lag-Llama decoder architecture, which outputs the hidden state vector. The hidden state vector is input into the parameterized generalized extreme value distribution GEV output layer to generate the probability prediction results of future time points.

[0019] The application process of the univariate time series probability prediction method provided in this embodiment specifically involves the following steps: Step S1: Time-frequency joint feature generation S1.1: Segment the univariate time series using a sliding window with a window length of L and a step size of S. S1.2: Extract time-domain features within the window: standardize the window data and calculate statistics (mean, variance, kurtosis); S1.3: Extract frequency domain features within the window: Perform FFT on the window data, retain the amplitudes of the first K main frequency components, and filter the preset noise frequency band; S1.4: Concatenate the time domain statistics and the frequency domain amplitude to generate a joint feature vector.

[0020] Step S2: Physical Constraint Frequency Domain Enhancement S2.1: Using the joint feature vector as input, perform at least one of the following enhancement operations: a) Freq-Mask: Select the original data of two adjacent windows and jointly calculate the FFT, randomly mask the specified high-frequency components, and then inverse transform to generate enhanced samples; b) Freq-Mix: For the FFT spectrum of data from similar devices, the key frequency band components are mixed according to the Bernoulli distribution; S2.2: Re-execute S1.2-S1.4 on the enhanced data to generate the enhanced joint feature vector.

[0021] Step S3: Dynamic position encoding and model processing S3.1: The joint feature vector is fed into the Lag-Llama decoder architecture, which includes an automatic lag selection module that dynamically selects time-delayed features based on PACF analysis. S3.2: Apply the improved RoPE position encoding in the Transformer layer, and introduce a time decay factor β^(t / T) into the rotation angle calculation, where β∈(0,1), t is the current position, and T is the total length of the sequence; S3.3: Process the feature sequence through a multi-layer Transformer and output the hidden state vector.

[0022] Step S4: Zero-sample probability prediction S4.1: Input the hidden state vector into the parameterized generalized extreme value distribution (GEV) prediction head, which outputs the location parameter μ, scale parameter σ, and shape parameter ξ; S4.2: Generate probabilistic predictions for future time points based on the GEV distribution; S4.3: Optimize the distribution parameters via the continuous graded probability score (CRPS) loss function and add a distribution smoothing regularization term (weight λ).

[0023] This embodiment proposes a solution using frequency domain enhancement and RoPE based on the Lag-Llama decoder architecture. The core innovations include: Lag-Llama Decoder Architecture Adopting a pure decoder Transformer, the key time delay features are dynamically captured through an automatic lag selection module (based on PACF analysis); An improved time-decayed RoPE is embedded, and a decay factor β^(t / T) is introduced into the rotation angle calculation to strengthen recent position relationships (for example, position sensitivity is improved by 40% when t ≤ 1000 steps) and suppress long-sequence noise interference.

[0024] Joint time-frequency representation and enhancement After the input sequence is segmented by the sliding window, the time domain normalized value and FFT low-frequency amplitude spectrum are extracted synchronously and spliced ​​into a joint token; Design physical constraints frequency domain enhancement: Freq-Mask: Randomly mask high-frequency components (retain >85% energy) after performing joint FFT on adjacent windows, and then inverse transform to generate enhanced data. Freq-Mix: Mixes the key frequency bands of two windows according to Bernoulli distribution.

[0025] Zero-sample probability prediction mechanism The output layer parameterizes the generalized extreme value distribution (GEV), predicts the location parameters μ, scale σ, and shape ξ, and optimizes the distribution tail through CRPS loss; The pre-training phase uses cross-domain mixed data (energy, industry, vehicle-side, etc.) to enable the model to learn common time series patterns and directly output the predicted distribution during zero-shot inference.

[0026] like Figure 1 As shown in the figure, the normalization pre-processing normalizes the input into a vector, and then inputs the original vector into the transformer layer through the linear transformation through the linear projection layer, which plays the role of adapting the downstream structure, extracting task-related features and reducing computational complexity. It is one of the standard operations in deep learning. Among them, Input is the input layer; RMSNorm is the root mean square normalization layer; Attention is the attention layer; Add&RMSNorm is the residual layer; FeedForward is the forward propagation layer.

[0027] The following describes the contents involved in the above embodiment in conjunction with a preferred embodiment.

[0028] The univariate time series probability prediction method provided in this embodiment includes the following steps: 1. Data Collection and Preprocessing (1) Data source Collect multimodal sensor data (vibration, temperature, current) from the equipment, with a sampling frequency of 1kHz, and retain historical data for the past three years; Tag data: equipment failure event records (fault type, occurrence time, maintenance records).

[0029] (2) Time-Frequency Tokenization 1) Sliding window segmentation: window length = 2048 points (about 2 seconds), step size = 512 points; 2) Temporal features: data normalization within the window (Z-score) and statistical extraction (mean, variance, kurtosis); 3) Frequency domain characteristics: a. Perform real-number FFT on the window data and extract the amplitudes of the first 50 main frequency components; b. Filter industrial noise frequency bands (such as >500Hz components).

[0030] 4) Feature concatenation: Time domain statistics (3D) + frequency domain amplitude (50D) → 53-dimensional joint token vector.

[0031] 2. Model training and enhancement (1) Frequency domain data enhancement 1) Freq-Mask (enabled during training): a. Randomly select two adjacent windows and jointly calculate FFT; b. Covering the 50-100Hz frequency band (common mechanical wear characteristic frequency band), coverage ratio = 40%; c. Inverse FFT generates enhanced samples, preserving physical interpretability.

[0032] 2) Freq-Mix: a. Calculate FFT for the data of similar equipment (such as pump A and pump B) separately; b. Replace the blade pass frequency (BPF) component of pump A with the spectrum of pump B to simulate early fault characteristics.

[0033] (2) Model configuration 1) Lag-Llama architecture: number of layers = 6, number of heads = 8, hidden dimension = 512; 2) RoPE improvement parameters: base=10000, β=0.95 (decay 5% every 1000 steps); 3) Output layer: GEV distribution parameter prediction head (μ, σ, ξ).

[0034] (3) Training strategy 1) Pre-training data: mixed data of 10 types of industrial equipment (motors, bearings, gearboxes); 2) Loss function: CRPS + distribution smoothing regularization term (λ = 0.1); 3) Batch size = 256, initial learning rate = 3e-5, number of training epochs = 100.

[0035] 3. Zero-shot Inference and Maintenance Decision (1) Real-time Prediction Input the latest 2048-point sensor data to generate time-frequency tokens; The model outputs the future 24-hour failure probability distribution (GEV parameters); Calculate the failure risk index: P(X≥threshold) × maintenance urgency coefficient.

[0036] (2) Dynamic Maintenance Strategy High risk (P>80%): Immediately stop the machine for maintenance and trigger spare part allocation; Medium risk (30%<P≤80%): Prioritize processing during the next week's maintenance window; Low risk (P≤30%): Include in the monthly planned inspection.

[0037] This embodiment provides a univariate time series probability prediction method. By performing a physically constrained frequency domain enhancement operation on the original data window sequence, extracting the time domain features and frequency domain features of the enhanced window sequence and splicing them into a joint feature vector, inputting the joint feature vector into a pre-constructed Lag-Llama decoder architecture, outputting a hidden state vector, and inputting the hidden state vector into a parameterized Generalized Extreme Value Distribution (GEV) output layer to generate the probability prediction result for future time points, it solves the problems of poor zero-shot generalization ability, insufficient modeling of long sequence dependencies, and data distribution shift in univariate time series prediction.

[0038] Embodiment 2. This embodiment provides a univariate time series probability prediction device, including: A segmentation module, used for performing sliding window segmentation on the univariate time series to obtain the original data window sequence; An enhancement module, used for performing a physically constrained frequency domain enhancement operation on the original data window sequence to obtain the enhanced window sequence; A splicing module, used for extracting the time domain features and frequency domain features of the enhanced window sequence and splicing them into a joint feature vector; A decoding module, used for inputting the joint feature vector into a pre-constructed Lag-Llama decoder architecture and outputting a hidden state vector; An output module, used for inputting the hidden state vector into a parameterized Generalized Extreme Value Distribution (GEV) output layer to generate the probability prediction result for future time points.

[0039] For the specific functional implementation of the above modules, refer to the relevant content in the method of Embodiment 1 and will not be elaborated here.

[0040] Embodiment 3: This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of any one of the methods described in Embodiment 1 are implemented.

[0041] Embodiment 4: This embodiment provides a computer device, including: Memory, used to store computer programs / instructions; A processor, configured to execute the computer program / instructions to implement the steps of any one of the methods described in Example 1.

[0042] Example 5: This embodiment provides a computer program product, including a computer program / instruction, which implements the steps of any method described in Example 1 when executed by a processor.

[0043] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

[0044] Those skilled in the art will appreciate that embodiments of the present disclosure may be provided as methods, systems, or computer program products. Thus, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROMs, optical storage, etc.) containing computer-usable program code.

[0045] The present disclosure is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0046] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0047] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0048] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure and are not intended to limit its scope of protection. Although the present disclosure has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that after reading the present disclosure, those skilled in the art can still make various changes, modifications or equivalent substitutions to the specific implementation methods of the invention, but these changes, modifications or equivalent substitutions are all within the scope of protection of the disclosed claims to be approved.

Claims

1. A univariate time series probability forecasting method, characterized in that: include: Perform sliding window segmentation on the univariate time series to obtain the original data window sequence; Performing a physical constraint frequency domain enhancement operation on the original data window sequence to obtain an enhanced window sequence; Extract the time domain features and frequency domain features of the enhanced window sequence and concatenate them into a joint feature vector; The joint feature vector is fed into the pre-built Lag-Llama decoder architecture, which outputs the hidden state vector. The hidden state vector is input into the parameterized generalized extreme value distribution GEV output layer to generate the probability prediction results of future time points.

2. The univariate time series probability forecasting method according to claim 1, characterized in that: The performing of a physical constraint frequency domain enhancement operation on the original data window sequence includes: Select the original data of two adjacent windows and jointly calculate FFT, randomly mask the specified high-frequency components, and then perform inverse transformation to generate enhanced samples; Alternatively, for the FFT spectrum of data from similar devices, key frequency band components are mixed according to Bernoulli distribution.

3. The univariate time series probability forecasting method according to claim 1, characterized in that: The step of extracting the time domain features and frequency domain features of the enhanced window sequence and splicing them into a joint feature vector includes: Normalize the window data and calculate the statistics to obtain the time domain characteristics of the window sequence; Perform FFT on the window data, retain the amplitudes of the first K main frequency components, and filter the preset noise frequency band to obtain the frequency domain characteristics of the window sequence; The time domain features and frequency domain features of the window sequence are concatenated into a joint feature vector.

4. The univariate time series probability forecasting method according to claim 3, characterized in that: The time domain feature includes at least one of mean, variance, and kurtosis.

5. The univariate time series probability forecasting method according to claim 1, characterized in that: The Lag-Llama decoder architecture adopts the Transformer decoder, which dynamically selects time delay features based on PACF analysis through an automatic lag selection module and applies an improved RoPE position encoding. Its rotation angle calculation introduces a time decay factor β^(t / T), where β∈(0,1), t is the current position, and T is the total length of the sequence.

6. The univariate time series probability forecasting method according to claim 1, characterized in that: The method further includes optimizing the distribution parameters of the GEV output layer through a continuous hierarchical probability score (CRPS) loss function and adding a distribution smoothing regularization term.

7. A univariate time series probability prediction device, characterized in that: include: The segmentation module is used to perform sliding window segmentation on the univariate time series to obtain the original data window sequence; An enhancement module is used to perform a physical constraint frequency domain enhancement operation on the original data window sequence to obtain an enhanced window sequence; A splicing module is used to extract the time domain features and frequency domain features of the enhanced window sequence and splice them into a joint feature vector; The decoding module is used to input the joint feature vector into the pre-built Lag-Llama decoder architecture and output the hidden state vector; The output module is used to input the hidden state vector into the parameterized generalized extreme value distribution (GEV) output layer to generate probability prediction results for future time points.

8. An electronic device, characterized in that: include: Memory, used to store computer programs / instructions; A processor, configured to execute the computer program / instructions to implement the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images