Method and device for generating prediction data, storage medium, and electronic device

Abstract

This application provides a method, apparatus, storage medium, and electronic device for generating prediction data. The method includes: obtaining global features of target time series data based on a first feature extraction layer of a prediction model, and obtaining local features of the target time series data based on a second feature extraction layer of the prediction model; performing weighted fusion of the global features and the local features according to a parameter matrix to obtain comprehensive features of the target time series data; and controlling the prediction model to generate prediction data corresponding to the target time series data based on the comprehensive features.

Description

Technical Field

[0001] This application relates to the field of data analysis, and more specifically, to a method and apparatus for generating predictive data, a storage medium, and an electronic device. Background Technology

[0002] In the field of time series analysis and forecasting, deep learning models have been widely applied in various scenarios, such as financial market forecasting, equipment health monitoring, and environmental data analysis. However, despite their powerful capabilities in processing time series data, their limitations in feature extraction still restrict the improvement of prediction accuracy.

[0003] While deep learning models have advantages in capturing long-sequence dependencies through their gating mechanisms, their internal state update process may lead to excessive compression of information when dealing with complex and high-dimensional time series data. This results in the model not having rich enough feature representation of the data, especially when dealing with nonlinear relationships and multimodal data. It may miss some key features, thus affecting the accuracy of predictions.

[0004] There is currently no effective solution to the problem that existing technologies fail to provide sufficiently rich features for time series data, resulting in low accuracy of the output prediction data.

[0005] Therefore, it is necessary to improve the relevant technology to overcome the aforementioned defects. Summary of the Invention

[0006] This application provides a method and apparatus for generating prediction data, a storage medium, and an electronic device to at least solve the problem in the prior art where the features of the time series data determined by the model are not rich enough, resulting in low accuracy of the output prediction data.

[0007] According to one embodiment of this application, a method for generating prediction data is provided, comprising: obtaining global features of target time series data based on a first feature extraction layer of a prediction model, and obtaining local features of the target time series data based on a second feature extraction layer of the prediction model; performing weighted fusion of the global features and the local features according to a parameter matrix to obtain comprehensive features of the target time series data; and controlling the prediction model to generate prediction data corresponding to the target time series data according to the comprehensive features.

[0008] In an exemplary embodiment, the first feature extraction layer based on the prediction model obtains global features of target time-series data, including: adding positional encoding information to the target time-series data and performing a linear transformation on the target time-series data after adding positional encoding information to obtain a target matrix, wherein the target matrix includes: a query matrix, a key matrix, and a matrix; dividing the target matrix to input the divided target matrix into multiple attention layers respectively, wherein the first feature extraction layer includes: the multiple attention layers; obtaining the attention output by each attention layer based on the divided target matrix, and obtaining global features of the target time-series data based on the attention.

[0009] In an exemplary embodiment, obtaining local features of the target time series data based on the second feature extraction layer of the prediction model includes: modeling the global features through a bidirectional memory network to obtain local features of the target time series data, wherein the second feature extraction layer includes: the bidirectional memory network.

[0010] In an exemplary embodiment, before obtaining the global features of the target time series data based on the first feature extraction layer of the prediction model, the method further includes: pre-training the original model corresponding to the prediction model based on the original training data to obtain the pre-trained model; performing noise processing on the original training data to obtain noisy data; and performing comparative learning training on the pre-trained model based on the noisy data and the original training data to obtain the prediction model.

[0011] In an exemplary embodiment, pre-training the original model corresponding to the prediction model based on the original training data to obtain the pre-trained model includes: constructing a first loss function corresponding to the original model based on a numerical error loss function, a distribution matching loss function, and a frequency domain consistency loss function; and pre-training the original model based on the original training data and the first loss function to obtain the pre-trained model.

[0012] In one exemplary embodiment, noise processing is performed on the original training data to obtain noise data, including: determining the total number of outliers corresponding to the original training data, and determining multiple location indices in the original training data based on the total number of outliers; randomly generating target noise, and replacing the data corresponding to the multiple location indices with the target noise to generate the noise data.

[0013] In an exemplary embodiment, the pre-trained model is subjected to comparative learning training based on the noise data and the original training data to obtain the prediction model, including: constructing a second loss function corresponding to the pre-trained model according to a numerical error loss function, a distribution matching loss function, a temporal basic loss function, and an adversarial loss function; and performing comparative learning training on the pre-trained model based on the noise data, the original training data, and the second loss function to obtain the prediction model.

[0014] According to another embodiment of this application, a predictive data generation apparatus is provided, comprising: an acquisition module, configured to acquire global features of target time series data based on a first feature extraction layer of a predictive model, and acquire local features of the target time series data based on a second feature extraction layer of the predictive model; a fusion module, configured to perform weighted fusion of the global features and the local features according to a parameter matrix to obtain comprehensive features of the target time series data; and a generation module, configured to control the predictive model to generate predictive data corresponding to the target time series data based on the comprehensive features.

[0015] According to yet another embodiment of this application, a computer-readable storage medium is also provided, wherein a computer program is stored therein, and the computer program is configured to perform the steps in any of the above method embodiments when it is run.

[0016] According to yet another embodiment of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0017] According to yet another embodiment of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0018] This application utilizes a first feature extraction layer of a prediction model to obtain global features of the target time-series data, and a second feature extraction layer of the prediction model to obtain local features of the target time-series data. The global and local features are then weighted and fused using a parameter matrix to obtain a comprehensive feature representation of the target time-series data. The prediction model is then controlled to generate predicted data corresponding to the target time-series data based on the comprehensive features. In this embodiment, the first feature extraction layer of the prediction model obtains global features of the target time-series data, and the second feature extraction layer captures local features of the target time-series data. The global and local features are then weighted and fused using a parameter matrix to obtain a comprehensive feature representation of the target time-series data. The prediction model is then controlled to generate predicted data based on the comprehensive features. In other words, by combining multi-level feature extraction with an intelligent fusion strategy, the model's ability to grasp the features of time-series data is improved, thereby solving the problem that the model's determination of the time-series data's features is insufficient, leading to low accuracy in the output predicted data. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a hardware structure block diagram of a computer device for a method of generating predictive data according to an embodiment of this application;

[0022] Figure 2 This is a flowchart of a method for generating prediction data according to an embodiment of this application;

[0023] Figure 3 This is a flowchart of training a prediction model according to an embodiment of this application;

[0024] Figure 4 This is a network structure diagram of the prediction model according to an embodiment of this application;

[0025] Figure 5 This is a structural block diagram of a predictive data generation apparatus according to an embodiment of this application. Detailed Implementation

[0026] The embodiments of this application will be described in detail below with reference to the accompanying drawings and examples.

[0027] It should be noted that the terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0028] The methods and embodiments provided in this application can be executed in a computer device or similar computing device. Taking running on a computer device as an example, Figure 1 This is a hardware structure block diagram of a computer device for a method of generating predictive data according to an embodiment of this application. Figure 1 As shown, a computer device may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The computer device may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the computer device described above. For example, the computer device may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0029] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the method for generating predictive data in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thus implementing the aforementioned method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to computer devices via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0030] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider for the computer equipment. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0031] This embodiment provides a method for generating prediction data. Figure 2 This is a flowchart of a method for generating prediction data according to an embodiment of this application, such as... Figure 2 As shown, the process includes the following steps:

[0032] Step S202: Obtain global features of the target time series data based on the first feature extraction layer of the prediction model, and obtain local features of the target time series data based on the second feature extraction layer of the prediction model;

[0033] Step S202 utilizes the first feature extraction layer (such as Transformer) in the prediction model to focus on acquiring global features of the target time series data. The Transformer's multi-head attention mechanism can capture long-distance dependencies across the entire sequence length. Secondly, the second feature extraction layer (such as BiLSTM) of the prediction model focuses on capturing local features of the target time series data. BiLSTM can read the sequence from both forward and backward directions, which helps the model learn local temporal changes in the sequence, such as short-term patterns and context at specific positions in the sequence.

[0034] Step S204: The global features and the local features are weighted and fused according to the parameter matrix to obtain the comprehensive features of the target time series data;

[0035] By weighting and fusing global and local features using a parameter matrix, a comprehensive feature representation of the target time-series data is obtained. The parameter matrix is ​​a learnable component that automatically adjusts its weights during training to ensure an optimal balance between the contributions of global and local features in the comprehensive representation. This overcomes the limitations of a single feature extraction layer, enabling the model to simultaneously utilize both the macroscopic trends and microscopic dynamics of the sequence to form a more comprehensive feature representation.

[0036] Step S206: Control the prediction model to generate prediction data corresponding to the target time series data based on the comprehensive features.

[0037] Controlling the predictive model to generate forecast data based on comprehensive features ensures that the model utilizes all available information from the time series data. By effectively integrating global and local features, the model can better understand and predict the behavior of complex time series, especially in scenarios with local anomalies or rapid changes.

[0038] Through the above steps, global features of the target time-series data are obtained based on the first feature extraction layer of the prediction model, and local features of the target time-series data are obtained based on the second feature extraction layer of the prediction model. The global features and local features are then weighted and fused according to a parameter matrix to obtain comprehensive features of the target time-series data. The prediction model is then controlled to generate predicted data corresponding to the target time-series data based on the comprehensive features. In this embodiment, the global features of the target time-series data are obtained using the first feature extraction layer of the prediction model, and local features of the target time-series data are captured through the second feature extraction layer of the prediction model. The global features and local features are then weighted and fused using a parameter matrix to obtain a comprehensive feature representation of the target time-series data. The prediction model is then controlled to generate predicted data based on the comprehensive features. That is, by combining multi-level feature extraction with an intelligent fusion strategy, the model's ability to grasp the features of time-series data is improved, thereby solving the problem that the model's determination of the features of time-series data is not rich enough, resulting in low accuracy of the output predicted data.

[0039] In an exemplary embodiment, the first feature extraction layer based on the prediction model obtains global features of target time-series data, including: adding positional encoding information to the target time-series data and performing a linear transformation on the target time-series data after adding positional encoding information to obtain a target matrix, wherein the target matrix includes: a query matrix, a key matrix, and a matrix; dividing the target matrix to input the divided target matrix into multiple attention layers respectively, wherein the first feature extraction layer includes: the multiple attention layers; obtaining the attention output by each attention layer based on the divided target matrix, and obtaining global features of the target time-series data based on the attention.

[0040] In this embodiment, the introduction of positional encoding enables the first feature extraction layer to perceive the positional relationships of sequence elements, while the multi-head attention mechanism enhances the model's ability to capture global dependencies by computing the interactions between different positions in parallel. Positional encoding assigns a unique identifier to each sequence position by adding a combination of sine and cosine functions, and linear transformation converts the positionally encoded data into query, key, and value representations suitable for the attention mechanism through a fully connected layer.

[0041] This embodiment significantly improves the model's ability to extract global features from time series data by employing positional encoding and multi-head attention mechanisms, thereby enhancing prediction accuracy. In other embodiments, other types of positional encoding, such as random or learnable positional encoding, and different attention mechanisms, such as self-attention variants or sparse attention, can be used to adapt to different data characteristics and prediction needs, addressing the global feature extraction problem in specific scenarios.

[0042] In an exemplary embodiment, obtaining local features of the target time series data based on the second feature extraction layer of the prediction model includes: modeling the global features through a bidirectional memory network to obtain local features of the target time series data, wherein the second feature extraction layer includes: the bidirectional memory network.

[0043] In this embodiment, the bidirectional gating mechanism of BiLSTM is used to further model the global features extracted by the first feature extraction layer in order to capture the local temporal dynamics of time series data. In principle, BiLSTM processes the sequence in parallel through two LSTM networks, forward and backward, which can simultaneously consider past and future information, thereby enhancing the model's sensitivity to local temporal features.

[0044] This embodiment, through BiLSTM local temporal feature extraction, can more accurately capture short-term changes and trends in the sequence, improving the robustness of the prediction model and its ability to handle outliers. In other embodiments, other types of bidirectional memory networks, such as bidirectional GRU or bidirectional RNN, can be used to adapt to different data characteristics and prediction needs, solving the problem of local temporal feature extraction in specific scenarios.

[0045] In an exemplary embodiment, before obtaining the global features of the target time series data based on the first feature extraction layer of the prediction model, the method further includes: pre-training the original model corresponding to the prediction model based on the original training data to obtain the pre-trained model; performing noise processing on the original training data to obtain noisy data; and performing comparative learning training on the pre-trained model based on the noisy data and the original training data to obtain the prediction model.

[0046] In this embodiment, the prediction model is first pre-trained using the original training data and optimized using loss functions such as MAE, MSE, and DTW. Next, the number and location of outliers are determined through Poisson distribution sampling, and impulse noise is used to simulate sensor anomalies for comparative learning training of the pre-trained model. The specific steps are as follows: sparse noise is introduced into the original data to generate noisy data, and then positive and negative sample pairs are constructed for comparative learning optimization. During the comparative learning process, both the time-based InfoNCE loss function and the adversarial loss function are used for training, which enhances the model's adaptability to noisy environments.

[0047] This embodiment enhances the model's noise resistance and robustness to outliers by combining pre-training and contrastive learning. The pre-training phase utilizes a large amount of unlabeled data, allowing the model to learn the inherent representation of the data through a self-supervised learning task. Contrastive learning, on the other hand, teaches the model to distinguish between normal and outlier data by comparing positive and negative samples.

[0048] This embodiment significantly improves the model's performance when processing noisy data through pre-training and contrastive learning, resulting in more stable and accurate predictions. Optionally, other types of pre-training tasks, such as predicting values ​​at the next time point or sequence reconstruction, and the introduction of different contrastive learning strategies, such as SimCLR or BYOL, can be used to adapt to different data characteristics and prediction needs, addressing the issue of improving noise resistance in specific scenarios.

[0049] In an exemplary embodiment, pre-training the original model corresponding to the prediction model based on the original training data to obtain the pre-trained model includes: constructing a first loss function corresponding to the original model based on a numerical error loss function, a distribution matching loss function, and a frequency domain consistency loss function; and pre-training the original model based on the original training data and the first loss function to obtain the pre-trained model.

[0050] This embodiment optimizes the model's predictive performance by designing a comprehensive loss function that combines numerical error, distribution matching, and frequency domain consistency. The frequency domain consistency loss function automatically balances the importance of different frequency components through Fourier transform and relative error design, enhancing the model's focus on the main signal components.

[0051] This embodiment significantly improves the model's accuracy in predicting periodic time series data by using a comprehensive loss function, while also enhancing its robustness to outliers and noise. In other embodiments, other types of loss functions, such as cross-entropy loss or KL divergence, can be introduced to adapt to different data characteristics and prediction needs, solving model optimization problems in specific scenarios.

[0052] In one exemplary embodiment, noise processing is performed on the original training data to obtain noise data, including: determining the total number of outliers corresponding to the original training data, and determining multiple location indices in the original training data based on the total number of outliers; randomly generating target noise, and replacing the data corresponding to the multiple location indices with the target noise to generate the noise data.

[0053] First, determine the total number of outliers. This step can be done through Poisson distribution sampling, with the parameter λ controlling the frequency of outliers. Then, randomly select the locations of the outliers, ensuring these locations are unique in the original data to avoid multiple noise injections at the same location. Third, generate impulse noise, i.e., salt noise and pepper noise, where the former represents the maximum value and the latter represents the minimum value. Both are selected with equal probability to simulate abrupt changes in amplitude. Finally, replace the original signal value at the selected locations with the impulse noise, completing the noise injection.

[0054] This embodiment uses a sparse noise injection mechanism to simulate sensor failures or data transmission anomalies, thereby improving the model's noise robustness. A Poisson distribution is used to determine the total number of outliers, ensuring sparsity in the time domain, while impulse noise simulates abrupt changes in the value domain, increasing the model's sensitivity to outliers.

[0055] This embodiment significantly enhances the model's robustness and generalization ability when processing real-world data by generating noisy data. Optionally, other types of noise injection strategies, such as Gaussian noise or random noise, can also be used to adapt to different data characteristics and prediction needs, solving data augmentation problems in specific scenarios.

[0056] In an exemplary embodiment, the pre-trained model is subjected to comparative learning training based on the noise data and the original training data to obtain the prediction model, including: constructing a second loss function corresponding to the pre-trained model according to a numerical error loss function, a distribution matching loss function, a temporal basic loss function, and an adversarial loss function; and performing comparative learning training on the pre-trained model based on the noise data, the original training data, and the second loss function to obtain the prediction model.

[0057] This embodiment further optimizes the model's predictive performance and noise resistance by employing a contrastive learning framework combined with adversarial loss based on generated samples. Contrastive learning enables the model to distinguish between normal and abnormal data through the comparison of positive and negative samples, while the adversarial loss, introduced through a classification network D, differentiates between predicted sequences based on real samples and predicted sequences based on generated samples.

[0058] In the contrastive learning architecture, a generative sample adversarial classification network D is added to identify whether the predicted sequence originates from real samples or generated samples. This helps the model learn to distinguish between normal and anomalous data, especially when the generated anomalous data is too realistic and may cause the model to have difficulty distinguishing it. Through adversarial learning, the model can gradually improve its ability to identify anomalous data during training, thereby better handling unknown noise or anomalous situations in actual predictions, significantly enhancing the model's stability and prediction accuracy.

[0059] This embodiment significantly improves the model's performance when processing noisy data by using contrastive learning and generated sample adversarial loss, resulting in more stable and accurate predictions. In other embodiments, other types of contrastive learning strategies, such as MoCo or SimCLR, and different adversarial loss calculation methods can be introduced to adapt to different data characteristics and prediction needs, addressing the problems of model optimization and noise resistance improvement in specific scenarios.

[0060] To better understand the process of generating the above-mentioned prediction data, the implementation flow of the above-mentioned prediction data generation method will be described below in conjunction with optional embodiments, but this is not intended to limit the technical solution of the embodiments of this application.

[0061] This embodiment provides a method for generating prediction data. Figure 3 This is a flowchart of the training prediction model according to an embodiment of this application, such as... Figure 3 As shown, the specific steps are as follows:

[0062] Step S301: Collect raw training data.

[0063] Step S302: Pre-train the time series prediction network model (equivalent to the original model in the above embodiment).

[0064] In the initial stage, the time series prediction network (combining Transformer and BiLSTM) is pre-trained using raw data to learn the basic characteristics and temporal patterns of the data. After the pre-training stage, the model initially possesses the ability to process time series data, but it may be sensitive to noise at this point.

[0065] Using the original training data, a loss function is employed. Model training is performed to complete the pre-training of the prediction network model, whereby... The numerical error loss function is... For distribution matching loss function, Let be the frequency domain consistency loss function. and This is a hyperparameter and can be determined based on experimental results.

[0066] The frequency domain consistency loss function is determined as follows:

[0067] ;in, This represents the amplitude spectrum of a real time series after Fourier transform.

[0068] ; This represents the amplitude spectrum of the predicted time series after Fourier transform;

[0069] ,in, It is a real time series sequence. To predict time series, Indicates Fourier transform, It's absolutely worth getting the amplitude spectrum. The number of effective frequency components (the first half of the FFT result), This is a smoothing factor to avoid division by zero.

[0070] Step S303: Noise injection.

[0071] Based on the pre-trained model, sparse noise is added to the original training data. That is, the number and location of outliers are determined by Poisson distribution, and impulse noise is used to simulate sensor failures or abnormal situations in data transmission.

[0072] Optionally, noise injection includes:

[0073] 1) Poisson distribution sampling;

[0074] The total number of outliers, n_anomalies, is determined using a Poisson distribution, with parameter λ controlling the outlier frequency. A max_anomalies parameter is added to prevent an excessive number of outliers. The formula is as follows:

[0075] n_anomalies=min(Poisson(λ), max_anomalies).

[0076] 2) Anomaly point location generation;

[0077] Randomly select n_anomalies of unique location indices from the time series.

[0078] 3) Impulse noise generation;

[0079] Amplitude abrupt change: Generating extreme noise based on the dynamic range of the data. This includes:

[0080] Salt noise (maximum value): max_val = percentile(data, 99);

[0081] Pepper noise (minimum): min_val = percentile(data, 1);

[0082] Directional randomness: Salt / pepper noise is selected with a 50% probability.

[0083] 4) Noise injection.

[0084] Replace the original signal value with impulse noise at the selected location (preserving the time-domain abrupt change characteristics).

[0085] Step S304: Comparative learning training.

[0086] Contrastive learning is used for training on a noisy dataset. This involves constructing positive sample pairs—the original dataset and its noisy version—and generating negative samples, typically fragments from other sequences. The model is optimized using a contrastive learning framework (such as MoCo) combined with InfoNCE loss functions in both the time and frequency domains, achieving good representational capabilities in both domains. Through this contrastive learning training, the model can more accurately distinguish between normal and abnormal sequences and improve its ability to identify periodic features, thus generating an optimized predictive model.

[0087] Optionally, comparative learning training includes:

[0088] 1. Construct positive sample pairs: ( , ).

[0089] 2. Negative sample generation: Randomly sample other sequence fragments across batches and verify that the difference between them and x is greater than a certain threshold.

[0090] 3. Using the MoCo contrastive learning architecture, an adversarial classification network for generating samples is added, and the loss function is used for contrastive learning training.

[0091] The loss function for contrastive learning is as follows:

[0092] ,in, Let A be the time-series basic loss function, and let B be the hyperparameters.

[0093] ; The prediction results are based on the original time series data. To simulate abnormal sensor noise in the prediction results after adding sparse noise to the original time-series data, Based on the prediction results of negative samples, Here, is the temperature parameter used to adjust the scale of similarity in contrastive learning, and N is the number of samples. For similarity calculation.

[0094] To avoid the overly demanding requirements for sample and input sample predictions introduced by adversarial learning, which could cause the network to lose its ability to perceive small differences and ignore important differences in real time series, we propose adding a classification network D to the contrastive learning framework. We design an adversarial loss function to distinguish between real sample prediction sequences and generated sample prediction sequences. The network structure is as follows: Figure 4 As shown. The prediction network is represented by P, and the adversarial loss function is:

[0095] ; y is a modified or enhanced version of the feature vector obtained by adding sparse noise to y.

[0096] The overall loss function then becomes End-to-end training can still be performed.

[0097] After completing all the above training steps, the training phase officially ends, and the model has been fully optimized and adjusted.

[0098] Step S305: Test the application.

[0099] Finally, the optimized model is used to predict new time series data. At this stage, the model can not only handle routine time series forecasting but also effectively handle noisy data, improving the accuracy and reliability of the predictions.

[0100] The backbone network structure of the prediction model in this embodiment is as follows:

[0101] 1. Input layer:

[0102] Input data types: One-dimensional time series data, such as voice signals, electrocardiogram signals, stock price series, etc.

[0103] Input data preprocessing: Normalize, denoise, and segment the input data to improve the robustness and accuracy of the model.

[0104] 2. Hierarchical feature extraction module:

[0105] The underlying Transformer feature extraction submodule:

[0106] Multi-head attention layer: This layer uses a multi-head attention mechanism to encode the input time-series data and capture global dependencies within the data. By computing multiple attention heads in parallel, the model can focus on the interactions between different locations in the data.

[0107] Position encoding layer: Add position encoding information to the input data, enabling the Transformer to perceive the positional relationship of elements in the sequence and enhancing its sensitivity to sequence information.

[0108] Top-level BiLSTM feature extraction submodule:

[0109] Bidirectional gating mechanism: This mechanism utilizes a bidirectional LSTM to further model the features extracted by the underlying Transformer, capturing the local temporal dynamics of time series data. The bidirectional LSTM can simultaneously consider information from the past and future, improving its ability to model time series data.

[0110] 3. Dynamic Weight Fusion Module:

[0111] Learnable parameter matrix: A learnable parameter matrix is ​​designed to achieve adaptive fusion of cross-layer features. The parameters of this matrix are automatically learned during the training process, and can dynamically adjust the weights of global features and local temporal features according to the characteristics of the input data and task requirements.

[0112] Feature fusion method: The global features extracted by the bottom-level Transformer and the local temporal features extracted by the top-level BiLSTM are weighted and fused using a learnable parameter matrix to obtain a comprehensive feature representation. The fused features contain both global dependency information and local temporal dynamic information, enabling a more comprehensive description of the input data.

[0113] 4. Output layer:

[0114] Classification / Regression Module: The appropriate output layer structure is adopted according to the specific task requirements. For example, for classification tasks, a fully connected layer with a softmax function can be used to output the probability of each category; for regression tasks, a fully connected layer can be used to directly output the predicted value.

[0115] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0116] This embodiment also provides a predictive data generation apparatus for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0117] Figure 5 This is a structural block diagram of a predictive data generation apparatus according to an embodiment of this application, such as... Figure 5 As shown, the device includes:

[0118] The acquisition module 52 is used to acquire global features of the target time series data based on the first feature extraction layer of the prediction model, and to acquire local features of the target time series data based on the second feature extraction layer of the prediction model.

[0119] The fusion module 54 is used to perform weighted fusion of the global features and the local features according to the parameter matrix to obtain the comprehensive features of the target time series data;

[0120] The generation module 56 is used to control the prediction model to generate prediction data corresponding to the target time series data based on the comprehensive features.

[0121] The aforementioned apparatus acquires global features of the target time-series data based on a first feature extraction layer of the prediction model, and acquires local features of the target time-series data based on a second feature extraction layer of the prediction model. The global and local features are then weighted and fused using a parameter matrix to obtain a comprehensive feature representation of the target time-series data. The prediction model is then controlled to generate predicted data corresponding to the target time-series data based on the comprehensive feature representation. In this embodiment, the global features of the target time-series data are acquired using a first feature extraction layer in the prediction model, and the local features of the target time-series data are captured using a second feature extraction layer of the prediction model. The global and local features are then weighted and fused using a parameter matrix to obtain a comprehensive feature representation of the target time-series data. The prediction model is then controlled to generate predicted data based on the comprehensive feature representation. In other words, by combining multi-level feature extraction with an intelligent fusion strategy, the model's ability to grasp the features of time-series data is improved, thereby solving the problem that the model's determination of the time-series data's features is insufficient, leading to low accuracy in the output predicted data.

[0122] In an exemplary embodiment, the acquisition module 52 is configured to add position encoding information to the target time series data and perform linear transformation on the target time series data after adding position encoding information to obtain a target matrix, wherein the target matrix includes: a query matrix, a key matrix, and a matrix; divide the target matrix so that the divided target matrix is ​​input into multiple attention layers respectively, wherein the first feature extraction layer includes: the multiple attention layers; acquire the attention output by each attention layer according to the divided target matrix, and acquire the global features of the target time series data according to the attention.

[0123] In an exemplary embodiment, the acquisition module 52 is configured to model the global features through a bidirectional memory network to obtain local features of the target time series data, wherein the second feature extraction layer includes the bidirectional memory network.

[0124] In an exemplary embodiment, the above apparatus includes: a training module, configured to, before obtaining global features of target time-series data based on a first feature extraction layer of a prediction model, further include: pre-training an original model corresponding to the prediction model based on original training data to obtain a pre-trained model; performing noise processing on the original training data to obtain noise data; and performing comparative learning training on the pre-trained model based on the noise data and the original training data to obtain the prediction model.

[0125] In an exemplary embodiment, the training module is configured to construct a first loss function corresponding to the original model based on a numerical error loss function, a distribution matching loss function, and a frequency domain consistency loss function; and to pre-train the original model based on the original training data and the first loss function to obtain a pre-trained model.

[0126] In an exemplary embodiment, a training module is configured to determine the total number of outliers corresponding to the original training data, and determine multiple location indices in the original training data based on the total number of outliers; randomly generate target noise, and replace the data corresponding to the multiple location indices with the target noise to generate the noise data.

[0127] In an exemplary embodiment, the training module is configured to construct a second loss function corresponding to the pre-trained model based on a numerical error loss function, a distribution matching loss function, a temporal basic loss function, and an adversarial loss function; and to perform comparative learning training on the pre-trained model based on the noise data, the original training data, and the second loss function to obtain the prediction model.

[0128] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0129] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above method embodiments when run.

[0130] Optionally, in this embodiment, the storage medium may be configured to store program code for performing the following steps:

[0131] S1, global features of the target time series data are obtained based on the first feature extraction layer of the prediction model, and local features of the target time series data are obtained based on the second feature extraction layer of the prediction model;

[0132] S2, the global features and the local features are weighted and fused according to the parameter matrix to obtain the comprehensive features of the target time series data;

[0133] S3, control the prediction model to generate prediction data corresponding to the target time series data based on the comprehensive features.

[0134] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0135] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0136] In one exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0137] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0138] S1, global features of the target time series data are obtained based on the first feature extraction layer of the prediction model, and local features of the target time series data are obtained based on the second feature extraction layer of the prediction model;

[0139] S2, the global features and the local features are weighted and fused according to the parameter matrix to obtain the comprehensive features of the target time series data;

[0140] S3, control the prediction model to generate prediction data corresponding to the target time series data based on the comprehensive features.

[0141] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0142] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0143] Embodiments of this application also provide a computer program that includes computer instructions stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps in any of the above method embodiments.

[0144] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0145] S1, global features of the target time series data are obtained based on the first feature extraction layer of the prediction model, and local features of the target time series data are obtained based on the second feature extraction layer of the prediction model;

[0146] S2, the global features and the local features are weighted and fused according to the parameter matrix to obtain the comprehensive features of the target time series data;

[0147] S3, control the prediction model to generate prediction data corresponding to the target time series data based on the comprehensive features.

[0148] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0149] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0150] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application.

Claims

1. A method for generating prediction data, characterized in that, include: The first feature extraction layer based on the prediction model obtains the global features of the target time series data, and the second feature extraction layer based on the prediction model obtains the local features of the target time series data. The global features and the local features are weighted and fused according to the parameter matrix to obtain the comprehensive features of the target time series data; The prediction model is controlled to generate prediction data corresponding to the target time series data based on the comprehensive features.

2. The method according to claim 1, characterized in that, The first feature extraction layer based on the prediction model obtains global features of the target time series data, including: Location encoding information is added to the target time series data, and the target time series data after adding location encoding information is linearly transformed to obtain a target matrix, wherein the target matrix includes: a query matrix, a key matrix, and a matrix; The target matrix is ​​divided into multiple attention layers, and the divided target matrix is ​​input into multiple attention layers respectively. The first feature extraction layer includes the multiple attention layers. Obtain the attention output of each attention layer based on the target matrix after partitioning, and obtain the global features of the target time series data based on the attention.

3. The method according to claim 1, characterized in that, The second feature extraction layer of the prediction model obtains local features of the target time series data, including: The global features are modeled by a bidirectional memory network to obtain local features of the target time series data, wherein the second feature extraction layer includes the bidirectional memory network.

4. The method according to claim 1, characterized in that, Before obtaining the global features of the target time series data based on the first feature extraction layer of the prediction model, the method further includes: The original model corresponding to the prediction model is pre-trained based on the original training data to obtain the pre-trained model. The original training data is subjected to noise processing to obtain noisy data; The pre-trained model is trained by comparison with the noise data and the original training data to obtain the prediction model.

5. The method according to claim 4, characterized in that, The original model corresponding to the prediction model is pre-trained based on the original training data to obtain the pre-trained model, including: The first loss function corresponding to the original model is constructed based on the numerical error loss function, the distribution matching loss function, and the frequency domain consistency loss function. The original model is pre-trained based on the original training data and the first loss function to obtain the pre-trained model.

6. The method according to claim 4, characterized in that, The original training data is subjected to noise processing to obtain noisy data, including: Determine the total number of outliers corresponding to the original training data, and determine multiple location indices in the original training data based on the total number of outliers; Randomly generate target noise, and replace the data corresponding to the multiple location indices with the target noise to generate the noise data.

7. The method according to claim 4, characterized in that, The pre-trained model is trained by comparison with the noise data and the original training data to obtain the prediction model, including: Construct a second loss function corresponding to the pre-trained model based on the numerical error loss function, the distribution matching loss function, the temporal basic loss function, and the adversarial loss function; The pre-trained model is trained by comparative learning based on the noise data, the original training data, and the second loss function to obtain the prediction model.

8. An apparatus for generating predictive data, characterized in that, include: The acquisition module is used to acquire global features of the target time series data based on the first feature extraction layer of the prediction model, and to acquire local features of the target time series data based on the second feature extraction layer of the prediction model. The fusion module is used to perform weighted fusion of the global features and the local features according to the parameter matrix to obtain the comprehensive features of the target time series data; The generation module is used to control the prediction model to generate prediction data corresponding to the target time series data based on the comprehensive features.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program, when executed, performs the method of any one of claims 1 to 7.

10. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the method of any one of claims 1 to 7 through the computer program.

Images