Data prediction method and device
Through the data prediction method of frequency domain sliding average adaptive normalization and group modeling, the non-stationarity and channel dependence of operation and maintenance data are solved, the prediction accuracy and efficiency are improved, and it is suitable for operation and maintenance applications such as system monitoring and abnormal detection.
Patent Information
- Application Number
- CN202510517252.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-23
- Publication Date
- 2025-08-08
AI Technical Summary
Existing data prediction methods cannot effectively solve the problems of non-stationarity, channel dependence and insufficient data volume of operation and maintenance data, resulting in low prediction accuracy and prediction efficiency.
By performing frequency domain sliding average adaptive normalization of the prediction data, group modeling, fine-tuning using large language models, and performing anti-normalization processing to obtain inter-channel dependency data inside and outside the group.
It significantly improves the prediction accuracy and prediction efficiency, can effectively capture the long-term trend of operation and maintenance data and the dependence between channels, and adapt to the prediction needs under different data volume conditions.
Smart Images
Figure CN120448715A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of prediction technology, and in particular to a data prediction method and device. Background Art
[0002] Operations and maintenance (Ops) data is a crucial tool for ensuring system performance and optimizing resource utilization. Time series data, in particular, has significant application value in areas such as system monitoring and anomaly detection. The application of time series forecasting (TSF) technology in O&M not only provides early warning of system failures and improves maintenance efficiency, but also plays a crucial role in resource scheduling and performance optimization. With the rise of all-in-one appliances, they not only generate large amounts of O&M data but also enable the deployment of large models for real-time analysis and prediction, making them ideal platforms for Ops data processing, particularly suitable for scenarios involving private data, low latency, and limited bandwidth.
[0003] On the one hand, operation and maintenance data usually has significant non-stationary characteristics, which is manifested in the fact that the statistical characteristics of the data (such as mean and standard deviation) fluctuate over time. On the other hand, operation and maintenance data usually has multi-channel correlation (such as network traffic, equipment temperature, system load, etc.) and complex dependencies between channels. On the other hand, operation and maintenance data does not always have sufficient data volume, and data is often missing in different channels or time.
[0004] However, some existing data prediction methods predict stationary data and cannot effectively capture the long-term trend of the data, resulting in low prediction accuracy; some independently model the data of each channel and then make data predictions, but cannot effectively capture the dependencies between channels, resulting in low prediction accuracy; some try to model the dependencies between channels and then make data predictions, but when faced with large-scale multi-channel data, the computational complexity grows exponentially, resulting in low prediction efficiency; and some can only make predictions for channel data with sufficient data volume. When the data volume is insufficient, the prediction accuracy will drop sharply.
[0005] In summary, the existing data prediction methods cannot fully solve the problems of non-stationarity, channel dependence and insufficient data volume of operation and maintenance data, resulting in low prediction accuracy and efficiency. Summary of the Invention
[0006] The embodiments of the present application provide a data prediction method and device to solve the technical problem that existing data prediction methods cannot fully solve the problems of non-stationarity, channel dependence and insufficient data volume of operation and maintenance data, resulting in low prediction accuracy and prediction efficiency.
[0007] In a first aspect, an embodiment of the present application provides a data prediction method, comprising: Perform frequency domain sliding average adaptive normalization on the predicted data to obtain the target normalized data; Performing group modeling on the target normalized data to obtain inter-channel dependency data within and outside the group; Inputting the inter-channel dependency data into a large language model, and fine-tuning the large language model to obtain prediction data output by the large language model; Denormalize the predicted data to obtain target predicted data.
[0008] In one embodiment, performing frequency domain sliding average adaptive normalization on the data to be predicted to obtain target normalized data includes: Performing discrete wavelet transform on the data to be predicted to obtain low-frequency approximation coefficients and high-frequency detail coefficients; Performing sliding average adaptive normalization on the low-frequency approximation coefficient and the high-frequency detail coefficient to obtain a low-frequency normalization coefficient and a high-frequency normalization coefficient; Performing inverse discrete wavelet transform on the low-frequency normalization coefficient and the high-frequency normalization coefficient to obtain target normalized data.
[0009] In one embodiment, performing sliding average adaptive normalization on the low-frequency approximation coefficient and the high-frequency detail coefficient to obtain a low-frequency normalization coefficient and a high-frequency normalization coefficient includes: Calculating a first mean, a first standard deviation, a maximum value, and a minimum value of the low-frequency approximation coefficients at each time step, and a second mean and a second standard deviation of the high-frequency detail coefficients based on a sliding window; Normalizing the low-frequency approximation coefficient based on the first mean and the first standard deviation to obtain a low-frequency normalized coefficient; Normalizing the low-frequency approximation coefficient based on the maximum value and the minimum value to obtain a normalized approximation coefficient; Normalizing the high-frequency detail coefficient based on the second mean and the second standard deviation to obtain a normalized detail coefficient; Performing Fourier transform on the normalized approximation coefficient and the normalized detail coefficient to obtain a transformed normalized approximation coefficient and a transformed normalized detail coefficient; Performing a weighted summation on the transform normalized approximation coefficient and the transform normalized detail coefficient to obtain a transform normalized comprehensive coefficient; Performing an inverse Fourier transform on the transformed normalized comprehensive coefficients to obtain high-frequency normalized coefficients.
[0010] In one embodiment, grouping and modeling the target normalized data to obtain inter-channel dependency data within and outside the group includes: Dividing the target normalized data into a plurality of data blocks of different channels at different time periods; Dividing data blocks of different channels in the same adjacent time period among the multiple data blocks into data block groups; Alternately acquiring the intra-group and extra-group dependency data of the data block group based on a period attention mechanism and a channel attention mechanism; Based on the intra-group and extra-group dependency data, inter-channel dependency data is obtained; The intra-group and extra-group dependency data include intra-group dependency data and extra-group dependency data. The intra-group dependency data includes dependency data between different time periods and different channels in the same data block group and dependency data between different channels in the same time period. The extra-group dependency data includes dependency data between different time periods and different channels in different data block groups and dependency data between different channels in the same time period.
[0011] In one embodiment, fine-tuning the large language model includes: A low-rank adapter is inserted into the large language model, and parameters of the large language model are fine-tuned based on the low-rank adapter until the parameters are adapted to the inter-channel dependency data.
[0012] In one embodiment, denormalizing the predicted data to obtain target predicted data includes: Based on the target prediction mean and the target prediction standard deviation, denormalizing the prediction data to obtain target prediction data; The target prediction mean is generated based on the first mean and the second mean, and the target prediction standard deviation is generated based on the first standard deviation and the second standard deviation.
[0013] In one embodiment, the target prediction mean and the target prediction standard deviation are generated based on the following steps: Inputting the first mean and the second mean into a mean feedforward network to obtain a first predicted mean and a second predicted mean for each time step output by the mean feedforward network; Inputting the first standard deviation and the second standard deviation into a standard deviation feedforward network to obtain a first predicted standard deviation and a second predicted standard deviation for each time step output by the standard deviation feedforward network; Performing an inverse discrete wavelet transform on the first predicted mean and the second predicted mean to obtain a target predicted mean; Performing an inverse discrete wavelet transform on the first prediction standard deviation and the second prediction standard deviation to obtain a target prediction standard deviation.
[0014] In a second aspect, an embodiment of the present application provides a data prediction device, comprising: The sliding average adaptive normalization module is used to perform frequency domain sliding average adaptive normalization on the predicted data to obtain target normalized data; An inter-channel dependency modeling module is used to: perform group modeling on the target normalized data to obtain inter-channel dependency data within and outside the group; A large language model fine-tuning module, configured to: input the inter-channel dependency data into the large language model, and fine-tune the large language model to obtain prediction data output by the large language model; The denormalization module is used to: perform denormalization on the predicted data to obtain target predicted data.
[0015] In a third aspect, an embodiment of the present application provides an electronic device comprising a processor and a memory storing a computer program, wherein when the processor executes the program, the steps of the data prediction method described in the first aspect are implemented.
[0016] In a fourth aspect, an embodiment of the present application provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the data prediction method described in the first aspect.
[0017] In a fifth aspect, an embodiment of the present application provides a non-transitory computer-readable storage medium, comprising a computer program, which, when executed by a processor, implements the steps of the data prediction method described in the first aspect.
[0018] The data prediction method and device provided in the present application perform frequency domain sliding average adaptive normalization on the predicted data to obtain target normalized data, perform group modeling on the target normalized data to obtain inter-channel dependency data within and outside the group, input the inter-channel dependency data into a large language model, and fine-tune the large language model to obtain predicted data output by the large language model, and perform denormalization on the predicted data to obtain target predicted data. On the one hand, the present application performs frequency domain sliding average adaptive normalization on the predicted data, which can reduce the volatility of the data in the frequency domain, suppress the non-stationary influence of the data, and retain key long-term trend information. It can also adaptively and dynamically adjust the normalization process of the data to further suppress the non-stationary influence of the data, thereby effectively capturing the long-term trend of the data and improving the prediction accuracy. On the other hand, the target normalized data is grouped and modeled to obtain the dependency relationship between channels inside and outside the group, which can effectively capture the complex interactions between different channels and improve the prediction accuracy. In addition, when facing large-scale multi-channel data, the computational complexity can be reduced by grouping calculations to improve the prediction efficiency. On the other hand, based on the fine-tuning of the large language model, it can be made more compatible with the input channel dependency data, so that when facing channel data with a smaller amount of data, it still has good prediction accuracy. In summary, the present application can fully solve the problems of non-stationarity, channel dependency and insufficient data volume of operation and maintenance data, thereby improving the prediction accuracy and prediction efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] In order to more clearly illustrate the technical solutions in the present application or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.
[0020] Figure 1 This is one of the flow charts of the data prediction method provided in the embodiment of the present application; Figure 2 This is the second flow chart of the data prediction method provided in the embodiment of the present application; Figure 3 This is a flow chart of the data prediction method provided in the embodiment of the present application; Figure 4 Schematic diagram of frequency domain sliding average adaptive normalization of the data prediction method provided in an embodiment of the present application; Figure 5 This is the third flow chart of the data prediction method provided in the embodiment of the present application; Figure 6 Schematic diagram of the structure of the data prediction device provided in the embodiment of the present application; Figure 7It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0021] To make the objectives, technical solutions, and advantages of this application more clear, the technical solutions in this application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this application.
[0022] Figure 1 This is one of the flow charts of the data prediction method provided in the embodiment of the present application. Figure 1 , an embodiment of the present application provides a data prediction method, which may include: 101. Perform frequency domain sliding average adaptive normalization on the data to be predicted to obtain target normalized data; 102. Perform group modeling on the target normalized data to obtain the channel dependency data within and outside the group; 103. Input the inter-channel dependency data into the large language model, and fine-tune the large language model to obtain the predicted data output by the large language model; 104. Denormalize the predicted data to obtain target predicted data.
[0023] In step 103, the large model hyper-converged all-in-one machine can be integrated with the pre-trained large language model. The hardware architecture of the all-in-one machine supports the rapid deployment and efficient computing of the large language model. The pre-installed software environment provides a unified operation and maintenance management platform, enabling the large language model to fully utilize its powerful computing power and rapid deployment characteristics. It can not only greatly improve the real-time prediction capability, but also run in a low-latency, high-performance environment, greatly enhancing the practicality of the model and meeting the efficiency requirements in operation and maintenance data processing.
[0024] The data processed in each step of this embodiment are all operation and maintenance time series data, among which the data to be predicted can be historical operation and maintenance time series data, and the target prediction data can be future operation and maintenance time series data. This embodiment predicts future operation and maintenance time series data based on historical operation and maintenance time series data. Assuming that the data to be predicted is historical multi-channel time series data , then this embodiment aims to predict future multi-channel time series data ,in, is the number of channels, that is, the number of variables, is the number of time steps of historical multi-channel time series data, is the number of time steps for future multi-channel time series data.
[0025] The data prediction method provided in this embodiment performs frequency domain sliding average adaptive normalization on the predicted data to obtain target normalized data, performs group modeling on the target normalized data to obtain inter-channel dependency data within and outside the group, inputs the inter-channel dependency data into a large language model, and fine-tunes the large language model to obtain predicted data output by the large language model, and denormalizes the predicted data to obtain target predicted data. On the one hand, this embodiment performs frequency domain sliding average adaptive normalization on the predicted data, which can reduce the volatility of the data in the frequency domain, suppress the non-stationary influence of the data, and retain key long-term trend information. It can also adaptively and dynamically adjust the normalization process of the data to further suppress the non-stationary influence of the data, thereby effectively capturing the long-term trend of the data and improving the prediction accuracy. On the other hand, the target normalized data is grouped and modeled to obtain the dependency relationship between channels inside and outside the group, which can effectively capture the complex interaction between different channels and improve the prediction accuracy. In addition, when facing large-scale multi-channel data, the computational complexity can be reduced by grouping calculation to improve the prediction efficiency. On the other hand, based on the fine-tuning of the large language model, it can be made more adaptable to the input channel dependency data, so that when facing channel data with a smaller amount of data, it still has good prediction accuracy. In summary, this embodiment can fully solve the problems of non-stationarity, channel dependency and insufficient data volume of operation and maintenance data, thereby improving the prediction accuracy and prediction efficiency, and providing more accurate and timely support for operation and maintenance applications such as system monitoring and anomaly detection.
[0026] Figure 2 This is the second flow chart of the data prediction method provided in the embodiment of this application. Figure 2 In one embodiment, performing frequency domain sliding average adaptive normalization on the data to be predicted to obtain target normalized data may include: 201. Perform discrete wavelet transform on the data to be predicted to obtain low-frequency approximate coefficients and high-frequency detail coefficients; 202. Perform sliding average adaptive normalization on the low-frequency approximation coefficient and the high-frequency detail coefficient to obtain a low-frequency normalization coefficient and a high-frequency normalization coefficient; 203. Perform inverse discrete wavelet transform on the low-frequency normalization coefficient and the high-frequency normalization coefficient to obtain target normalized data.
[0027] Reference Figure 3 : In step 201, for The Sequence data Perform discrete wavelet transform and decompose it into low-frequency approximate coefficients and high frequency detail coefficient , which can be specifically expressed as the following formula: ; in, represents discrete wavelet transform, and The mother wavelet function low-pass filter and high-pass filter.
[0028] In step 202, the low-frequency approximation coefficients and high-frequency detail coefficients are also time series data. For any type of coefficient, normalization is achieved through a sliding window, such as Figure 4 The specific steps are as follows: 202a. Calculate, based on the sliding window, a first mean, a first standard deviation, a maximum value, and a minimum value of the low-frequency approximation coefficients at each time step, and a second mean and a second standard deviation of the high-frequency detail coefficients; 202b. Normalize the low-frequency approximation coefficient based on the first mean and the first standard deviation to obtain a low-frequency normalized coefficient; 202c. Normalizing the low-frequency approximation coefficient based on the maximum value and the minimum value to obtain a normalized approximation coefficient; 202d. Normalize the high-frequency detail coefficient based on the second mean and the second standard deviation to obtain a normalized detail coefficient; 202e. Performing Fourier transform on the normalized approximation coefficient and the normalized detail coefficient to obtain a transformed normalized approximation coefficient and a transformed normalized detail coefficient; 202f. Performing weighted summation on the transformation normalization approximation coefficient and the transformation normalization detail coefficient to obtain a transformation normalization comprehensive coefficient; 202g. Perform an inverse Fourier transform on the transformed normalized comprehensive coefficient to obtain a high-frequency normalized coefficient.
[0029] In step 202a, each sliding window corresponds to a time step and includes multiple time steps before and after the time step. The first mean, first standard deviation, maximum value, and minimum value of the low-frequency approximation coefficient of each time step are the mean, standard deviation, maximum value, and minimum value of the low-frequency approximation coefficient of each time step in the corresponding sliding window. The second mean and second standard deviation of the high-frequency detail coefficient of each time step are the mean and standard deviation of the high-frequency detail coefficient of each time step in the corresponding sliding window. The mean and standard deviation can be calculated according to the following formula: ; ; in, For the The first mean or second mean of the time steps, For the The first standard deviation or second standard deviation of the time step, is half the size of the sliding window, For the The low-frequency approximation coefficient or high-frequency detail coefficient of the time step, and the The time step is at Sliding window corresponding to time steps middle.
[0030] for and The boundary position is filled by copying the nearest mean and standard deviation to keep the sequence length consistent before and after normalization, so that all time steps in the sequence have corresponding mean and standard deviation.
[0031] In step 202b, it is assumed that the first mean is , the first standard deviation is , the low-frequency normalization coefficient can be obtained using the following formula : ; in, To prevent the denominator from being a very small positive number of zero, we can take .
[0032] In steps 202c to 202g, in order to reduce the problem of reduced predictive ability and noise amplification caused by excessive stabilization, it is assumed that the maximum value is , the minimum value is , the second mean is , the second standard deviation is , we can first use the following formula to get the normalized approximate coefficient : ; Then use the following formula to get the normalized detail coefficient : ; Finally, the high-frequency normalization coefficient is obtained using the following formula : ; in, represents the fast Fourier transform, represents the inverse fast Fourier transform, is an adjustable hyperparameter used to balance and contribution.
[0033] In step 203, Sequence data The corresponding normalized data , can be obtained according to the following formula: ; in, Represents the inverse discrete wavelet transform.
[0034] Then Connect the normalized data corresponding to each sequence data in to obtain the target normalized data .
[0035] This embodiment uses discrete wavelet transform to decompose the data to be predicted into low-frequency approximate coefficients and high-frequency detail coefficients, effectively separating low-frequency trends and high-frequency noise, and then uses frequency domain sliding average adaptive normalization to specifically normalize the low-frequency trends and high-frequency noise, greatly reducing the non-stationarity in the operation and maintenance time series data, and retaining important long-term trends in the data. It can capture time series patterns more accurately, thereby significantly improving the accuracy and stability of subsequent predictions.
[0036] Figure 5 This is the third flow chart of the data prediction method provided in the embodiment of this application. Figure 5 In one embodiment, grouping and modeling the target normalized data to obtain inter-channel dependency data within and outside the group may include: 501. Divide the target normalized data into multiple data blocks of different time periods and channels; 502. Divide data blocks in different channels in the same adjacent time period among the multiple data blocks into data block groups; 503. Alternately obtain the intra-group and extra-group dependency data of the data block group based on the period attention mechanism and the channel attention mechanism; 504. Based on the intra-group and extra-group dependency data, obtain inter-channel dependency data.
[0037] The intra-group and extra-group dependency data include intra-group dependency data and extra-group dependency data. The intra-group dependency data includes the dependency data between different time periods and different channels in the same data block group and the dependency data between different channels in the same time period. The extra-group dependency data includes the dependency data between different time periods and different channels in different data block groups and the dependency data between different channels in the same time period.
[0038] In step 501, each data block can also be converted into a dimension that matches the input of the large language model through linear mapping , then the data block ,in, is the number of data blocks.
[0039] In step 502, refer to Figure 3 , the same color 、 、 Represents data blocks of different time periods in the same channel, with different colors Represents data blocks of different channels in the same period, with different colors and different colors Similarly, the data blocks of different channels in the same adjacent time period in multiple data blocks are divided into data block groups, that is, the data blocks of different colors are divided into data block groups. 、 、 Divide into a data block group, such as Figure 3 Group 1 and Group 2 in the data block group, and each color in each data block group 、 、 They are all data blocks in the same adjacent time period.
[0040] In step 503, refer to Figure 3 , both the period attention mechanism and the channel attention mechanism are executed by the group channel dependency building module, which can be set Grouped channel-dependent building blocks, each of which includes layer normalization , Modeling Feedforward Network And group self-attention, the group self-attention in each building block is alternately set as period attention and channel attention , Used to model the dependencies between different time periods and different channels, It is used to model the dependency between different channels within the same period, as shown in the following formula: ; ; in, for The group channel dependency building module The output of the building block, if The group self-attention in the building block is ,but After group self-attention in the next building block ,get , After the modeling feedforward network in the next building block, Perform layer normalization to obtain the output of the next module. , and so on; if The group self-attention in the building block is ,but After group self-attention in the next building block ,get , After the modeling feedforward network in the next building block, Perform layer normalization to obtain the output of the next module. , and so on; it should be noted that when hour, This is the data block group that is input into the first building block.
[0041] Reference Figure 3 , and Both adopt a two-stage attention mechanism of intra-group attention and inter-group attention, where intra-group attention is used to capture the dependencies between channels in the same group, and inter-group attention is used to capture the global dependencies between channels in different groups.
[0042] Specifically, you can Divided into groups, each containing channels. If the number of channels Cannot be If the data is divisible, zero padding channels are added to ensure uniform grouping, and the data block groups are obtained. First, attention is calculated on the channels within the same group, namely intra-group attention calculation, and then attention is calculated on the channels between different groups, namely inter-group attention calculation, to capture the complex dependencies in the channel and time dimensions.
[0043] for , the calculation formulas for intra-group attention and inter-group attention are as follows: ; ; ; ; for , the calculation formulas for intra-group attention and inter-group attention are as follows: ; ; ; ; in, for After vector reshaping operation After the data block group, For Conduct in-group attention The calculated intra-group dependence data, For Conduct intergroup attention The calculated inter-group dependence data is also called the out-group dependence data.
[0044] This embodiment first divides the target normalized data into multiple data blocks and maps them into dimensions suitable for inputting a large language model. It then divides the multiple data blocks into data block groups, and uses time period attention and channel attention to alternately capture the dependencies between different time periods and different channels, as well as the dependencies between different channels within the same time period, so that channel dependencies can be finely modeled at different levels. Both time period attention and channel attention include intra-group attention mechanism and inter-group attention mechanism, which can effectively reduce the computational complexity and ensure the efficient capture of global and local information, thereby fully capturing the interactions in multi-channel data, and effectively improving the prediction accuracy and prediction efficiency when facing large-scale multi-channel data in subsequent predictions.
[0045] In one embodiment, fine-tuning a large language model may include: Insert a low-rank adapter into the large language model and fine-tune the parameters of the large language model based on the low-rank adapter until the parameters are adapted to the inter-channel dependency data.
[0046] Reference Figure 3 , the inter-channel dependency data output by the last group channel dependency building module is flattened to obtain , input it into a large language model, such as GPT2 (Generative Pre-trained Transformer 2, the second generation of generative pre-trained transformer) or LLaMA2 (Large Language Model Meta AI2, the second generation of Meta AI large language model), and fine-tune the pre-trained weights of the large language model by inserting a low-rank adapter into the large language model until the pre-trained weights are consistent with the Adaptation enables large language models to adapt to specific prediction tasks of operation and maintenance time series data.
[0047] This embodiment can significantly reduce parameter complexity by fine-tuning and optimizing the pre-trained large language model through a low-rank adapter, so that the large language model can still maintain efficient and reliable prediction performance in the face of insufficient data, and achieve accurate prediction of operation and maintenance time series data.
[0048] In one embodiment, denormalizing the predicted data to obtain target predicted data may include: Based on the target prediction mean and target prediction standard deviation, the prediction data is denormalized to obtain the target prediction data; The target prediction mean is generated based on the first mean and the second mean, and the target prediction standard deviation is generated based on the first standard deviation and the second standard deviation.
[0049] The target prediction mean and target prediction standard deviation are generated based on the following steps: Inputting the first mean and the second mean into the mean feedforward network to obtain a first predicted mean and a second predicted mean for each time step output by the mean feedforward network; Inputting the first standard deviation and the second standard deviation into the standard deviation feedforward network, obtaining the first predicted standard deviation and the second predicted standard deviation of each time step output by the standard deviation feedforward network; Perform inverse discrete wavelet transform on the first prediction mean and the second prediction mean to obtain the target prediction mean; Perform inverse discrete wavelet transform on the first prediction standard deviation and the second prediction standard deviation to obtain the target prediction standard deviation.
[0050] Reference Figure 3 , the prediction data output by the large language model is flattened to obtain , and perform a linear mapping on it , so that its dimension is Correspondingly, the target prediction mean and target prediction standard deviation are used to denormalize it to obtain the target prediction data , which is the future multi-channel time series data, can be specifically shown as follows: ; in, for The mean of the target prediction means corresponding to each sequence data in , for The standard deviation of the target prediction standard deviation corresponding to each sequence data in is connected, ,in, The Sequence data The corresponding target prediction mean and target prediction standard deviation It can be obtained according to the following formula: ; ; in, is the first predicted mean, is the second predicted mean, is the first prediction standard deviation, is the second prediction standard deviation, and: ; ; in, is a mean feedforward network, is a standard deviation feedforward network, used to learn and obtain Corresponding future sequence data The predicted mean and the predicted standard deviation .
[0051] This embodiment performs denormalization on the prediction data output by the large prediction model to restore the non-stationarity of the operation and maintenance time series data, and finally obtains the target prediction data. In this process, the mean and standard deviation of the future sequence data are predicted to ensure that the data is fully preprocessed before prediction, thereby improving the prediction accuracy.
[0052] In one embodiment, the prediction effect of the present invention method is compared with other prediction models, as shown in the following table: Table 1 Prediction effect comparison table
[0053] The large language model used in this application is GPT-2, and the power data set used is a data set that is very similar to the operation and maintenance data (such as system power consumption), so it can well reflect the performance of this application in the actual operation and maintenance data prediction. As can be seen from Table 1, when the input data length is , that is, when the time steps are 96, the present application shows better results than the existing methods in all evaluation indicators. Taking 96 time steps as an example, the MSE (Mean Squared Error) of our prediction results is 0.140, significantly lower than the 0.185 of GPT4TS (Generative Pre-trained Transformer for Time Series), 0.148 of iTransformer (Inverted Transformer), and 0.195 of PatchTST (Patch Time Series Transformer). The MAE (Mean Absolute Error) is 0.233, also lower than the 0.272 of GPT4TS, 0.240 of iTransformer, and 0.285 of PatchTST. These results demonstrate that our method performs well at shorter prediction lengths, achieving significantly higher prediction accuracy than other methods. Our method maintains its leading performance as the prediction length increases. For example, when the prediction length is 720 time steps, the MSE of this application is 0.218, which is lower than GPT4TS's 0.245, iTransformer's 0.225, and PatchTST's 0.256; the MAE is 0.298, which is lower than GPT4TS's 0.324, iTransformer's 0.317, and PatchTST's 0.337. This shows that this application also shows high prediction accuracy and reliability in longer prediction lengths.
[0054] To further verify the effectiveness of the frequency domain sliding average adaptive normalization and group modeling proposed in this application, we can also apply frequency domain sliding average adaptive normalization and group modeling to iTransformer to observe their effect improvement, as shown in the following table: Table 2 iTransformer prediction results
[0055] As can be seen from Table 2, the input data length When adding only frequency domain sliding average adaptive normalization or group modeling, the prediction performance of iTransformer can be improved to varying degrees for each prediction length. When both are added at the same time, the effect is most significantly improved.
[0056] The data prediction device provided in an embodiment of the present application is described below. The data prediction device described below and the data prediction method described above can be referenced to each other.
[0057] Figure 6Schematic diagram of the structure of the data prediction device provided in the embodiment of the present application. Figure 6 , an embodiment of the present application provides a data prediction device, which may include: The sliding average adaptive normalization module 601 is used to perform frequency domain sliding average adaptive normalization on the data to be predicted to obtain target normalized data; The inter-channel dependency modeling module 602 is used to: perform group modeling on the target normalized data to obtain inter-channel dependency data within and outside the group; A large language model fine-tuning module 603 is configured to input the inter-channel dependency data into a large language model and fine-tune the large language model to obtain prediction data output by the large language model. The denormalization module 604 is configured to perform denormalization on the predicted data to obtain target predicted data.
[0058] The data prediction device provided in this embodiment performs frequency domain sliding average adaptive normalization on the data to be predicted to obtain target normalized data, performs group modeling on the target normalized data to obtain inter-channel dependency data within and outside the group, inputs the inter-channel dependency data into a large language model, and fine-tunes the large language model to obtain predicted data output by the large language model, and denormalizes the predicted data to obtain target predicted data. On the one hand, this embodiment performs frequency domain sliding average adaptive normalization on the predicted data, which can reduce the volatility of the data in the frequency domain, suppress the non-stationary influence of the data, and retain key long-term trend information. It can also adaptively and dynamically adjust the normalization process of the data to further suppress the non-stationary influence of the data, thereby effectively capturing the long-term trend of the data and improving the prediction accuracy. On the other hand, the target normalized data is grouped and modeled to obtain the dependency relationship between channels inside and outside the group, which can effectively capture the complex interaction between different channels and improve the prediction accuracy. In addition, when facing large-scale multi-channel data, the computational complexity can be reduced by grouping calculation to improve the prediction efficiency. On the other hand, based on the fine-tuning of the large language model, it can be made more adaptable to the input channel dependency data, so that when facing channel data with a smaller amount of data, it still has good prediction accuracy. In summary, this embodiment can fully solve the problems of non-stationarity, channel dependency and insufficient data volume of operation and maintenance data, thereby improving the prediction accuracy and prediction efficiency, and providing more accurate and timely support for operation and maintenance applications such as system monitoring and anomaly detection.
[0059] In one embodiment, the sliding average adaptive normalization module 601 is specifically configured to: Performing discrete wavelet transform on the data to be predicted to obtain low-frequency approximation coefficients and high-frequency detail coefficients; Performing sliding average adaptive normalization on the low-frequency approximation coefficient and the high-frequency detail coefficient to obtain a low-frequency normalization coefficient and a high-frequency normalization coefficient; Performing inverse discrete wavelet transform on the low-frequency normalization coefficient and the high-frequency normalization coefficient to obtain target normalized data.
[0060] In one embodiment, the sliding average adaptive normalization module 601 is specifically configured to: Calculating a first mean, a first standard deviation, a maximum value, and a minimum value of the low-frequency approximation coefficients at each time step, and a second mean and a second standard deviation of the high-frequency detail coefficients based on a sliding window; Normalizing the low-frequency approximation coefficient based on the first mean and the first standard deviation to obtain a low-frequency normalized coefficient; Normalizing the low-frequency approximation coefficient based on the maximum value and the minimum value to obtain a normalized approximation coefficient; Normalizing the high-frequency detail coefficient based on the second mean and the second standard deviation to obtain a normalized detail coefficient; Performing Fourier transform on the normalized approximation coefficient and the normalized detail coefficient to obtain a transformed normalized approximation coefficient and a transformed normalized detail coefficient; Performing a weighted summation on the transform normalized approximation coefficient and the transform normalized detail coefficient to obtain a transform normalized comprehensive coefficient; Performing an inverse Fourier transform on the transformed normalized comprehensive coefficients to obtain high-frequency normalized coefficients.
[0061] In one embodiment, the inter-channel dependency modeling module 602 is specifically configured to: Dividing the target normalized data into a plurality of data blocks of different channels at different time periods; Dividing data blocks of different channels in the same adjacent time period among the multiple data blocks into data block groups; Alternately acquiring the intra-group and extra-group dependency data of the data block group based on a period attention mechanism and a channel attention mechanism; Based on the intra-group and extra-group dependency data, inter-channel dependency data is obtained; The intra-group and extra-group dependency data include intra-group dependency data and extra-group dependency data. The intra-group dependency data includes dependency data between different time periods and different channels in the same data block group and dependency data between different channels in the same time period. The extra-group dependency data includes dependency data between different time periods and different channels in different data block groups and dependency data between different channels in the same time period.
[0062] In one embodiment, the large language model fine-tuning module 603 is specifically configured to: A low-rank adapter is inserted into the large language model, and parameters of the large language model are fine-tuned based on the low-rank adapter until the parameters are adapted to the inter-channel dependency data.
[0063] In one embodiment, the denormalization module 604 is specifically configured to: Based on the target prediction mean and the target prediction standard deviation, denormalizing the prediction data to obtain target prediction data; The target prediction mean is generated based on the first mean and the second mean, and the target prediction standard deviation is generated based on the first standard deviation and the second standard deviation.
[0064] In one embodiment, a target prediction statistic value generation module (not shown in the figure) is further included, which is used to: Inputting the first mean and the second mean into a mean feedforward network to obtain a first predicted mean and a second predicted mean for each time step output by the mean feedforward network; Inputting the first standard deviation and the second standard deviation into a standard deviation feedforward network to obtain a first predicted standard deviation and a second predicted standard deviation for each time step output by the standard deviation feedforward network; Performing an inverse discrete wavelet transform on the first predicted mean and the second predicted mean to obtain a target predicted mean; Performing an inverse discrete wavelet transform on the first prediction standard deviation and the second prediction standard deviation to obtain a target prediction standard deviation.
[0065] Figure 7 is a schematic diagram of the structure of the electronic device provided in the embodiment of the present application, such as Figure 7 As shown, the electronic device may include: a processor 710, a communication interface 720, a memory 730, and a communication bus 740, wherein the processor 710, the communication interface 720, and the memory 730 communicate with each other via the communication bus 740. The processor 710 may call a computer program in the memory 730 to execute the steps of the data prediction method, for example, including: Perform frequency domain sliding average adaptive normalization on the predicted data to obtain the target normalized data; Performing group modeling on the target normalized data to obtain inter-channel dependency data within and outside the group; Inputting the inter-channel dependency data into a large language model, and fine-tuning the large language model to obtain prediction data output by the large language model; Denormalize the predicted data to obtain target predicted data.
[0066] In addition, the logical instructions in the above-mentioned memory 730 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application, or the part that contributes to the existing technology, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program code.
[0067] On the other hand, embodiments of the present application further provide a computer program product, comprising a computer program. The computer program may be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can perform the steps of the data prediction method provided in the above embodiments, for example, including: Perform frequency domain sliding average adaptive normalization on the predicted data to obtain the target normalized data; Performing group modeling on the target normalized data to obtain inter-channel dependency data within and outside the group; Inputting the inter-channel dependency data into a large language model, and fine-tuning the large language model to obtain prediction data output by the large language model; Denormalize the predicted data to obtain target predicted data.
[0068] On the other hand, an embodiment of the present application further provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program is configured to cause a processor to execute the steps of the data prediction method provided in the above embodiments, for example, including: Perform frequency domain sliding average adaptive normalization on the predicted data to obtain the target normalized data; Performing group modeling on the target normalized data to obtain inter-channel dependency data within and outside the group; Inputting the inter-channel dependency data into a large language model, and fine-tuning the large language model to obtain prediction data output by the large language model; Denormalize the predicted data to obtain target predicted data.
[0069] The non-transitory computer-readable storage medium can be any available medium or data storage device that can be accessed by the processor, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical storage (such as CDs, DVDs, BDs, HVDs, etc.), and semiconductor storage (such as ROMs, EPROMs, EEPROMs, non-volatile memories (NAND FLASH), solid-state drives (SSDs), etc.).
[0070] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.
[0071] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.
[0072] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. A data prediction method, characterized in that: include: Perform frequency domain sliding average adaptive normalization on the predicted data to obtain the target normalized data; Performing group modeling on the target normalized data to obtain inter-channel dependency data within and outside the group; Inputting the inter-channel dependency data into a large language model, and fine-tuning the large language model to obtain prediction data output by the large language model; Denormalize the predicted data to obtain target predicted data.
2. The data prediction method according to claim 1, characterized in that The step of performing frequency domain sliding average adaptive normalization on the data to be predicted to obtain target normalized data includes: Performing discrete wavelet transform on the data to be predicted to obtain low-frequency approximation coefficients and high-frequency detail coefficients; Performing sliding average adaptive normalization on the low-frequency approximation coefficient and the high-frequency detail coefficient to obtain a low-frequency normalization coefficient and a high-frequency normalization coefficient; Performing inverse discrete wavelet transform on the low-frequency normalization coefficient and the high-frequency normalization coefficient to obtain target normalized data.
3. The data prediction method according to claim 2, characterized in that: The step of performing sliding average adaptive normalization on the low-frequency approximation coefficient and the high-frequency detail coefficient to obtain a low-frequency normalization coefficient and a high-frequency normalization coefficient includes: Calculating a first mean, a first standard deviation, a maximum value, and a minimum value of the low-frequency approximation coefficients at each time step, and a second mean and a second standard deviation of the high-frequency detail coefficients based on a sliding window; Normalizing the low-frequency approximation coefficient based on the first mean and the first standard deviation to obtain a low-frequency normalized coefficient; Normalizing the low-frequency approximation coefficient based on the maximum value and the minimum value to obtain a normalized approximation coefficient; Normalizing the high-frequency detail coefficient based on the second mean and the second standard deviation to obtain a normalized detail coefficient; Performing Fourier transform on the normalized approximation coefficient and the normalized detail coefficient to obtain a transformed normalized approximation coefficient and a transformed normalized detail coefficient; Performing a weighted summation on the transform normalized approximation coefficient and the transform normalized detail coefficient to obtain a transform normalized comprehensive coefficient; Performing an inverse Fourier transform on the transformed normalized comprehensive coefficients to obtain high-frequency normalized coefficients.
4. The data prediction method according to claim 1, wherein: The grouping and modeling of the target normalized data to obtain inter-channel dependency data within and outside the group includes: Dividing the target normalized data into a plurality of data blocks of different channels at different time periods; Dividing data blocks of different channels in the same adjacent time period among the multiple data blocks into data block groups; Alternately acquiring the intra-group and extra-group dependency data of the data block group based on a period attention mechanism and a channel attention mechanism; Based on the intra-group and extra-group dependency data, inter-channel dependency data is obtained; The intra-group and extra-group dependency data include intra-group dependency data and extra-group dependency data. The intra-group dependency data includes dependency data between different time periods and different channels in the same data block group and dependency data between different channels in the same time period. The extra-group dependency data includes dependency data between different time periods and different channels in different data block groups and dependency data between different channels in the same time period.
5. The data prediction method according to claim 1, characterized in that: The fine-tuning of the large language model includes: A low-rank adapter is inserted into the large language model, and parameters of the large language model are fine-tuned based on the low-rank adapter until the parameters are adapted to the inter-channel dependency data.
6. The data prediction method according to claim 3, characterized in that: Denormalizing the predicted data to obtain target predicted data includes: Based on the target prediction mean and the target prediction standard deviation, denormalizing the prediction data to obtain target prediction data; The target prediction mean is generated based on the first mean and the second mean, and the target prediction standard deviation is generated based on the first standard deviation and the second standard deviation.
7. The data prediction method according to claim 6, characterized in that: The target prediction mean and the target prediction standard deviation are generated based on the following steps: Inputting the first mean and the second mean into a mean feedforward network to obtain a first predicted mean and a second predicted mean for each time step output by the mean feedforward network; Inputting the first standard deviation and the second standard deviation into a standard deviation feedforward network to obtain a first predicted standard deviation and a second predicted standard deviation for each time step output by the standard deviation feedforward network; Performing an inverse discrete wavelet transform on the first predicted mean and the second predicted mean to obtain a target predicted mean; Performing an inverse discrete wavelet transform on the first prediction standard deviation and the second prediction standard deviation to obtain a target prediction standard deviation.
8. A data prediction device, characterized in that: include: The sliding average adaptive normalization module is used to perform frequency domain sliding average adaptive normalization on the predicted data to obtain target normalized data; An inter-channel dependency modeling module is used to: perform group modeling on the target normalized data to obtain inter-channel dependency data within and outside the group; A large language model fine-tuning module, configured to: input the inter-channel dependency data into the large language model, and fine-tune the large language model to obtain prediction data output by the large language model; The denormalization module is used to: perform denormalization on the predicted data to obtain target predicted data.
9. An electronic device comprising a processor and a memory storing a computer program, characterized in that: When the processor executes the computer program, the steps of the data prediction method according to any one of claims 1 to 7 are implemented.
10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the data prediction method according to any one of claims 1 to 7 are implemented.
11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the data prediction method according to any one of claims 1 to 7 are implemented.