Natural gas demand prediction method based on GPT driving mode perception segmentation matching

Abstract

The invention discloses a natural gas demand prediction method based on GPT driving mode perception segmentation matching, and the method comprises the steps: S10, obtaining a natural gas demand time sequence, inputting a GPT-TempFeat-based time sequence feature learning model GPT-TempFeat employing low-rank adaptation LoRA for fine tuning, capturing a potential mode in the natural gas demand time sequence, and keeping the original knowledge of the natural gas demand time sequence; s20, dynamically segmenting the time sequence into different segments through a rolling window segmentation algorithm by using the time features extracted by the GPT-TempFeat; s30, for each identified segment, training an individual GPT-Forecast model by using LoRA fine tuning to obtain a segment model; s40, in a prediction stage, identifying the most suitable segment model of each input window by adopting a mode matching method based on cosine similarity; and performing prediction by using the selected section model to obtain a natural gas demand prediction result.

Description

Technical Field

[0001] The present invention belongs to the technical field of data processing, and particularly relates to a natural gas demand prediction method based on GPT-driven pattern perception segmentation and matching. Background Art

[0002] In the field of natural gas demand prediction, existing methods have various deficiencies. Statistical models have been widely used, such as ARIMA and its extended forms, various regression models, etc., and they perform well in dealing with linear relationships. However, in the face of the complex non-linear patterns formed by the intertwined influence of multiple factors such as economic growth, weather conditions, technological innovation, and policy adjustments on natural gas demand in reality, such models are stretched. They are difficult to accurately capture the subtle non-linear correlations in demand data, and even less able to adapt to the dynamic changes in demand patterns in a timely manner, resulting in a large deviation between the prediction results and the actual demand, and unable to provide strong support for key decisions such as energy planning.

[0003] Although artificial intelligence models based on machine learning and deep learning have advantages and can better handle non-linear relationships in big data and adapt to diverse data characteristics. But in practical applications, these models mostly rely on their own learning mechanisms and lack in-depth analysis of data characteristics. When the consumption pattern suddenly changes or encounters a strong external impact, the model cannot keenly perceive the internal change logic of the data, resulting in a significant decline in prediction accuracy in complex scenarios. For example, when extreme weather causes a sudden change in natural gas demand or a major policy adjustment triggers a transformation of the consumption pattern, the prediction accuracy of the model is severely damaged.

[0004] Although hybrid models attempt to combine the advantages of statistical and artificial intelligence methods, some studies combine functional autoregression and convolutional neural networks, or integrate seasonal adjustment models, long short-term memory networks, and grey models, etc., and use data decomposition techniques to handle the characteristics of complex consumption data, which improves the prediction performance to a certain extent. However, these hybrid models have complex structures and numerous parameters, and need to be finely tuned when applied in different scenarios, with poor generality, which greatly limits their wide application and efficient promotion, and increases the application cost and technical threshold for energy enterprises and planning departments.

[0005] Particularly crucial is that existing research as a whole lacks sufficient exploration of the rich, diverse, and dynamically changing patterns contained in natural gas demand data. Some only focus on the overall pattern and ignore local differences, or simply divide stages based on major events, without comprehensively analyzing pattern changes from a multi-dimensional and dynamic perspective. This limitation makes the model slow to respond and have poor accuracy when dealing with pattern variations caused by sudden policy changes, extreme weather impacts, and unexpected reversals of the economic situation, and cannot meet the urgent need for accurate and reliable demand prediction technology in the energy field. There is an urgent need for innovative methods to break through the bottleneck. Summary of the Invention

[0006] To solve the above problems, the present invention proposes a natural gas demand prediction method based on GPT-driven pattern-aware segmentation matching, achieving improvements in multiple aspects.

[0007] To achieve the above object, the technical solution adopted by the present invention is: a natural gas demand prediction method based on GPT-driven pattern-aware segmentation matching, including the steps of:

[0008] S10. Obtain the natural gas demand time series and input it into the GPT-based time series feature learning model GPT-TempFeat fine-tuned using Low-Rank Adaptation (LoRA) to capture the potential patterns in the natural gas demand time series while retaining the original knowledge of the natural gas demand time series;

[0009] S20. Utilize the time features extracted by GPT-TempFeat and dynamically segment the time series into different segments through a rolling window segmentation algorithm;

[0010] S30. For each identified segment, use LoRA fine-tuning to train an individual GPT-Forecast model to obtain a segment model;

[0011] S40. In the prediction stage, adopt a pattern matching method based on cosine similarity to identify the most suitable segment model for each input window; then use the selected segment model for prediction to obtain the natural gas demand prediction result.

[0012] Furthermore, a reconstruction loss function is adopted in the GPT-TempFeat model.

[0013] Furthermore, for the fine-tuning of the GPT-TempFeat model, Low-Rank Adaptation (LoRA) is used to fine-tune the GPT model to capture the potential patterns in the natural gas demand data, including:

[0014] Given a univariate natural gas demand time series X T , after normalization and embedding, it becomes E T ;

[0015] The GPT model processes the embedded sequence to generate an output H T , and calculates the loss function through the reconstruction error:

[0016]

[0017] where W T and b T are learnable parameters, θ T represents the model parameters, and L represents the sequence length.

[0018] Furthermore, in the rolling window segmentation algorithm, the rolling window is gradually expanded until a significant feature change is detected, that is, greater than a predefined feature difference threshold δ or reaching the maximum window size.

[0019] Furthermore, the extraction of temporal features adopts a max-pooling strategy:

[0020]

[0021] where X = (x 1 , …, x l ) represents the input sequence, Embed is the embedding function, φ represents the GPT transformation layer, f TF represents the GPT-based temporal feature extraction model, i represents the time series subscript, and d ff represents the dimension of the feed-forward neural network.

[0022] Furthermore, for the training of the GPT-Forecast model for each segment, based on the segments {seg 1 , seg 2 , …, seg n} obtained by segmentation, where n represents the number of segments, an independent GPT-Forecast model is trained for each segment, including:

[0023] For each segment X F , the GPT model processes the embedded sequence to generate the output H F , and generates the predicted value and calculates the loss through the following formula:

[0024]

[0025] where W F , b F and θ F represent the model parameters, represents the predicted value, L seg represents the length of the segment, i represents the segment subscript, is the SAMPE loss function. Furthermore, the loss function adopts the symmetric mean absolute percentage error.

[0026] Furthermore, in the prediction stage, based on the temporal feature extraction using cosine similarity, the segment model that best matches the input window is identified, including:

[0027] Calculate the cosine similarity between the input window feature and each segment feature :

[0028]

[0029] where simcos is the cosine similarity function, test k is the k-th test window, seg i is the i-th segment;

[0030] Select the segment seg with the highest similarity i The corresponding prediction model Make a prediction:

[0031]

[0032] is the predicted value, is the k-th prediction input window.

[0033] Beneficial effects of adopting this technical solution:

[0034] Existing statistical models are difficult to capture non-linearity and adapt to pattern changes. AI models lack consideration of data characteristics. Hybrid models are complex and have poor applicability, and generally ignore diverse dynamic patterns. This invention is significantly different. First, GPT is used to construct a time feature learning model (GPT-TempFeat) and fine-tune it. Utilize its powerful sequence modeling ability to perceive potential patterns, covering subtle local changes and mutations caused by external factors, solve the problem of adapting to complex patterns, and improve the comprehensiveness and accuracy of capturing data characteristics.

[0035] Based on this, an innovative rolling window segmentation algorithm is proposed. Dynamically identify the segmentation pattern according to GPT-TempFeat, overcome the limitations of static methods, accurately determine the pattern boundary, divide the time series into multiple segments of different patterns, construct a dedicated GPT prediction model (GPT-Forecast) for each segment and fine-tune it with LoRA, focus on learning the characteristics within the segment, enhance the model's pertinence and flexibility, and reduce the overall modeling complexity.

[0036] During prediction, use cosine similarity and GPT-TempFeat to extract features to match the optimal historical segment and model, discard the drawbacks of traditional matching that only rely on morphological similarity, and achieve accurate matching prediction. Verified by experiments on multiple datasets, the prediction accuracy and robustness of this framework far exceed traditional and advanced methods, effectively utilize diverse patterns to improve prediction quality, provide a more reliable basis for energy planning and decision-making, and strongly promote the development of natural gas demand prediction technology. Brief Description of the Drawings

[0037] Figure 1 is a schematic flow chart of a natural gas demand prediction method based on GPT-driven pattern perception segmentation and matching according to the present invention;

[0038] Figure 2 is a schematic diagram of the monthly natural gas consumption data of China, the United States, the United Kingdom, and Mexico in the embodiments of the present invention;

[0039] Figure 3 Schematic diagram of the rank-order statistical test for all models on 4 datasets in the embodiments of the present invention. Detailed implementation manners

[0040] To make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described below with reference to the accompanying drawings.

[0041] In this embodiment, as shown in Figure 1 a natural gas demand prediction method based on GPT-driven pattern perception segmentation matching is proposed, including the steps of:

[0042] S10. Obtain the natural gas demand time series and input it into the GPT-based time series feature learning model GPT-TempFeat fine-tuned using Low-Rank Adaptation (LoRA) to capture the potential patterns in the natural gas demand time series while retaining the original knowledge of the natural gas demand time series;

[0043] S20. Use the time features extracted by GPT-TempFeat and, through a rolling window segmentation algorithm, dynamically segment the time series into different segments;

[0044] S30. For each identified segment, use LoRA fine-tuning to train an individual GPT-Forecast model to obtain a segment model;

[0045] S40. In the prediction stage, adopt a pattern matching method based on cosine similarity to identify the most suitable segment model for each input window; then use the selected segment model for prediction to obtain the natural gas demand prediction result.

[0046] As an optimized solution of the above embodiment, a reconstruction loss function is adopted in the GPT-TempFeat model.

[0047] The fine-tuning of the GPT-TempFeat model uses Low-Rank Adaptation (LoRA) to fine-tune the GPT model to capture the potential patterns in the natural gas demand data, including:

[0048] Given a univariate natural gas demand time series X T , after normalization and embedding, obtain E T ;

[0049] After the GPT model processes the embedded sequence, generate the output H T , and calculate the loss function through the reconstruction error:

[0050]

[0051] where W T and bT are learnable parameters, θ T denotes the model parameters, and L denotes the sequence length.

[0052] As an optimized solution of the above embodiment, for the rolling window segmentation algorithm, by gradually expanding the rolling window until a significant feature change is detected, that is, greater than a predefined feature difference threshold δ or reaching the maximum window size.

[0053] The extraction of time features adopts the max pooling strategy:

[0054]

[0055] where X = (x 1 , …, x l ) represents the input sequence, Embed is the embedding function, φ represents the GPT transformation layer, f TF represents the time series feature extraction model based on GPT, i represents the time series subscript, and d ff represents the dimension of the feed-forward neural network.

[0056] As an optimized solution of the above embodiment, for the training of the GPT-Forecast model for each segment, based on the segments {seg 1 , seg 2 , …, seg n} obtained by segmentation, where n represents the number of segments, train an independent GPT-Forecast model for each segment, including:

[0057] For each segment X F , after the GPT model processes the embedded sequence, it generates the output H F , and generates the predicted value and calculates the loss through the following formula:

[0058]

[0059] where W F , b F and θ F denote the model parameters, denotes the predicted value, and L seg denotes the length of the segment, and i denotes the subscript of the segment. is the SAMPE loss function. The loss function adopts the symmetric mean absolute percentage error.

[0060] As an optimized solution of the above embodiment, in the prediction stage, based on the time feature extraction using the cosine similarity, identify the segment model that best matches the input window, including:

[0061] Calculate the input window feature and each segment feature Cosine similarity:

[0062]

[0063] where sim cos is the cosine similarity function, test k is the k-th test window, and seg i is the i-th segment;

[0064] Select the segment seg i corresponding to the highest similarity, and the prediction model model segi is used for prediction:

[0065]

[0066] is the predicted value, and is the k-th predicted input window.

[0067] The present invention has conducted extensive experiments on the monthly natural gas consumption datasets in four countries (China, USA, UK, and Mexico), as Figure 2 shown, verifying the effectiveness of the GPT-PASMF framework. The experimental results show that GPT-PASMF outperforms traditional statistical models and advanced deep learning models in most cases.

[0068] (1) Overall prediction results: See Table 1. The mean absolute percentage error (MAPE) of GPT-PASMF on the four datasets is 0.067 for China, 0.053 for the USA, 0.102 for the UK, and 0.121 for Mexico. Among them, the prediction effect of the US dataset is the best due to its obvious periodicity; while the prediction effect of the Mexican dataset is relatively poor due to the lack of obvious trends and periodicity.

[0069] (2) Segment matching accuracy: See Table 1. GPT-PASMF also performs well in segment matching. For example, on the US dataset, GPT-PASMF reaches 1.0 in the Top-3 matching accuracy (MACC), indicating that the model can accurately identify the historical segment most similar to the input window.

[0070] Table 1 Prediction metrics of GPT-PASMF on four datasets

[0071]

[0072] (3) Comparison with the benchmark models: As shown in Table 2, GPT-PASMF outperforms the traditional ARIMA, Prophet, LSTM, GRU, RF, and XGBoost models, as well as advanced deep learning models such as N-BEATS, TCN, and TFT on all four datasets. Especially on the Chinese and UK datasets, GPT-PASMF performs best in all evaluation metrics.

[0073] (4) Ablation experiments: To verify the effectiveness of each component of GPT-PASMF, ablation experiments were also conducted in this paper. As shown in Table 2, the results show that the PASMF component of GPT-PASMF significantly improves the prediction performance, especially when dealing with complex patterns and structural changes. For example, in the absence of the PASMF component, the MAPE of the GPT-Forecast model increases by 0.03 on the UK dataset.

[0074] Table 2 Performance comparison of all models on four datasets

[0075]

[0076] (5) Statistical analysis: Through the Friedman test and the Nemenyi post hoc test, this paper further verifies the significant advantages of GPT-PASMF on multiple datasets and evaluation metrics. The results show that the ranking of GPT-PASMF is significantly higher than that of other models in all three evaluation metrics (MAPE, MAE, and RMSE), as Figure 3 shown.

[0077] The above shows and describes the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. What is described in the above embodiments and the specification only illustrates the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements all fall within the scope of the present invention claimed. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.

Claims

1. A natural gas demand forecasting method based on GPT driving mode perception segmentation matching, characterized in that: Includes steps: S10, obtain the natural gas demand time series and input it into the GPT-TempFeat, a time series feature learning model based on GPT fine-tuned using low-rank adaptive LoRA, to capture the potential patterns in the natural gas demand time series while retaining the original knowledge of the natural gas demand time series; S20, uses the temporal features extracted by GPT-TempFeat to dynamically segment the time series into different segments through a rolling window segmentation algorithm; S30, for each identified segment, use LoRA fine-tuning to train an individual GPT-Forecast model to obtain a segment model; S40, in the prediction stage, a pattern matching method based on cosine similarity is used to identify the most suitable segment model for each input window; then the selected segment model is used to perform prediction to obtain a natural gas demand prediction result.

2. According to claim 1, a natural gas demand forecasting method based on GPT driving mode perception segmentation matching is characterized in that: The reconstruction loss function is adopted in the GPT-TempFeat model.

3. According to claim 2, a natural gas demand forecasting method based on GPT driving mode perception segmentation matching is characterized in that: Fine-tuning of the GPT-TempFeat model,The GPT model is fine-tuned using low-rank adaptation LoRA to capture the potential patterns in the natural gas demand data, including: Given a univariate natural gas demand time series X T , after normalization and embedding, we get E T ; The GPT model processes the embedded sequence and generates the output H T , and calculate the loss function by reconstruction error: in W T and b T is a learnable parameter, θ T represents the model parameters, and L represents the sequence length.

4. According to claim 1, a natural gas demand forecasting method based on GPT driving mode perception segmentation matching is characterized in that: The rolling window segmentation algorithm gradually expands the rolling window until significant feature changes are detected.

5. A natural gas demand forecasting method based on GPT driving mode perception segmentation matching according to claim 4, characterized in that: The maximum pooling strategy is used to extract temporal features: Where X=(x1,…,x l ) represents the input sequence, Embed is the embedding function, φ represents the GPT transformation layer, and f TF represents the GPT-based temporal feature extraction model, i represents the time series subscript, d ff Represents the dimension of the feedforward neural network.

6. A natural gas demand forecasting method based on GPT driving mode perception segmentation matching according to claim 1, characterized in that: GPT-Forecast model training for each segment is based on the segmentation {seg1, seg2, …, seg n }, n represents the number of segments, and a separate GPT-Forecast model is trained for each segment, including: For each segment X F , the GPT model processes the embedded sequence and generates the output H F , and generate predictions and calculate losses using the following formula: Where W F , b F and θ F represents the model parameters, represents the predicted value, L seg Indicates the length of the fragment, i indicates the subscript of the fragment, is the SAMPE loss function.

7. A natural gas demand forecasting method based on GPT driving mode perception segmentation matching according to claim 6, characterized in that: The loss function uses the symmetric mean absolute percentage error.

8. The natural gas demand forecasting method based on GPT driving mode perception segmentation matching according to claim 1 is characterized in that: In the prediction phase, cosine similarity is used to extract temporal features and identify the segmentation model that best matches the input window, including: Calculate input window characteristics With each segment feature Cosine similarity of: Among them, sim cos is the cosine similarity function, test k is the kth test window, seg i is the i-th fragment; Select the segment with the highest similarity i The corresponding prediction model Make predictions: is the predicted value, is the k-th prediction input window.

Images