A natural gas demand prediction method based on GPT driving mode perception segmentation matching
By using GPT-based time-series feature learning and a rolling window segmentation algorithm, natural gas demand time series are dynamically segmented and individual models are trained, solving the problems of complexity and accuracy in natural gas demand forecasting in existing technologies and achieving more efficient forecasting results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SICHUAN UNIV
- Filing Date
- 2025-02-19
- Publication Date
- 2026-04-17
AI Technical Summary
Existing natural gas demand forecasting methods struggle to accurately capture subtle correlations in demand data when faced with complex nonlinear patterns and dynamic changes. Furthermore, existing hybrid models are complex in structure, have numerous parameters, and lack versatility, failing to meet the precise needs of energy planning.
We employ a GPT-based temporal feature learning model (GPT-TempFeat) combined with a rolling window segmentation algorithm to dynamically segment time series and train individual GPT-Forecast models. We then use cosine similarity to match the optimal model for prediction, thereby enhancing the model's relevance and flexibility.
It improves the accuracy and robustness of natural gas demand forecasting, reduces model complexity, and provides a more reliable basis for energy planning.
Smart Images
Figure CN120146262B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of data processing technology, and in particular relates to a natural gas demand forecasting method based on GPT-driven mode perception segmentation and matching. Background Technology
[0002] Existing methods in the field of natural gas demand forecasting have several shortcomings. Statistical models, such as ARIMA and its extensions, and various regression models, have been widely used and perform reasonably well in handling linear relationships. However, these models fall short when faced with the complex nonlinear patterns of natural gas demand in reality, which are influenced by a complex interplay of factors including economic growth, weather conditions, technological innovation, and policy adjustments. They struggle to accurately capture the subtle nonlinear relationships within demand data and cannot adapt to dynamic changes in demand patterns in a timely manner, resulting in significant discrepancies between forecasts and actual demand. Consequently, they fail to provide solid support for critical decisions such as energy planning.
[0003] While AI models based on machine learning and deep learning have advantages, such as their ability to handle nonlinear relationships in big data and adapt to diverse data characteristics, in practical applications, these models often rely heavily on their own learning mechanisms and lack in-depth analysis of data characteristics. When consumption patterns suddenly change or are subjected to strong external shocks, the models cannot keenly perceive the inherent logic of data changes, causing a significant drop in prediction accuracy in complex scenarios. For example, when extreme weather causes a sudden change in natural gas demand or major policy adjustments trigger a transformation in consumption patterns, the model's prediction accuracy is severely compromised.
[0004] While hybrid models attempt to combine the strengths of statistical and artificial intelligence methods—with some studies integrating functional autoregression with convolutional neural networks, or combining seasonally adjusted models, long short-term memory networks, and grey models, and employing data decomposition techniques to handle complex consumer data characteristics, thus improving predictive performance to some extent—these hybrid models are structurally complex with numerous parameters. They require meticulous tuning for different application scenarios, resulting in poor versatility and significantly limiting their widespread application and efficient promotion. This increases the application costs and technical barriers for energy companies and planning departments.
[0005] Crucially, existing research has not adequately explored the rich and diverse, dynamically changing patterns inherent in natural gas demand data. Some studies focus solely on overall patterns while ignoring local variations, or simply divide periods based on major events, failing to analyze pattern changes comprehensively from a multi-faceted and dynamic perspective. This limitation makes models slow to react and inaccurate when dealing with pattern variations caused by sudden policy changes, extreme weather shocks, or unexpected economic reversals. These limitations fail to meet the urgent need in the energy sector for precise and reliable demand forecasting technologies, necessitating innovative methods to overcome these bottlenecks. Summary of the Invention
[0006] To address the aforementioned issues, this invention proposes a natural gas demand forecasting method based on GPT-driven mode perception segmentation and matching, achieving improvements in several aspects.
[0007] To achieve the above objectives, the technical solution adopted by this invention is: a natural gas demand forecasting method based on GPT-driven mode sensing segmentation and matching, comprising the following steps:
[0008] S10: Obtain the natural gas demand time series and input it into the GPT-TempFeat time series feature learning model, which is fine-tuned using low-rank adaptive LoRA, to capture the latent patterns in the natural gas demand time series while preserving the original knowledge of the natural gas demand time series.
[0009] S20 utilizes the time features extracted by GPT-TempFeat and dynamically segments the time series into different segments using a rolling window segmentation algorithm;
[0010] S30, For each identified segment, LoRA is used to fine-tune and train the individual GPT-Forecast model to obtain the segment model;
[0011] S40. In the prediction phase, a pattern matching method based on cosine similarity is used to identify the most suitable segment model for each input window; then, the selected segment model is used for prediction to obtain the natural gas demand prediction results.
[0012] Furthermore, a reconstruction loss function is used in the GPT-TempFeat model.
[0013] Furthermore, the GPT-TempFeat model is fine-tuned using low-rank adaptive LoRA to capture latent patterns in natural gas demand data, including:
[0014] Given a univariate natural gas demand time series X T After normalization and embedding, E is obtained. T ;
[0015] The GPT model processes the embedded sequence to generate output H. T And calculate the loss function through reconstruction error:
[0016] ;
[0017] in , and These are learnable parameters. This represents the model parameters, and L represents the sequence length.
[0018] Furthermore, the rolling window segmentation algorithm gradually expands the rolling window until a significant feature change is detected, i.e., greater than a predefined feature difference threshold. Or it may reach the maximum window size.
[0019] Furthermore, the extraction of temporal features employs a max-pooling strategy:
[0020] ;
[0021] Where X=(x1,…,x l () represents the input sequence, and Embed is the embedding function. f represents the GPT transform layer. TF This represents a time series feature extraction model based on GPT, where i represents the time series index, and d... ff This represents the dimension of the feedforward neural network.
[0022] Furthermore, for each segment, the GPT-Forecast model is trained based on the segment {seg1, seg2, ..., seg...} obtained from the segmentation. n}, where n represents the number of segments, and an independent GPT-Forecast model is trained for each segment, including:
[0023] For each segment X F The GPT model processes the embedded sequences to generate the output H. F The predicted value is generated and the loss is calculated using the following formula:
[0024] ;
[0025] ;
[0026] Among them, W F b F and Indicates model parameters, L represents the predicted value. seg This indicates the length of the segment, and 'i' represents the index of the segment. This is the SMAPE loss function.
[0027] Furthermore, the loss function employs a symmetric mean absolute percentage error.
[0028] Furthermore, during the prediction phase, cosine similarity is used to extract time features and identify the segmented model that best matches the input window, including:
[0029] Calculate input window features With each segment feature Cosine similarity:
[0030] ;
[0031] Where, sim cos For cosine similarity function, test k For the k-th test window, seg i For the i-th segment;
[0032] Select the segment with the highest similarity (seg). i Corresponding prediction model Make a prediction:
[0033] ;
[0034] For predicted values, This is the k-th prediction input window.
[0035] The beneficial effects of adopting this technical solution are:
[0036] Existing statistical models struggle to capture nonlinearity and adapt to changing patterns, AI models lack consideration for data characteristics, and hybrid models are complex, have poor applicability, and generally ignore diverse dynamic patterns. This invention differs significantly. First, it employs GPT to construct and fine-tune a time-feature-based learning model (GPT-TempFeat). Leveraging GPT's powerful sequence modeling capabilities, it perceives latent patterns, encompassing subtle local changes and externally induced mutations, thus solving the challenge of adapting to complex patterns and improving the comprehensiveness and accuracy of data feature capture.
[0037] Based on this, an innovative rolling window segmentation algorithm is developed. It dynamically identifies segmentation patterns using GPT-TempFeat, overcomes the limitations of static methods, accurately determines pattern boundaries, subdivides the time series into multiple segments with different patterns, constructs a dedicated GPT prediction model (GPT-Forecast) for each segment and fine-tunes it using LoRA, focuses on learning the characteristics within the segment, enhances the model's relevance and flexibility, and reduces the overall modeling complexity.
[0038] During prediction, cosine similarity and GPT-TempFeat are used to extract features for optimal historical segments and models, overcoming the drawbacks of traditional matching based solely on morphological similarity, and achieving accurate matching predictions. Experiments on multiple datasets have validated that the prediction accuracy and robustness of this framework far surpass traditional and advanced methods. It effectively utilizes diverse patterns to improve prediction quality, providing a more reliable basis for energy planning and decision-making, and powerfully promoting the development of natural gas demand forecasting technology. Attached Figure Description
[0039] Figure 1 This is a schematic diagram of a natural gas demand forecasting method based on GPT-driven mode perception segmentation and matching according to the present invention.
[0040] Figure 2 This is a schematic diagram illustrating the monthly natural gas consumption data for China, the United States, the United Kingdom, and Mexico in an embodiment of the present invention.
[0041] Figure 3 This is a schematic diagram illustrating the ordinal value statistical test of all models on four datasets in an embodiment of the present invention. Detailed Implementation
[0042] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described below with reference to the accompanying drawings.
[0043] In this embodiment, see Figure 1 As shown, this invention proposes a natural gas demand forecasting method based on GPT-driven mode perception segmentation and matching, including the following steps:
[0044] S10: Obtain the natural gas demand time series and input it into the GPT-TempFeat time series feature learning model, which is fine-tuned using low-rank adaptive LoRA, to capture the latent patterns in the natural gas demand time series while preserving the original knowledge of the natural gas demand time series.
[0045] S20 utilizes the time features extracted by GPT-TempFeat and dynamically segments the time series into different segments using a rolling window segmentation algorithm;
[0046] S30, For each identified segment, LoRA is used to fine-tune and train the individual GPT-Forecast model to obtain the segment model;
[0047] S40. In the prediction phase, a pattern matching method based on cosine similarity is used to identify the most suitable segment model for each input window; then, the selected segment model is used for prediction to obtain the natural gas demand prediction results.
[0048] As an optimization of the above embodiments, a reconstruction loss function is adopted in the GPT-TempFeat model.
[0049] Fine-tuning of the GPT-TempFeat model, using low-rank adapted LoRA to fine-tune the GPT model to capture latent patterns in natural gas demand data, including:
[0050] Given a univariate natural gas demand time series X T After normalization and embedding, E is obtained. T ;
[0051] The GPT model processes the embedded sequence to generate output H. T And calculate the loss function through reconstruction error:
[0052] ;
[0053] in W T and b T These are learnable parameters. This represents the model parameters, and L represents the sequence length.
[0054] As an optimization of the above embodiments, the rolling window segmentation algorithm gradually expands the rolling window until a significant feature change is detected, i.e., greater than a predefined feature difference threshold. Or it may reach the maximum window size.
[0055] The extraction of temporal features employs a max-pooling strategy:
[0056] ;
[0057] Where X=(x1,…,x l () represents the input sequence, and Embed is the embedding function. f represents the GPT transform layer. TF This represents a time series feature extraction model based on GPT, where i represents the time series index, and d... ff This represents the dimension of the feedforward neural network.
[0058] As an optimization of the above embodiment, the GPT-Forecast model is trained for each segment, based on the segment {seg1, seg2, ..., seg...} obtained from the segmentation. n}, where n represents the number of segments, and an independent GPT-Forecast model is trained for each segment, including:
[0059] For each segment X F The GPT model processes the embedded sequences to generate the output H. F The predicted value is generated and the loss is calculated using the following formula:
[0060] ;
[0061] ;
[0062] Among them, W F b F and Indicates model parameters, L represents the predicted value. seg This indicates the length of the segment, and 'i' represents the index of the segment. The loss function is SMAPE. The loss function uses symmetric mean absolute percentage error.
[0063] As an optimization of the above embodiments, in the prediction stage, cosine similarity is used to extract time features and identify the segmented model that best matches the input window, including:
[0064] Calculate input window features With each segment feature Cosine similarity:
[0065] ;
[0066] Where, sim cos For cosine similarity function, test k For the k-th test window, seg i For the i-th segment;
[0067] Select the segment with the highest similarity (seg). i Corresponding prediction model Make a prediction:
[0068] ;
[0069] For predicted values, This is the k-th prediction input window.
[0070] This invention has been extensively tested on monthly natural gas consumption datasets from four countries (China, USA, UK, and Mexico), such as... Figure 2 As shown, the effectiveness of the GPT-PASMF framework is verified. Experimental results demonstrate that GPT-PASMF outperforms traditional statistical models and advanced deep learning models in most cases.
[0071] (1) Overall prediction results: As shown in Table 1, the mean absolute percentage error (MAPE) of GPT-PASMF on the four datasets is 0.067 for China, 0.053 for the United States, 0.102 for the United Kingdom, and 0.121 for Mexico. Among them, the United States dataset has the best prediction effect due to its obvious periodicity, while the Mexican dataset has a relatively poor prediction effect due to the lack of obvious trends and periodicity.
[0072] (2) Segment matching accuracy: As shown in Table 1, GPT-PASMF also performs well in segment matching. For example, on the US dataset, GPT-PASMF achieved a Top-3 matching accuracy (MACC) of 1.0, indicating that the model can accurately identify the historical segments most similar to the input window.
[0073] Table 1. Predictive metrics of GPT-PASMF on four datasets.
[0074]
[0075] (3) Comparison with benchmark models: As shown in Table 2, GPT-PASMF outperforms traditional ARIMA, Prophet, LSTM, GRU, RF and XGBoost models, as well as advanced deep learning models such as N-BEATS, TCN and TSMixer, on all four datasets. In particular, GPT-PASMF performs best on all evaluation metrics on the Chinese and British datasets.
[0076] (4) Ablation Experiments: To verify the effectiveness of each component of GPT-PASMF, ablation experiments were also conducted. As shown in Table 2, the results show that the PASMF component of GPT-PASMF significantly improves the prediction performance, especially when dealing with complex patterns and structural changes. For example, without the PASMF component, the MAPE of the GPT-Forecast model on the UK dataset increased by 0.03.
[0077] Table 2 shows the performance comparison of all models on the four datasets.
[0078]
[0079] (5) Statistical Analysis: Through Friedman's test and Nemenyi post-hoc test, this paper further validates the significant advantages of GPT-PASMF on multiple datasets and evaluation metrics. The results show that GPT-PASMF ranks significantly higher than other models on all three evaluation metrics (MAPE, MAE, and RMSE), such as... Figure 3 As shown.
[0080] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.
Claims
1. A natural gas demand forecasting method based on GPT-driven mode perception segmentation and matching, characterized in that, Including the following steps: S10: Obtain the natural gas demand time series and input it into the GPT-TempFeat time series feature learning model, which is fine-tuned using low-rank adaptive LoRA, to capture the latent patterns in the natural gas demand time series while preserving the original knowledge of the natural gas demand time series. include: Given a univariate natural gas demand time series X T After normalization and embedding, E is obtained. T ; The GPT model processes the embedded sequence to generate output H. T And calculate the loss function through reconstruction error: ； in W T and b T These are learnable parameters. This represents the model parameters, and L represents the sequence length; S20 utilizes the time features extracted by GPT-TempFeat and dynamically segments the time series into different segments using a rolling window segmentation algorithm; S30, For each identified segment, LoRA is used to fine-tune and train the individual GPT-Forecast model to obtain the segment model; For each segment, the GPT-Forecast model is trained based on the segment {seg1, seg2, ..., seg...} obtained from the segmentation. n }, where n represents the number of segments, and an independent GPT-Forecast model is trained for each segment, including: For each segment X F The GPT model processes the embedded sequences to generate the output H. F The predicted value is generated and the loss is calculated using the following formula: ； ； Among them, W F b F and Indicates model parameters, L represents the predicted value. seg This indicates the length of the segment, and 'i' represents the index of the segment. The SMAPE loss function; S40. In the prediction phase, a pattern matching method based on cosine similarity is used to identify the most suitable segment model for each input window; then, the selected segment model is used for prediction to obtain the natural gas demand prediction results.
2. The natural gas demand forecasting method based on GPT-driven mode perception segmentation and matching according to claim 1, characterized in that, The rolling window segmentation algorithm expands the rolling window incrementally until a significant feature change is detected.
3. The natural gas demand forecasting method based on GPT-driven mode perception segmentation and matching according to claim 2, characterized in that, The extraction of temporal features employs a max-pooling strategy: ； Where X=(X1,…,X) L () represents the input sequence, and Embed is the embedding function. f represents the GPT transform layer. TF This represents a time series feature extraction model based on GPT, where j represents the time series index, and d... ff This represents the dimension of the feedforward neural network.
4. The natural gas demand forecasting method based on GPT-driven mode perception segmentation and matching according to claim 1, characterized in that, In the prediction phase, cosine similarity is used to extract time features and identify the segmented model that best matches the input window, including: Calculate input window features With each segment feature Cosine similarity: ； Where, sim cos For cosine similarity function, test k For the k-th test window, seg i For the i-th segment; Select the segment with the highest similarity (seg). i Corresponding prediction model Make a prediction: ； For predicted values, This is the k-th prediction input window.