A natural gas demand prediction method and system based on contrastive learning

Abstract

The present application relates to the technical field of artificial intelligence predicting energy demand, and particularly relates to a natural gas demand prediction method and system based on contrast learning. The original time series data is defined, the original time series data is preprocessed to obtain complete time series data, the complete time series data is subjected to contrast learning task design and training to obtain an overall target loss function, the overall target loss function is adjusted using a root mean square error loss to obtain a natural gas demand prediction model based on contrast learning, and the natural gas demand is predicted according to the natural gas demand prediction model based on contrast learning to obtain a natural gas demand result. The present application solves the technical problem of inaccurate prediction caused by excessive noise in the natural gas data set by fusing contrast learning and noise filtering design, and achieves the technical effect of accurately predicting natural gas demand.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence-based energy demand forecasting technology, and in particular to a natural gas demand forecasting method and system based on comparative learning. Background Technology

[0002] Achieving efficient natural gas dispatch requires accurate demand forecasting, but two key challenges need to be addressed. First, industrial datasets often contain significant noise, which can mask true natural gas consumption patterns and complicate accurate demand forecasting. Noise in the data can originate from various sources, such as sensor errors, data transmission problems, or data logging issues. These inaccuracies can distort actual usage data, making it difficult to identify genuine consumption trends and anomalies. Second, natural gas consumption patterns vary considerably across different industries. For example, in industrial applications, natural gas is used for power generation, heating, steam production, and machinery operation, where usage is typically driven by varying production cycles and operational demands, resulting in unpredictable spikes and troughs. In the commercial sector, natural gas is primarily used for cooking, heating, and hot water supply, with demand influenced by operating hours and seasonal variations, such as increased heating demand during colder periods. These differences in consumption patterns across industries complicate the modeling process and reduce the forecasting accuracy of traditional methods.

[0003] Traditional methods for natural gas demand forecasting, such as time series analysis and regression models, lay the foundation for forecasting by utilizing historical data to identify trends and patterns. With the development of deep learning, recent research has focused primarily on models based on recurrent neural networks, convolutional neural networks, and the Transformer architecture and its variants. These models have shown significant improvements in prediction accuracy and generalization compared to traditional approaches. However, most of these methods still rely on end-to-end learning methods. In recent years, self-supervised representation learning has made significant progress in computer vision and natural language processing, and is increasingly being applied to time series forecasting to improve prediction performance.

[0004] Most academic research on time series forecasting is based on publicly available datasets, neglecting the significant noise and industry-specific issues commonly found in real-world industrial datasets. Current state-of-the-art research using contrastive learning for time series forecasting typically combines masking modeling with contrastive learning to achieve better representations and thus better predictions. However, due to noise, masking modeling and contrastive learning may actually lead to learning incorrect features, resulting in worse representations and failing to achieve better predictions. These studies also rarely consider the relationships arising from user industry characteristics in multi-user scenarios, leading to underutilization of existing features and suboptimal representation learning and prediction results.

[0005] In conclusion, forecasting natural gas demand based on real-world industrial and commercial natural gas consumption data is an innovative topic with significant research and application value. Summary of the Invention

[0006] The present invention aims to solve at least one of the technical problems existing in the related art. To this end, the present invention provides a natural gas demand forecasting method and system based on contrastive learning.

[0007] This invention provides a natural gas demand forecasting method based on contrastive learning, comprising the following steps:

[0008] S1: Define the original time series data, preprocess the original time series data to obtain complete time series data;

[0009] S2: Design and train a contrastive learning task on the complete time series data to obtain the overlapping sampling contrastive learning loss function, the noise enhancement contrastive learning loss function, and the Smooth L1 loss function;

[0010] S3: Based on the overlapping sampling contrastive learning loss function, the noise enhancement contrastive learning loss function, and the SmoothL1 loss function, obtain the overall target loss function, and train the pre-trained model based on the overall target loss function.

[0011] S4: Calculate the root mean square error loss, adjust the pre-trained model, and obtain a natural gas demand forecasting model based on contrastive learning;

[0012] S5: Predict natural gas demand based on the natural gas demand prediction model based on contrastive learning to obtain the natural gas demand result.

[0013] According to the present invention, a natural gas demand forecasting method based on contrastive learning is provided, wherein S1 includes the following steps:

[0014] S11: Organize the collected time series data to obtain a length of... raw time series data , ,in, The data is the original time series data. The data ordinal number of the original time series data. ;

[0015] S12: Filter out raw time series data whose length is less than the length threshold;

[0016] The Z-score of the original time series data is calculated using the following formula:

[0017]

[0018] in, For Z-scores, The average value of the original time series data. The standard deviation of the original time series data.

[0019] Calculate the proportion of the original time series data whose Z-score is greater than the Z-score threshold. :

[0020]

[0021] in, The number of data points in a set of original time series data whose Z-scores are greater than the Z-score threshold;

[0022] like Greater than the Z-score proportion threshold, The corresponding original time series data is then removed, if Less than or equal to the Z-score proportion threshold The corresponding original time series data is retained.

[0023] The filtered raw time series data is obtained;

[0024] S13: Use the SAITS model to impute missing values ​​in the filtered original time series data to obtain the complete time series data.

[0025] According to the present invention, a natural gas demand forecasting method based on contrastive learning is provided, wherein S2 includes the following steps:

[0026] S21: Sample the complete time series data at the same sampling interval and sampling size to obtain several initial time period samples of the same length, and generate multiple overlapping time period samples using a sliding window method. The initial time period samples and the overlapping time period samples constitute the time period samples. The time period samples are processed Encode and generate embedding vectors

[0027]

[0028] in, for Encoding function, For learnable location embedding;

[0029] The time period sample There exists For the first overlapping sample Second Sample The first sample and the second sample constitute the sampling sample. ,in, The number of overlapping logarithms;

[0030] The sample Random selection Each sampled sample is used as a positive example sample. The remaining sampled samples are used as negative samples. ,in, and The sample ordinal number, , ;

[0031] The first sample The corresponding first embedding vector is ;

[0032] The second sample The corresponding second embedding vector is ;

[0033] The positive sample The corresponding positive embedding vector is ;

[0034] The negative sample The corresponding negative embedding vector is ;

[0035] S22: Perform false negative sample filtering on the positive and negative embedding vectors to obtain the false negative sample set. ;

[0036] S23: By performing comparative learning on the first embedding vector, the second embedding vector, the positive embedding vector, and the negative embedding vector after removing the false negative sample set, the overlapping sampling contrastive learning loss function is obtained. ;

[0037] S24: Perform noise enhancement comparison on the positive embedding vector and the negative embedding vector to obtain the reinforced positive embedding vector; perform noise enhancement contrastive learning on the positive embedding vector, the second embedding vector, and the reinforced positive embedding vector to obtain the noise enhancement contrastive learning loss function. ;

[0038] S25: Using the positive embedding vector, negative embedding vector, and reinforced positive embedding vector, noise filtering is performed on the samples in the time period to obtain the Smooth L1 loss function. .

[0039] According to the present invention, a natural gas demand forecasting method based on contrastive learning is provided, wherein S22 includes the following steps:

[0040] S221: Calculate the cosine similarity between the positive and negative embedding vectors:

[0041]

[0042] in, Cosine similarity of the embedded vectors. Take the modulus of the vector;

[0043] S222: Obtain the first set of false negative embedding vectors :

[0044]

[0045]

[0046] in, The top-ranked cosine similarity of the computed embedding vectors A set of cosine similarities, As the ranking threshold, This is a preset ranking ratio hyperparameter;

[0047] S223: Obtain the second set of pseudo-negative embedding vectors :

[0048]

[0049] in, This indicates the industry corresponding to the positive or negative embedding vector. Indicates the same;

[0050] S224: Obtain the false negative sample set using the first false negative embedding vector set and the second false negative embedding vector set. :

[0051] .

[0052] This invention provides a natural gas demand forecasting method based on contrastive learning, wherein the overlapping sampling contrastive learning loss function... The calculation formula is:

[0053]

[0054] in, This indicates that the embedding vector does not contain the set of false negative samples. This refers to the temperature parameter.

[0055] According to the present invention, a natural gas demand forecasting method based on contrastive learning is provided, wherein S24 includes the following steps:

[0056] S241: The enhanced positive embedding vector is obtained by mixing the negative embedding vector into the positive embedding vector. The calculation formula is:

[0057]

[0058] in, These are weight parameters;

[0059] S242: Perform noise enhancement contrastive learning on the positive embedding vector, negative embedding vector, and reinforced positive embedding vector to obtain the noise enhancement contrastive learning loss function. The calculation formula is:

[0060] .

[0061] According to the present invention, a natural gas demand forecasting method based on contrastive learning is provided, wherein S25 includes the following steps:

[0062] S251: Training the decoder, calculated using the following formula:

[0063]

[0064]

[0065]

[0066] in, For querying the matrix, The key matrix, For value matrices, To query the weight matrix of the matrix, The weight matrix is ​​the key matrix. The weight matrix is ​​the value matrix;

[0067] S252: Calculate the attention score of the decoder using an attention mechanism.

[0068]

[0069] in, Let be the attention function. For activation function, Let be the dimension of the key matrix. This is the transpose of the matrix;

[0070] S253: Use a multilayer perceptron to denoise the attention score and the positive embedding vector:

[0071]

[0072] in, These are the denoised time period samples. It is a multilayer perceptron. For layer normalization operation;

[0073] S254: Samples for a given time period The Smooth L1 loss function is obtained by performing noise filtering training. The calculation formula is:

[0074]

[0075] in, These are the preset hyperparameters for the Smooth L1 loss.

[0076] This invention provides a natural gas demand forecasting method based on contrastive learning, wherein the overall objective loss function... The formula is:

[0077]

[0078] in, As the first parameter, For the second parameter,

[0079] Training was performed using the Adam optimizer, when When the value is minimized, the pre-trained model is considered to have converged, and the resulting pre-trained model is denoted as ( ). .

[0080] According to the present invention, a natural gas demand forecasting method based on contrastive learning is provided, wherein S4 includes the following steps:

[0081]

[0082]

[0083] in, The prediction results obtained by calculating the samples of the time period using the model constructed using the overall objective loss function are as follows: This is the true result of the original time series data. Let the root mean square error loss function be . This represents the distance from the ordinal position of the original time series data to the starting point of the prediction. It is a pre-trained model function, which is achieved by adjusting the first parameter and the second parameter. The minimum value is considered the model convergence, resulting in the natural gas demand forecasting model based on contrastive learning.

[0084] This invention also provides a natural gas demand forecasting system based on contrastive learning, comprising:

[0085] Time series processing module: Defines raw time series data, preprocesses the raw time series data to obtain complete time series data;

[0086] Loss function calculation module: Performs comparative learning task design and training on the complete time series data to obtain the overlapping sampling comparative learning loss function, the noise enhancement comparative learning loss function, and the Smooth L1 loss function; Based on the overlapping sampling comparative learning loss function, the noise enhancement comparative learning loss function, and the Smooth L1 loss function, the overall target loss function is obtained; and a pre-trained model is obtained by training based on the overall target loss function.

[0087] Model fine-tuning module: Calculates the root mean square error loss, adjusts the pre-trained model, and obtains a natural gas demand forecasting model based on contrastive learning;

[0088] Data calculation module: Predicts natural gas demand based on the comparative learning-based natural gas demand prediction model, and obtains the natural gas demand results.

[0089] The above-described one or more technical solutions in the embodiments of the present invention have at least one of the following technical effects:

[0090] This invention provides a natural gas demand forecasting method and system based on contrastive learning. By combining three loss functions—overlapping sampling contrastive learning loss function, noise-enhanced contrastive learning loss function, and Smooth L1 loss loss function—and adjusting the proportions of these three functions using root mean square error, the following advantages are achieved:

[0091] 1. Innovative pre-training scheme: This invention proposes to combine contrastive learning with noise filtering tasks, which makes the prediction task more robust to noise. Compared with previous time series prediction based on contrastive learning, combining noise filtering tasks with contrastive learning tasks can better enhance the robustness to noise that is prevalent in the dataset, obtain better representation, and thus make better predictions of users' future natural gas demand.

[0092] 2. Multi-user dataset adaptation: This invention addresses the difference between the dataset and conventional publicly available time-series datasets, which are univariate multi-user datasets. It makes special adaptations for this type of dataset and specifies corresponding data partitioning strategies, dividing the test set according to date, so that it can be better applied in the corresponding scenarios.

[0093] 3. Effective Negative Sample Removal Strategy: This invention proposes to remove negative samples in comparative learning based on two indicators: user industry information and sample similarity. This approach makes more effective use of existing information while avoiding the incorrect identification of potentially positive samples as negative or positive samples in the calculation, thus achieving suboptimal results and ultimately obtaining better prediction results.

[0094] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0095] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0096] Figure 1 A flowchart illustrating a natural gas demand forecasting method based on contrastive learning provided by this invention;

[0097] Figure 2 This is a structural block diagram of a natural gas demand forecasting device based on contrastive learning provided by the present invention.

[0098] Figure label:

[0099] 101. Time series processing module; 102. Loss function calculation module; 103. Model fine-tuning module; 104. Data calculation module. Detailed Implementation

[0100] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention. The following embodiments are used to illustrate this invention but cannot be used to limit the scope of this invention.

[0101] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0102] The following is combined with Figure 1 and Figure 2 This invention is described.

[0103] Example

[0104] like Figure 1 As shown, this invention provides a natural gas demand forecasting method based on contrastive learning:

[0105] S1: Define the original time series data, preprocess the original time series data to obtain complete time series data;

[0106] S2: Design and train a contrastive learning task on the complete time series data to obtain the overlapping sampling contrastive learning loss function, the noise enhancement contrastive learning loss function, and the Smooth L1 loss function;

[0107] S3: Based on the overlapping sampling contrastive learning loss function, the noise enhancement contrastive learning loss function, and the SmoothL1 loss function, obtain the overall target loss function, and train the pre-trained model based on the overall target loss function.

[0108] S4: Calculate the root mean square error loss, adjust the pre-trained model, and obtain a natural gas demand forecasting model based on contrastive learning;

[0109] S5: Predict natural gas demand based on the natural gas demand prediction model based on contrastive learning to obtain the natural gas demand result.

[0110] Specifically, step S1 includes the following steps:

[0111] S11: Organize the collected time series data to obtain a length of... raw time series data , ,in, The data is the original time series data. The data ordinal number of the original time series data. ;

[0112] S12: Filter out raw time series data whose length is less than the length threshold;

[0113] The Z-score of the original time series data is calculated using the following formula:

[0114]

[0115] in, For Z-scores, The average value of the original time series data. The standard deviation of the original time series data;

[0116] Pre-set the Z-score threshold and Z-score proportion threshold.

[0117] Calculate the proportion of the original time series data whose Z-score is greater than the Z-score threshold. :

[0118]

[0119] in, The number of data points in a set of original time series data whose Z-scores are greater than the Z-score threshold;

[0120] like Greater than the Z-score proportion threshold, The corresponding original time series data is then removed, if Less than or equal to the Z-score proportion threshold The corresponding original time series data is retained.

[0121] The filtered raw time series data is obtained;

[0122] S13: Use the SAITS (Self-Attention-based Imputation for Time Series) model to impute missing values ​​in the filtered original time series data to obtain the complete time series data.

[0123] Specifically, step S2 includes the following steps:

[0124] S21: Sample the complete time series data according to the same sampling interval and sampling size to obtain several initial time period samples of the same length, and generate multiple overlapping time period samples using a sliding window method. The initial time period samples and the overlapping time period samples are the time period samples. conduct Encode and generate embedding vectors

[0125]

[0126] in, for Encoding function, For learnable location embedding;

[0127] The time period sample There exists For the first overlapping sample Second Sample The first sample and the second sample constitute the sampling sample. ,in, The number of overlapping logarithms;

[0128] The sample Random selection positive sample The remaining sampled samples are negative samples. ,in, and The sample ordinal number, , ;

[0129] First Sample The corresponding first embedding vector is ;

[0130] Second Sample The corresponding second embedding vector is ;

[0131] Positive sample The corresponding positive embedding vector is ;

[0132] Negative sample The corresponding negative embedding vector is ;

[0133] S22: Filter the embedded vectors for false negative samples to obtain a set of false negative samples. :

[0134] Calculate the cosine similarity between the positive and negative embedding vectors:

[0135]

[0136] in, Cosine similarity of the embedded vectors. Modulo the vector.

[0137] Calculate the first set of false negative embedding vectors Second set of pseudo-negative embedding vectors :

[0138]

[0139]

[0140]

[0141] in, The top-ranked vectors with the highest cosine similarity after calculation are the embedded vectors. A set of cosine similarities, As the ranking threshold, This is the preset ranking ratio hyperparameter. This indicates the industry corresponding to the positive or negative time period samples. Indicates equality.

[0142] False negative sample set The first set of pseudo-negative embedding vectors Second set of pseudo-negative embedding vectors The union of, .

[0143] S23: By performing comparative learning on the first embedding vector, the second embedding vector, the positive embedding vector, and the negative embedding vector after removing the false negative sample set, the overlapping sampling contrastive learning loss function is obtained. The calculation formula is:

[0144]

[0145] in, This indicates that the embedding vector does not contain false negative samples. This refers to the temperature parameter.

[0146] S24: Perform noise enhancement comparison on the positive embedding vector and the negative embedding vector to obtain the reinforced positive embedding vector; perform noise enhancement contrastive learning on the positive embedding vector, the negative embedding vector, and the reinforced positive embedding vector to obtain the noise enhancement contrastive learning loss function. The specific steps are as follows:

[0147] The enhanced positive embedding vector is obtained by mixing a small number of negative time period samples into the positive embedding vector:

[0148]

[0149] in, These are weight parameters;

[0150] The noise enhancement contrastive learning loss function is obtained by performing noise enhancement contrastive learning on the positive embedding vector, negative embedding vector, and reinforced positive embedding vector. :

[0151] .

[0152] S25: Using the positive embedding vector, the second embedding vector, and the reinforced positive embedding vector, noise filtering is learned on the samples in the time period to obtain the Smooth L1 loss function. The specific steps are as follows:

[0153] The decoder training process is as follows:

[0154]

[0155]

[0156]

[0157] in, For querying the matrix, The key matrix, For value matrices, To query the weight matrix of the matrix, The weight matrix is ​​the key matrix. The weight matrix is ​​the value matrix;

[0158] The attention score of the decoder is calculated using an attention mechanism:

[0159]

[0160] in, Let be the attention function. For activation function, Let be the dimension of the key matrix. This is the matrix transpose.

[0161] The attention score of the decoder is calculated using an attention mechanism:

[0162]

[0163] in, Let be the attention function. For activation function, Let be the dimension of the key matrix. This is the matrix transpose.

[0164] The attention score and the positive embedding vector are denoised using a multilayer perceptron:

[0165]

[0166] in, These are the denoised time period samples. It is a multilayer perceptron. This is a layer normalization operation.

[0167] Time period samples The Smooth L1 loss function is obtained by performing noise filtering training. :

[0168]

[0169] in, For the preset hyperparameters of the Smooth L1 loss, this embodiment of the invention sets... .

[0170] S3: Based on the overlapping sampling contrastive learning loss function, the noise enhancement contrastive learning loss function, and the SmoothL1 loss function, obtain the overall target loss function. The overall objective loss function is a weighted sum of the overlapping sampling contrastive learning loss function, the noise enhancement contrastive learning loss function, and the Smooth L1 loss function, as shown in the formula:

[0171]

[0172] in, As the first parameter, For the second parameter,

[0173] Training is performed using the Adam optimizer, when When the value is minimized, the pre-trained model is considered to have converged, and the resulting pre-trained model is denoted as ( ). .

[0174] S4: Calculate the root mean square error loss, adjust the overall objective loss function, and obtain the natural gas demand forecasting model based on contrastive learning;

[0175] Includes the following steps:

[0176]

[0177]

[0178] in, The prediction results obtained by calculating the samples of the time period using the model constructed using the overall objective loss function are as follows: This is the true result of the original time series data. Let the root mean square error loss function be . This represents the distance from the ordinal position of the original time series data to the starting point of the prediction. It is a pre-trained model function; by adjusting the first and second parameters, it becomes... The minimum value is considered the model convergence, thus yielding the natural gas demand forecasting model based on contrastive learning. .

[0179] S5: Predict natural gas demand based on the natural gas demand prediction model based on contrastive learning to obtain the natural gas demand result.

[0180] The natural gas demand results are as follows:

[0181]

[0182] in, This is a result of natural gas demand.

[0183] In this embodiment of the invention, the input to the model is the original time series data. The output is The overall loss function value of the comparative learning natural gas demand forecasting model.

[0184] The effectiveness of the embodiments of the present invention will be demonstrated below:

[0185] This invention was validated using the ENN dataset, a private dataset of ENN Gas Company. The experiment included a medium- to long-term forecasting task. For this task, the forecasting step size was set to 60, 90, 120, 150, and 180 days. The medium- to long-term forecasts were compared with nine corresponding time series forecasting models, and the prediction results were evaluated using MSE (mean-square error) and MAE (mean absolute error), two widely used metrics in time series forecasting. Specific results are shown in Table 1.

[0186] Table 1. Comparison of results of the present invention and other methods in medium- and long-term prediction tasks.

[0187]

[0188] Among them, PITS (Patch Independence for Time Series), SimMTM (Simple pre-training framework for Masked Time-series Modeling), PatchTST (Patch TimeSeries Transformer), iTransformer (Inverted Transformer), TimesNet, DLinear (DLinear), Nonstationary, Pyraformer, FEDformer (Frequency Enhanced DecomposedTransformer), Autoformer and Informer are all existing prediction methods.

[0189] Table 1 presents the performance of existing recommendation models and methods on various datasets after applying this invention. Experimental results show that the natural gas demand prediction model based on contrastive learning proposed in this invention reduces the error between the prediction results and the actual values, achieving better prediction performance.

[0190] This invention also provides a project-based candidate project pool fairness recommendation system, including:

[0191] Time series processing module 101: Defines raw time series data, preprocesses the raw time data sequence to obtain complete time series data;

[0192] Loss function calculation module 102: Designs and trains a contrastive learning task on the complete time series data to obtain the overlapping sampling contrastive learning loss function, the noise enhancement contrastive learning loss function, and the Smooth L1 loss function; obtains the overall target loss function based on the overlapping sampling contrastive learning loss function, the noise enhancement contrastive learning loss function, and the Smooth L1 loss function; and trains a pre-trained model based on the overall target loss function.

[0193] Model fine-tuning module 103: Calculates the root mean square error loss, adjusts the pre-trained model, and obtains a natural gas demand forecasting model based on contrastive learning;

[0194] Data calculation module 104: Predicts natural gas demand based on the natural gas demand prediction model based on contrastive learning, and obtains the natural gas demand result.

[0195] The advantages and positive effects of the invention mainly include the following aspects:

[0196] 1. Innovative pre-training scheme: This invention proposes to combine contrastive learning with noise filtering tasks, which makes the prediction task more robust to noise. Compared with previous time series prediction based on contrastive learning, combining noise filtering tasks with contrastive learning tasks can better enhance the robustness to noise that is prevalent in the dataset, obtain better representation, and thus make better predictions of users' future natural gas demand.

[0197] 2. Multi-user dataset adaptation: This invention addresses the difference between the dataset and conventional publicly available time-series datasets, which are univariate multi-user datasets. It makes special adaptations for this type of dataset and specifies corresponding data partitioning strategies, dividing the test set according to date, so that it can be better applied in the corresponding scenarios.

[0198] 3. Effective Negative Sample Removal Strategy: This invention proposes to remove negative samples in comparative learning based on two indicators: user industry information and sample similarity. This approach makes more effective use of existing information while avoiding the incorrect identification of potentially positive samples as negative or positive samples in the calculation, thus achieving suboptimal results and ultimately obtaining better prediction results.

[0199] 4. Excellent Experimental Results: Experiments were conducted on a real ENN Group private natural gas usage dataset. The results show that the method in this embodiment outperforms existing end-to-end and pre-trained benchmark models in natural gas demand forecasting tasks. This demonstrates that the proposed method can effectively improve the accuracy of natural gas demand forecasting tasks.

[0200] In summary, the advantages and positive effects of this invention include innovative pre-training schemes, multi-user dataset adaptation, effective negative sample removal strategies, and excellent experimental results, providing a new solution for natural gas demand forecasting tasks.

[0201] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

[0202] It should be noted that the embodiments of this disclosure can be implemented using hardware, software, or a combination of both. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by a suitable instruction execution system, such as a microprocessor or dedicated-design hardware. Those skilled in the art will understand that the above-described devices and methods can be implemented using computer-executable instructions and / or included in processor control code, for example, such code provided on a programmable memory or a data carrier such as an optical or electronic signal carrier.

[0203] Furthermore, although the operation of the methods of this disclosure is described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Rather, the steps depicted in the flowcharts may be performed in a different order. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps. It should also be noted that the features and functions of two or more devices according to this disclosure may be embodied in one device. Conversely, the features and functions of one device described above may be further divided and embodied by multiple devices.

[0204] While this disclosure has been described with reference to several specific embodiments, it should be understood that this disclosure is not limited to the specific embodiments disclosed. This disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A natural gas demand forecasting method based on contrastive learning, characterized in that, Includes the following steps: S1: Define the original time series data, preprocess the original time series data to obtain complete time series data; S2: Design and train a contrastive learning task on the complete time series data to obtain the overlapping sampling contrastive learning loss function, the noise enhancement contrastive learning loss function, and the Smooth L1 loss function; Includes the following steps: S21: Sample the complete time series data at the same sampling interval and sampling size to obtain several initial time period samples of the same length, and generate multiple overlapping time period samples using a sliding window method. The initial time period samples and the overlapping time period samples constitute the time period samples. The time period samples are processed Encode and generate embedding vectors : in, for Encoding function, For learnable location embedding; The time period sample There exists For the first overlapping sample Second Sample The first sample and the second sample constitute the sampling sample. ,in, The number of overlapping logarithms; From the sampled sample Random selection Each sampled sample is used as a positive example sample. The remaining sampled samples are used as negative samples. ,in, and The sample ordinal number, , ; The first sample The corresponding first embedding vector is ; The second sample The corresponding second embedding vector is ; The positive sample The corresponding positive embedding vector is ; The negative sample The corresponding negative embedding vector is ; S22: Perform false negative sample filtering on the positive and negative embedding vectors to obtain the false negative sample set. The steps include: S221: Calculate the cosine similarity between the positive and negative embedding vectors: in, Cosine similarity of the embedded vectors. Take the modulus of the vector; S222: Obtain the first set of false negative embedding vectors : in, The top-ranked cosine similarity of the computed embedding vectors A set of cosine similarities, As the ranking threshold, This is a preset ranking ratio hyperparameter; S223: Obtain the second set of pseudo-negative embedding vectors : in, This indicates the industry corresponding to the positive or negative embedding vector. Indicates the same; S224: Obtain the false negative sample set using the first false negative embedding vector set and the second false negative embedding vector set. : ； S23: By performing comparative learning on the first embedding vector, the second embedding vector, the positive embedding vector, and the negative embedding vector after removing the false negative sample set, the overlapping sampling contrastive learning loss function is obtained. The overlapping sampling contrastive learning loss function The calculation formula is: in, This indicates that the embedding vector does not contain the set of false negative samples. For temperature parameters; S24: Perform noise enhancement comparison on the positive embedding vector and the negative embedding vector to obtain the reinforced positive embedding vector; perform noise enhancement contrastive learning on the positive embedding vector, the negative embedding vector, and the reinforced positive embedding vector to obtain the noise enhancement contrastive learning loss function. The steps include: S241: The enhanced positive embedding vector is obtained by mixing the negative embedding vector into the positive embedding vector. : in, These are weight parameters; S242: Perform noise enhancement contrastive learning on the positive embedding vector, negative embedding vector, and reinforced positive embedding vector to obtain the noise enhancement contrastive learning loss function. The calculation formula is: ； S25: Using the second embedding vector, the positive embedding vector, and the reinforced positive embedding vector, noise filtering is performed on the samples in the time period to obtain the Smooth L1 loss function. ; S3: Based on the overlapping sampling contrastive learning loss function, the noise enhancement contrastive learning loss function, and the Smooth L1 loss function, obtain the overall target loss function, and train the model using the overall target loss function to obtain a pre-trained model; the overall target loss function The formula is: in, As the first parameter, For the second parameter, Training is performed using the Adam optimizer, when When the value is minimized, the pre-trained model is considered to have converged, and the resulting pre-trained model is denoted as ( ). ; S4: Calculate the root mean square error loss, adjust the pre-trained model, and obtain a natural gas demand forecasting model based on contrastive learning; S5: Predict natural gas demand based on the natural gas demand prediction model based on contrastive learning to obtain the natural gas demand result.

2. The natural gas demand forecasting method based on contrastive learning according to claim 1, characterized in that, S1 includes the following steps: S11: Organize the collected time series data to obtain a length of... raw time series data , ,in, The data is the original time series data. The data ordinal number of the original time series data ; S12: Filter out raw time series data whose length is less than the length threshold; The Z-score of the original time series data is calculated using the following formula: in, For Z-scores, The average value of the original time series data. The standard deviation of the original time series data. Calculate the proportion of the original time series data whose Z-score is greater than the Z-score threshold. : in, Data of a set of raw time series data The number of Z-scores greater than the Z-score threshold; like Greater than the Z-score proportion threshold, The corresponding original time series data is then removed, if Less than or equal to the Z-score proportion threshold The corresponding original time series data is retained. The filtered raw time series data is obtained; S13: Use the SAITS model to impute missing values ​​in the filtered original time series data to obtain the complete time series data.

3. The natural gas demand forecasting method based on contrastive learning according to claim 1, characterized in that, S25 includes the following steps: S251: Training the decoder: in, For querying the matrix, The key matrix, For value matrices, To query the weight matrix of the matrix, The weight matrix is ​​the key matrix. The weight matrix is ​​the value matrix; S252: Calculate the attention score of the decoder using an attention mechanism: in, Let be the attention function. For activation function, Let be the dimension of the key matrix. This is the transpose of the matrix; S253: Use a multilayer perceptron to denoise the attention score and the positive embedding vector: in, These are the denoised time period samples. It is a multilayer perceptron. For layer normalization operation; S254: Samples for a given time period The Smooth L1 loss function is obtained by performing noise filtering training. : in, These are the preset hyperparameters for the Smooth L1 loss.

4. The natural gas demand forecasting method based on contrastive learning according to claim 3, characterized in that, S4 includes the following steps: in, The prediction results obtained by calculating the samples of the time period using the model constructed using the overall objective loss function are as follows: This is the true result of the original time series data. Let the root mean square error loss function be . This represents the distance from the ordinal position of the original time series data to the starting point of the prediction. It is a pre-trained model function, which is achieved by adjusting the first parameter and the second parameter. The minimum value is considered the model convergence, resulting in the natural gas demand forecasting model based on contrastive learning.

5. A natural gas demand forecasting system based on contrastive learning, used to execute any one of the natural gas demand forecasting methods based on contrastive learning as described in claims 1 to 4, characterized in that, include: Time series processing module: Defines raw time series data, preprocesses the raw time series data to obtain complete time series data; Loss function calculation module: Performs comparative learning task design and training on the complete time series data to obtain the overlapping sampling comparative learning loss function, the noise enhancement comparative learning loss function, and the Smooth L1 loss function; Based on the overlapping sampling comparative learning loss function, the noise enhancement comparative learning loss function, and the Smooth L1 loss function, the overall target loss function is obtained; and a pre-trained model is obtained by training based on the overall target loss function. Model fine-tuning module: Calculates the root mean square error loss, adjusts the pre-trained model, and obtains a natural gas demand forecasting model based on contrastive learning; Data calculation module: Predicts natural gas demand based on the comparative learning-based natural gas demand prediction model, and obtains the natural gas demand results.

Images