Gas load prediction method and device based on attention mechanism, equipment and medium
By employing an attention-based gas load forecasting algorithm, which preserves temperature and weather characteristics, the nonlinearity problem in gas load forecasting is solved, thus improving forecast accuracy.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- GCI SCI & TECH
- Filing Date
- 2022-12-13
- Publication Date
- 2026-06-09
AI Technical Summary
Gas load forecasting is affected by factors such as temperature, weather conditions, and human living habits, exhibiting nonlinear characteristics. Existing technologies are difficult to predict accurately, resulting in large deviations in forecast results and failing to meet the scheduling needs of the gas supply system.
An attention-based prediction algorithm is adopted to improve prediction accuracy by masking the load sequence, preserving temperature and weather characteristics, and utilizing weather data features.
By improving the decoder mask design and making full use of temperature and weather characteristics, the accuracy of gas load forecasting can be improved.
Smart Images
Figure CN115829135B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of gas load prediction technology, and in particular to a gas load prediction method, apparatus, equipment and medium based on an attention mechanism. Background Technology
[0002] Gas load forecasting is an important basis for the operation and scheduling of the gas supply system. It is necessary to minimize the amount of gas stored and reduce gas storage costs while meeting the gas demand of users.
[0003] However, gas load is affected by a variety of factors such as temperature, weather conditions, and human work and living habits, exhibiting obvious nonlinear characteristics, which brings difficulties to gas load forecasting. If gas load is forecasted based solely on experience, the forecast results will have large deviations and will not meet the needs of system operation and scheduling.
[0004] With the development of machine learning technology, attention-based prediction algorithms have achieved good performance in time series prediction. However, when designing the decoder mask, the algorithm does not make full use of known prediction information such as temperature and weather conditions. Summary of the Invention
[0005] This invention provides a gas load prediction method, apparatus, equipment, and medium based on an attention mechanism. The method masks the load sequence to be predicted with "0", retaining weather, temperature, and other characteristic information, making full use of weather data features, and improving prediction accuracy.
[0006] To achieve the above objectives, embodiments of the present invention provide a gas load prediction method based on an attention mechanism, comprising:
[0007] Historical data on factors correlated with gas load were collected;
[0008] The historical data is preprocessed to obtain a training dataset; wherein, the data sample at time t in the training dataset is denoted as... Where X1 is the time series, and X2 to X5 are the feature sequences;
[0009] A first embedding vector is obtained by performing vector mapping processing on each data sample in the training dataset.
[0010] The first embedding vector is input into the encoder to generate a first query matrix, a first keyword matrix, a first value matrix, and the attention is calculated.
[0011] The second embedding vector is obtained by concatenating the sequence to be predicted and a portion of the first embedding vector and then performing vector mapping.
[0012] The second embedding vector is input into the decoder to generate a second query matrix. Attention is then calculated based on the second query matrix, the first keyword matrix, and the first value matrix to obtain the reconstructed second value matrix.
[0013] The second value matrix is input into the fully connected layer to obtain the gas load prediction result.
[0014] The factors related to gas load include the date, temperature, weather type, and holiday type.
[0015] Specifically, the preprocessing of the historical data includes:
[0016] Weather types are quantified; heavy rain or torrential rain is quantified as 1.0; thunderstorms or moderate rain is quantified as 0.9; light rain is quantified as 0.8; and other weather types are quantified as 0.7.
[0017] Furthermore, the step of performing vector mapping processing on each data sample in the training dataset to obtain the first embedding vector specifically includes:
[0018] The time series X1 in the data sample is time-encoded as follows:
[0019] Timestamps are extracted at the granularity of day, month, and year, and denoted as T1, T2, and T3 respectively; where T1 = (a1-1) / (b1-1)-0.5, a1 represents the week index corresponding to the day, and b1 represents the number of days in a week; T2 = (a2-1) / (b2-1)-0.5, a2 represents the month index corresponding to the day, and b2 represents the number of days in a month; T3 = (a3-1) / (b3-1)-0.5, a3 represents the year index corresponding to the day, and b3 represents the number of days in a year; T1, T2, and T3 are input into a one-dimensional convolutional layer to obtain the global timestamp S;
[0020] The feature sequences X2 to X5 in the data sample are positionally encoded, specifically as follows:
[0021]
[0022]
[0023] Where pos is the absolute position index of the feature sequence, and d model is the dimension of the input feature vector, j is the position index of the feature vector, 2j represents an even position, and 2j+1 represents an odd position;
[0024] The feature sequences X2 to X5 are input into a one-dimensional convolutional layer to obtain a high-dimensional feature vector u;
[0025] The first embedding vector is obtained by superimposing the high-dimensional feature vector u, the position code P, and the global timestamp S.
[0026] Furthermore, the step of inputting the first embedding vector into the encoder to generate the first query matrix, the first keyword matrix, and the first value matrix, and calculating the attention, specifically includes:
[0027] The first embedding vector is respectively compared with the weight matrix W. Q W K W V Multiplying them together yields the first query matrix Q. en First Key Matrix K en First value matrix V en matrix;
[0028] Select the first query matrix Q en middle The largest value Q en Matrix, denoted as
[0029]
[0030] Where, q i k i For Q en K en The i-th row, L K For K en The number of rows, where d is the dimension of the input feature vector;
[0031] According to the first query matrix Q en First Key Matrix K en First value matrix V en Matrix calculation of attention, and for V en Update;
[0032]
[0033] in, d is the dimension of the input feature vector;
[0034] For Q that was not selected en The corresponding V en Value using V en Update the mean; update the V en The input is fed into a one-dimensional convolutional layer, and after using ELU as the activation function, it is then fed into a pooling layer for downsampling calculation to obtain the hidden representation of the layer encoder:
[0035]
[0036] Where l represents the current encoder layer number; The input is fed into the next encoder layer for iteration, and the final encoder output can be obtained.
[0037] Furthermore, the step of concatenating the sequence to be predicted and a portion of the first embedding vector, followed by vector mapping to obtain the second embedding vector, specifically includes:
[0038] The sequence to be predicted is concatenated with a portion of the first embedding vector to obtain...
[0039]
[0040] The sequence to be predicted is denoted as . Among them Substitute the data from the weather forecast. Then use a mask of "0" to cover it; This is a portion of the first embedding vector;
[0041] Will The second embedding vector is obtained by performing vector mapping.
[0042] Furthermore, the step of inputting the second embedding vector into the decoder to generate a second query matrix, and performing attention calculation based on the second query matrix, the first keyword matrix, and the first value matrix to obtain the reconstructed second value matrix specifically includes:
[0043] The decoder generates a second query matrix Q based on the second embedding vector. de And according to the second query matrix Q de K generated by the encoder en V en Calculate attention and generate the second-valued matrix V de :
[0044]
[0045] Accordingly, embodiments of the present invention also provide a gas load prediction device based on an attention mechanism, comprising:
[0046] The data acquisition module is used to collect historical data on factors that are related to gas load;
[0047] The data preprocessing module is used to preprocess the historical data to obtain a training dataset; wherein, the data sample at time t in the training dataset is denoted as... Where X1 is the time series, and X2 to X5 are the feature sequences;
[0048] The first vector mapping module is used to perform vector mapping processing on each data sample in the training dataset to obtain a first embedding vector.
[0049] The encoding module is used to input the first embedding vector into the encoder to generate a first query matrix, a first keyword matrix, a first value matrix, and to calculate attention;
[0050] The second vector mapping module is used to concatenate the sequence to be predicted and a portion of the first embedding vector and then perform vector mapping processing to obtain the second embedding vector.
[0051] The decoding module is used to input the second embedding vector into the decoder to generate a second query matrix, and to perform attention calculation based on the second query matrix, the first keyword matrix and the first value matrix to obtain the reconstructed second value matrix.
[0052] The prediction module is used to input the second value matrix into the fully connected layer to obtain the gas load prediction result.
[0053] Accordingly, embodiments of the present invention also provide a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements any of the above-described attention-based gas load prediction methods.
[0054] Accordingly, embodiments of the present invention also provide a computer-readable storage medium, the computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to execute any of the above-described attention-based gas load prediction methods.
[0055] Compared with the prior art, the present invention has the following beneficial effects:
[0056] This invention employs an attention-based gas load prediction algorithm to process and predict long sequences. Furthermore, it improves the mask design of the decoder, preserving feature information such as weather and temperature, thereby fully utilizing the characteristics of the prediction information and improving the accuracy of gas load prediction. Attached Figure Description
[0057] Figure 1 This is a flowchart illustrating a gas load prediction method based on an attention mechanism provided in an embodiment of the present invention.
[0058] Figure 2 This is a schematic diagram of a gas load prediction device based on an attention mechanism provided in an embodiment of the present invention. Detailed Implementation
[0059] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0060] See Figure 1 This is a flowchart illustrating a gas load prediction method based on an attention mechanism provided in an embodiment of the present invention.
[0061] This invention provides a gas load prediction method based on an attention mechanism, comprising steps S1 to S7:
[0062] S1, collect historical data on factors that are related to gas load;
[0063] S2, preprocess the historical data to obtain a training dataset; wherein, the data sample at time t in the training dataset is denoted as... Where X1 is the time series, and X2 to X5 are the feature sequences;
[0064] S3, perform vector mapping processing on each data sample in the training dataset to obtain the first embedding vector;
[0065] S4, input the first embedding vector into the encoder to generate the first query matrix, the first keyword matrix, the first value matrix, and calculate the attention;
[0066] S5, after concatenating the sequence to be predicted and a portion of the first embedding vector, perform vector mapping processing to obtain the second embedding vector;
[0067] S6, input the second embedding vector into the decoder to generate a second query matrix, and perform attention calculation based on the second query matrix, the first keyword matrix and the first value matrix to obtain the reconstructed second value matrix;
[0068] S7. Input the second value matrix into the fully connected layer to obtain the gas load prediction result.
[0069] Specifically, based on correlation analysis, the daily gas load is related to factors such as the temperature, weather type, and holiday type of the day. The historical data mentioned above are collected as a training dataset.
[0070] To facilitate analysis, the historical data was preprocessed, and the weather types were quantified. Specifically, heavy rain or rainstorm was quantified as 1.0; thunderstorm or moderate rain was quantified as 0.9; light rain was quantified as 0.8; and other weather types were quantified as 0.7.
[0071] In a specific embodiment, after quantization, the training samples can be established as shown in the table below:
[0072] date Maximum temperature (°C) Minimum temperature (°C) Weather type <![CDATA[Load (m 3 )]]> January 1, 2018 17 10 0.7 1291214 January 2, 2018 20 12 0.7 1366900 January 3, 2018 22 13 0.7 1406688 January 4, 2018 22 15 0.7 1383200 January 5, 2018 18 18 0.7 1368399 … … … … …
[0073] Then the data sample at time t in the training dataset can be represented as: X1 is called the time series, which corresponds to the date in the table. X2 to X5 are called the feature series, which correspond to the highest temperature, lowest temperature, weather type and load in the table, respectively.
[0074] Furthermore, a first embedding vector is obtained by performing vector mapping processing on each data sample in the training dataset, specifically including:
[0075] The time series X1 in the data sample is time-encoded as follows:
[0076] Timestamps are extracted using the day, month, and year of the date as granularity, and denoted as T1, T2, and T3 respectively. Where T1 = (a1-1) / (b1-1)-0.5, a1 represents the week index corresponding to the day, and b1 represents the number of days in a week; T2 = (a2-1) / (b2-1)-0.5, a2 represents the month index corresponding to the day, and b2 represents the number of days in a month; T3 = (a3-1) / (b3-1)-0.5, a3 represents the year index corresponding to the day, and b3 represents the number of days in a year; specifically, T1 = (a1-1) / (b1-1) -0.5, a1 represents the weekly index corresponding to the day, where the weekly index indicates the day of the week, and b1 is generally 7; T2 = (a2-1) / (b2-1)-0.5, a2 represents the monthly index corresponding to the day, where the monthly index indicates the day of the month, and b2 can be 28, 29, 30 or 31, depending on the specific number of days in the month; T3 = (a3-1) / (b3-1)-0.5, a3 represents the yearly index corresponding to the day, where the yearly index indicates the day of the year, and b3 can be 365 or 366.
[0077] The T1, T2, and T3 are input into a one-dimensional convolutional layer to obtain the global timestamp S;
[0078] The feature sequences X2 to X5 in the data sample are positionally encoded, specifically as follows:
[0079]
[0080]
[0081] Where pos is the absolute position index of the feature sequence, and d model is the dimension of the input feature vector, j is the position index of the feature vector, 2j represents an even position, and 2j+1 represents an odd position;
[0082] The feature sequences X2 to X5 are input into a one-dimensional convolutional layer to obtain a high-dimensional feature vector u;
[0083] The first embedding vector is obtained by superimposing the high-dimensional feature vector u, the position code P, and the global timestamp S.
[0084] Specifically, the encoder input can be obtained as:
[0085] Furthermore, the first embedding vector is input into the encoder to generate a first query matrix, a first keyword matrix, and a first value matrix, and attention is calculated, specifically including:
[0086] The first embedding vector is respectively compared with the weight matrix W. Q W K W V Multiplying them together yields the first query matrix Q. en First Key Matrix K en First value matrix V en matrix;
[0087] Specifically, W Q W K W V These are parameters that need to be learned.
[0088] Select the first query matrix Q en middle The largest value Q en Matrix, denoted as
[0089]
[0090] Where, q i k i For Q en K en The i-th row, L K For K en The number of rows, where d is the dimension of the input feature vector;
[0091] It should be noted that the above formula is used to obtain... We only need to calculate the attention for the selected Q, thus reducing computational complexity:
[0092] According to the first query matrix Q en First Key Matrix K en First value matrix V en Matrix calculation of attention, and for V en Update;
[0093]
[0094] in, d is the dimension of the input feature vector;
[0095] For Q that was not selected en The corresponding V en Value using V en Update the mean; update the V en The input is fed into a one-dimensional convolutional layer, and after using ELU as the activation function, it is then fed into a pooling layer for downsampling calculation to obtain the hidden representation of the layer encoder:
[0096]
[0097] Where l represents the current encoder layer number; The input is fed into the next encoder layer for iteration, and the final encoder output can be obtained.
[0098] Furthermore, the sequence to be predicted and a portion of the first embedding vector are concatenated and then subjected to vector mapping to obtain the second embedding vector, specifically including:
[0099] The sequence to be predicted is concatenated with a portion of the first embedding vector to obtain...
[0100]
[0101] The sequence to be predicted is denoted as . Among them Substitute the data from the weather forecast. Then use a mask of "0" to cover it; This is a portion of the first embedding vector;
[0102] Will The second embedding vector is obtained by performing vector mapping.
[0103] It should be noted that the vector mapping process described here is the same as the operation of time encoding, position encoding and convolution processing followed by superposition to obtain the embedded vector in step S3, and will not be repeated here.
[0104] In a specific embodiment, assuming the encoder input is a 96-bit sequence and the decoder input is a 72-bit sequence, the decoder input is structured as follows: the first 48 bits are the last 48 bits of the encoder input sequence, followed by the next 24 bits, which are the sequence to be predicted.
[0105] Furthermore, the second embedding vector is input into the decoder to generate a second query matrix. Attention is then performed based on the second query matrix, the first keyword matrix, and the first value matrix to obtain the reconstructed second value matrix, specifically including:
[0106] The decoder generates a second query matrix Q based on the second embedding vector. de And according to the second query matrix Q de K generated by the encoder en V en Calculate attention and generate the second-valued matrix V de :
[0107]
[0108] The second value matrix V de The data is input to the fully connected layer to obtain the gas load prediction results.
[0109] It should be noted that during the training phase, V... de After obtaining the prediction results through the fully connected layer, the loss function is calculated by comparing the predicted values with the true values. The weight matrix is then iterated through backpropagation until the loss function converges to its minimum value.
[0110] This invention employs an attention-based gas load prediction algorithm to process and predict long sequences. Furthermore, it improves the mask design of the decoder, preserving feature information such as weather and temperature, thereby fully utilizing the features of the prediction information and improving the accuracy of gas load prediction.
[0111] Accordingly, this invention provides a gas load prediction device based on an attention mechanism, which can realize all the processes of the gas load prediction method based on an attention mechanism provided in any of the above embodiments. The functions and technical effects of each module and unit in the device are the same as those of the gas load prediction method based on an attention mechanism provided in the above embodiments, and will not be repeated here.
[0112] See Figure 2 This is a schematic diagram of an embodiment of the gas load prediction device based on the attention mechanism provided by the present invention.
[0113] An embodiment of the present invention provides a gas load prediction device based on an attention mechanism, comprising:
[0114] Data acquisition module 1 is used to collect historical data on factors that are related to gas load;
[0115] Data preprocessing module 2 is used to preprocess the historical data to obtain a training dataset; wherein, the data sample at time t in the training dataset is denoted as... Where X1 is the time series, and X2 to X5 are the feature sequences;
[0116] The first vector mapping module 3 is used to perform vector mapping processing on each data sample in the training dataset to obtain a first embedding vector.
[0117] Encoding module 4 is used to input the first embedding vector into the encoder to generate a first query matrix, a first keyword matrix, a first value matrix, and to calculate attention;
[0118] The second vector mapping module 5 is used to concatenate the sequence to be predicted and a portion of the first embedding vector and then perform vector mapping processing to obtain the second embedding vector.
[0119] Decoding module 6 is used to input the second embedding vector into the decoder to generate a second query matrix, and to perform attention calculation based on the second query matrix, the first keyword matrix and the first value matrix to obtain the reconstructed second value matrix;
[0120] Prediction module 7 is used to input the second value matrix into the fully connected layer to obtain the gas load prediction result.
[0121] Accordingly, embodiments of the present invention also provide a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement any of the above-described attention-based gas load prediction methods.
[0122] Furthermore, embodiments of the present invention also provide a computer-readable storage medium, the computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to execute any of the above-described attention-based gas load prediction methods.
[0123] Through the above description of the embodiments, those skilled in the art can clearly understand that the present invention can be implemented by means of software plus necessary hardware platforms, and of course, it can also be implemented entirely by hardware. Based on this understanding, all or part of the technical solution of the present invention that contributes to the background art can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of the present invention.
[0124] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.
Claims
1. A gas load prediction method based on an attention mechanism, characterized in that, include: Historical data on factors correlated with gas load were collected; The historical data is preprocessed to obtain a training dataset; wherein, the data sample at time t in the training dataset is denoted as... ,in It is a time series. For characteristic sequences; A first embedding vector is obtained by performing vector mapping processing on each data sample in the training dataset. The first embedding vector is input into the encoder to generate a first query matrix, a first keyword matrix, a first value matrix, and the attention is calculated. The second embedding vector is obtained by concatenating the sequence to be predicted and a portion of the first embedding vector and then performing vector mapping. The second embedding vector is input into the decoder to generate a second query matrix. Attention is then calculated based on the second query matrix, the first keyword matrix, and the first value matrix to obtain the reconstructed second value matrix. The second value matrix is input into the fully connected layer to obtain the gas load prediction result; The step of inputting the first embedding vector into the encoder to generate the first query matrix, the first keyword matrix, and the first value matrix, and calculating the attention, specifically includes: The first embedding vector is respectively compared with , , Multiplying them together yields the first query matrix. First Key Matrix First-value matrix matrix; Select the first query matrix The largest value indivual Matrix, denoted as : in, The i-th row , d is the dimension of the input feature vector; According to the first query matrix First Key Matrix First-value matrix Matrix calculation of attention, and for Update; in, d is the dimension of the input feature vector; For those not selected Corresponding Value Update the mean; then update the mean. The input is fed into a one-dimensional convolutional layer, and after using ELU as the activation function, it is then fed into a pooling layer for downsampling calculation to obtain the hidden representation of the current layer's encoder: in, Indicates the current encoder layer number; The input is fed into the next encoder layer for iteration, yielding the final encoder output. .
2. The gas load prediction method based on an attention mechanism as described in claim 1, characterized in that, The factors that are relevant to gas load include the date, temperature, weather type, and holiday type.
3. The gas load prediction method based on an attention mechanism as described in claim 2, characterized in that, The preprocessing of the historical data specifically includes: Weather types are quantified; heavy rain or torrential rain is quantified as 1.0; thunderstorms or moderate rain is quantified as 0.9; light rain is quantified as 0.8; and other weather types are quantified as 0.
7.
4. The gas load prediction method based on an attention mechanism as described in claim 3, characterized in that, The step of performing vector mapping processing on each data sample in the training dataset to obtain the first embedding vector specifically includes: Time series data from the data sample Time encoding is performed, specifically as follows: Timestamps are extracted using the day, month, and year of the date as the granularity, and denoted as T1, T2, and T3 respectively; where, , This indicates the weekly index corresponding to the current day. Indicates the number of days in a week; , This indicates the monthly index corresponding to the current day. Indicates the number of days in January; , This indicates the year index corresponding to that day. Indicates the number of days in a year; The T1, T2, and T3 are input into a one-dimensional convolutional layer to obtain the global timestamp S; Feature sequences in data samples Position encoding is performed, specifically as follows: in, This is the absolute position index of the feature sequence. is the dimension of the input feature vector, j is the position index of the feature vector, 2j represents an even position, and 2j+1 represents an odd position; The feature sequence The input is fed into a one-dimensional convolutional layer to obtain a high-dimensional feature vector u; The first embedding vector is obtained by superimposing the high-dimensional feature vector u, the position code P, and the global timestamp S.
5. The gas load prediction method based on an attention mechanism as described in claim 1, characterized in that, The step of concatenating the sequence to be predicted and a portion of the first embedding vector, followed by vector mapping to obtain the second embedding vector, specifically includes: The sequence to be predicted is concatenated with a portion of the first embedding vector to obtain... : The sequence to be predicted is denoted as . , among them Substitute the data from the weather forecast. Then use a mask of "0" to cover it; This is a portion of the first embedding vector; Will The second embedding vector is obtained by performing vector mapping.
6. The gas load prediction method based on an attention mechanism as described in claim 1, characterized in that, The step of inputting the second embedding vector into the decoder to generate a second query matrix, and performing attention calculation based on the second query matrix, the first keyword matrix, and the first value matrix to obtain the reconstructed second value matrix specifically includes: The decoder generates based on the second embedding vector. and according to With encoder generated Calculate attention and generate a second-valued matrix. : 。 7. A gas load prediction device based on an attention mechanism, characterized in that, The apparatus for implementing the gas load prediction method based on an attention mechanism as described in any one of claims 1 to 6, comprising: The data acquisition module is used to collect historical data on factors that are related to gas load; The data preprocessing module is used to preprocess the historical data to obtain a training dataset; wherein, the data sample at time t in the training dataset is denoted as... ,in It is a time series. For characteristic sequences; The first vector mapping module is used to perform vector mapping processing on each data sample in the training dataset to obtain a first embedding vector. The encoding module is used to input the first embedding vector into the encoder to generate a first query matrix, a first keyword matrix, a first value matrix, and to calculate attention; The second vector mapping module is used to concatenate the sequence to be predicted and a portion of the first embedding vector and then perform vector mapping processing to obtain the second embedding vector. The decoding module is used to input the second embedding vector into the decoder to generate a second query matrix, and to perform attention calculation based on the second query matrix, the first keyword matrix and the first value matrix to obtain the reconstructed second value matrix. The prediction module is used to input the second value matrix into the fully connected layer to obtain the gas load prediction result; The encoding module includes a first encoding unit, which is used to encode the first embedding vector with... , , Multiplying them together yields the first query matrix. First Key Matrix First-value matrix matrix; Select the first query matrix The largest value indivual Matrix, denoted as : in, The i-th row , d is the dimension of the input feature vector; According to the first query matrix First Key Matrix First-value matrix Matrix calculation of attention, and for Update; in, d is the dimension of the input feature vector; For those not selected Corresponding Value Update the mean; then update the mean. The input is fed into a one-dimensional convolutional layer, and after using ELU as the activation function, it is then fed into a pooling layer for downsampling calculation to obtain the hidden representation of the current layer's encoder: in, Indicates the current encoder layer number; The input is fed into the next encoder layer for iteration, yielding the final encoder output. .
8. A terminal device, characterized in that, The system includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements a gas load prediction method based on an attention mechanism as described in any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device containing the computer-readable storage medium to perform a gas load prediction method based on an attention mechanism as described in any one of claims 1 to 6.