A secondary water supply water consumption prediction method based on an improved Transformer model

Abstract

The application discloses a kind of based on the secondary water supply water consumption prediction method of improved Transform model, comprising: obtaining historical water consumption data and carrying out data preprocessing;The historical water consumption data after preprocessing is converted into the data group that conforms to the data group of the training structure requirement of Transform model, and is added with position coding to obtain data sample;Improved Transform model including scaling dot-attention module and multi-head attention module is established, and data sample is input to carry out training and obtain water consumption prediction model;Water consumption is predicted by water consumption prediction model.The historical data of the secondary water supply pump house of the present application is analyzed and preprocessed, and then input into the improved Transform model for training, so that the model automatically learns the law during training, and continuously iteratively optimizes during the model training process to make the water consumption prediction ability meet the accuracy and reliability requirements.

Description

Technical Field

[0001] This invention relates to the field of secondary water supply technology, and in particular to a method for predicting secondary water consumption based on an improved Transformer model. Background Technology

[0002] Secondary water supply water consumption forecasting primarily involves controlling water tank intake based on predicted water consumption to improve water quality and energy consumption in secondary water supply pump stations. Underestimating water consumption may lead to water outages in residential areas, while overestimating it may result in water quality degradation and increased energy costs. Traditional secondary water supply mechanism models rely on data such as the number of users, water demand, peak hours, and water usage trends, which present significant uncertainties. Furthermore, water consumption is influenced not only by the number of users and water usage trends but also by factors like holidays. Therefore, modeling the impact on water consumption cannot be based solely on a few factors with complete data; the combined effects of other factors must be considered. This requires a vast and complex amount of basic data. Specialized data comes from the experience of experts and practitioners; real-time data comes from sensing layer instruments, but much historical data was not previously recorded, leading to data gaps and significant deviations between predicted and actual water consumption, resulting in low data accuracy and reliability.

[0003] The patent document "A Method, Device, and Storage Medium for Predicting Secondary Water Supply Based on Big Data," published in Chinese patent literature (publication number CN115423224A, publication date 2022-12-02), includes: establishing a single-household water consumption prediction model; specifically including: acquiring single-household water consumption data to generate a dataset; dividing all residential water consumption into 5 levels according to a normal distribution and labeling all acquired residential water consumption data into levels; training the single-household water consumption prediction model using a decision tree algorithm; predicting single-household water consumption based on the single-household water consumption prediction model; calculating the predicted water consumption value of the secondary water supply point based on the single-household water consumption; and correcting the single-household water consumption prediction model if the predicted single-household water consumption value and the actual single-household water consumption value do not belong to the same water consumption level. This technology solves the problems of existing secondary water supply prediction models, such as simple model input parameters and lack of feedback optimization mechanisms. However, the technology requires too much basic data for model building and training, and also needs to consider the impact of factors such as holidays. Therefore, historical data of each influencing factor is required. However, in actual operation, only some influencing factors have complete historical data, which will cause the water consumption prediction results to deviate significantly from the actual water consumption, resulting in low data accuracy and insufficient reliability. Summary of the Invention

[0004] This invention aims to overcome the problem that existing water consumption prediction technologies suffer from significant uncertainties and incompleteness in data such as the number of users, water demand, peak periods, and water consumption trends, resulting in low accuracy and reliability of prediction results. It provides a secondary water supply water consumption prediction method based on an improved Transformer model. Historical data from secondary water supply pump stations is analyzed and preprocessed before being input into the improved Transformer model for training. This allows the model to automatically learn patterns during training and continuously iterate and optimize its water consumption prediction capabilities to meet accuracy and reliability requirements.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A method for predicting secondary water supply consumption based on an improved Transformer model includes:

[0007] Acquire historical water consumption data and perform data preprocessing;

[0008] The preprocessed historical water consumption data is transformed into a data set that meets the requirements of the Transformer model training structure, and then added to the location code to obtain the data sample.

[0009] An improved Transformer model, including a scaling dot product attention module and a multi-head attention module, was established and trained using input data samples to obtain a water consumption prediction model.

[0010] Water consumption is predicted using a water consumption prediction model.

[0011] In this invention, the Transformer model completely bypasses the sequential constraint in time series prediction problems. Based on positional encoding technology, it can also obtain positional information between sequences. Compared with the shortcomings of existing technologies that require a large amount of different types of basic data for model building and training, the method in this invention does not require processing of data on influencing factors such as weekdays and holidays. The preprocessed water consumption data can be directly input into the model, allowing the model to automatically learn the patterns during training. This improves the model's generalization ability to a certain extent. At the same time, by predicting the water consumption of secondary water supply, it can be applied to the intelligent control of the number of water inlets and the amount of water inlet in secondary water supply.

[0012] Preferably, the data preprocessing process includes:

[0013] Box plots were used to detect anomalies in historical water consumption data, and outliers were treated as missing values.

[0014] For missing values ​​in historical water consumption data, a moving median filter is used to fill them in;

[0015] Historical water consumption data were subjected to Box-Cox transformation and then standardized.

[0016] The data preprocessing in this invention includes steps such as anomaly detection, missing data imputation, and feature transformation. Since water consumption data is collected by instruments, factors such as instrument installation or maintenance may cause missing data or outliers like "spikes," so these outliers need to be processed first. Considering the correlation between time series data in this invention, moving median filtering is used for imputation, which has a certain degree of resistance to data sensitivity. Feature transformation is used to process the data into a form that is conducive to model training.

[0017] Preferably, the process of transforming preprocessed historical water consumption data into a dataset that meets the training structure requirements of the Transformer model includes:

[0018] Historical water consumption data is divided into training and testing sets, and the input window size m and output window size 1 are set.

[0019] The sequence data and target data are generated by combining the sliding window traversal method. The dimension of both data is [Tm,m,1], where T is the training set or the amount of data in the training set.

[0020] In this invention, historical water consumption data is a one-dimensional structure, while the data used for training the Transformer model is a three-dimensional structure, namely: [sample_size, seq_len, feature], where sample_size is the number of samples, seq_len is the length of the sample sequence, and feature is the feature dimension of the sequence. Therefore, the water consumption data needs to be transformed in dimension before training. The total historical water consumption data is divided into training and testing sets according to a certain ratio to train the model and test the trained model respectively.

[0021] As a preferred option, after the preprocessed historical water consumption data is transformed and added to the location code, it is necessary to perform sample filling and splicing to obtain the data sample;

[0022] The sample filling refers to filling the top row of data in the transformed data group with a value of 0.

[0023] The splicing refers to combining the filled data group X t The first half of the column is concatenated with the second half of the data group X0 before filling, and used as the final input data sample for the improved Transformer model.

[0024] In this invention, the Transformer model essentially performs three processes: adding global information of the sequence samples to each sample point; quantifying the attention each sample point has in the sequence, i.e., its weight; and repeating previously occurring combinations of sample points. The model is not good at completing the last process, which can significantly reduce the model's learning efficiency and may even lead to overfitting errors. Therefore, the processes of sample padding and splicing are added. The spliced ​​data samples introduce prior information into the model's training process, making the sample points before and after the sequence more correlated. This allows the model to obtain this information without learning, thereby accelerating training.

[0025] Preferably, the Box-Cox transformation is as follows:

[0026]

[0027] Where λ is the transformation parameter, y i This is the water consumption data for the i-th time.

[0028] The historical water consumption data used in this invention may require data transformation to suit the model training due to different distribution patterns. Since the influent turbidity data are all positive, they just meet the conditions for using Box-Cox transformation. At the same time, using Box-Cox transformation can make the water consumption data tend to a normal distribution.

[0029] Preferably, the standardization process is as follows:

[0030]

[0031] x represents the data before processing, and x' represents the data after processing. min and x max These are the minimum and maximum values ​​before standardization, respectively.

[0032] In this invention, because the water consumption data is relatively large, it is easy for the gradient value to be large during model training, which directly reaches the saturation region of the activation function, making it difficult to update the model. Therefore, the data needs to be standardized, scaled proportionally to fall into a characteristic range, and the data units are removed to obtain dimensionless pure values.

[0033] Preferably, in the improved Transformer model, the decoder of the Transformer model is replaced with a linear transformation layer; the improved Transformer model is trained using the training set of data samples to obtain a water consumption prediction model, and the water consumption prediction model is tested using the test set of data samples; during training, the minimum mean squared error (MSE) is used as the evaluation index to determine the batch size of a single training session.

[0034] This invention focuses on water consumption prediction, which falls under the category of time-series regression problems. Existing Transformer models consist of an encoder and a decoder. For the decoder, errors accumulate in regression problems, leading to poor model prediction performance. Therefore, in the improved model design, the decoding part is removed and replaced directly with a linear transformation layer, thereby improving the model's prediction accuracy.

[0035] Preferably, the improved Transformer model is trained iteratively using the RMSProp optimization algorithm:

[0036]

[0037]

[0038] and dW represents the accumulated weight gradient momentum during the t-th and t-1-th iterations, respectively; dW is the gradient of the loss function with respect to the weights W; β is the exponent of gradient accumulation; α is the learning rate; and ε is a constant.

[0039] In this invention, during the training process, considering the model gradient update, the same learning rate is used in areas where the loss function changes significantly or gradually. If the learning rate is too large, it is easy to skip the optimal solution directly in areas where the loss function changes significantly; if the learning rate is too small, learning will be slow and convergence will be difficult in areas where the loss function is relatively flat. Therefore, the RMSProp optimization algorithm is used to ensure that when the loss function changes significantly, the learning rate is automatically changed to a smaller value, and vice versa, thereby accelerating the model's learning speed.

[0040] As a preferred option, the output results of the water consumption prediction model test using the test set of the data sample also need to undergo Box-Cox inverse transformation and inverse normalization transformation to obtain the water consumption prediction value.

[0041] The predicted water consumption and the actual water consumption in the test set are fitted with an accuracy R. 2 The prediction results of the water consumption prediction model are obtained:

[0042]

[0043] y i This is actual water consumption data; For predicted water consumption data; This is the average value of actual water consumption data.

[0044] In this invention, the prediction accuracy and effectiveness of the water consumption prediction model are tested by fitting the accuracy of two sequences: the predicted water consumption value and the actual water consumption value. Since the data samples used for training the model have undergone Box-Cox transformation and standardization, the test results of the water consumption prediction model using the test set need to be inversely transformed to obtain the final water consumption prediction value.

[0045] The present invention has the following beneficial effects: After analyzing and preprocessing the historical data of the secondary water supply pumping station, the data is input into the improved Transformer model for training, so that the model can automatically learn the patterns during training and continuously iterate and optimize the model during the training process to make its water consumption prediction ability meet the accuracy and reliability requirements; the present invention does not require processing of data on influencing factors such as weekdays and holidays, the quantity requirement of basic data is not high, and the required historical data is the most common detection data of the secondary water supply pumping station, so the data is relatively complete, making the water consumption prediction results consistent with the actual water consumption and the data accuracy high. Attached Figure Description

[0046] Figure 1 This is a flowchart of the secondary water supply water consumption prediction method of the present invention;

[0047] Figure 2 This is a framework design diagram of an existing Transformer model in an embodiment of the present invention;

[0048] Figure 3 These are existing scaled dot product attention structure diagrams and multi-head attention structure diagrams in the embodiments of this invention;

[0049] Figure 4 This is a portion of the historical water consumption data in this embodiment of the invention;

[0050] Figure 5 This is a box plot based on anomaly detection in an embodiment of the present invention;

[0051] Figure 6 This is a partial result of water consumption data preprocessing in an embodiment of the present invention;

[0052] Figure 7 This is a schematic diagram of sample filling of sequence data in an embodiment of the present invention;

[0053] Figure 8 This is a structural diagram of the improved Transformer model part in an embodiment of the present invention;

[0054] Figure 9 This is a comparison chart of the predicted and actual water consumption values ​​in an embodiment of the present invention. Detailed Implementation

[0055] The present invention will now be further described with reference to the accompanying drawings and specific embodiments.

[0056] like Figure 1 As shown, a method for predicting secondary water supply consumption based on an improved Transformer model includes:

[0057] Acquire historical water consumption data and perform data preprocessing; the data preprocessing process includes:

[0058] Box plots were used to detect anomalies in historical water consumption data, and outliers were treated as missing values.

[0059] For missing values ​​in historical water consumption data, a moving median filter is used to fill them in;

[0060] Historical water consumption data were subjected to Box-Cox transformation and then standardized.

[0061] The preprocessed historical water consumption data is transformed into a data set that meets the requirements of the Transformer model training structure, and then added to the location code to obtain the data sample.

[0062] The process of transforming preprocessed historical water consumption data into a dataset that meets the training structure requirements of the Transformer model includes:

[0063] Historical water consumption data is divided into training and test sets, and the input window size m and output window size 1 are set. Sequence data and target data are generated by combining the sliding window traversal method, and their dimensions are both [Tm,m,1], where T is the training set or the amount of data in the training set.

[0064] After the preprocessed historical water consumption data is transformed and added to the location code, it is necessary to perform sample filling and splicing to obtain the data sample;

[0065] The sample filling refers to filling the top row of data in the transformed data group with a value of 0.

[0066] The splicing refers to combining the filled data group X t The first half of the column is concatenated with the second half of the data group X0 before filling, and used as the final input data sample for the improved Transformer model.

[0067] An improved Transformer model, including a scaling dot product attention module and a multi-head attention module, was established and trained using input data samples to obtain a water consumption prediction model.

[0068] The improved Transformer model replaces the decoder with a linear transformation layer; the improved Transformer model is trained on the training set of data samples to obtain the water consumption prediction model, and tested on the test set of data samples; during training, the minimum mean squared error (MSE) is used as the evaluation metric to determine the batch size for a single training session.

[0069] The improved Transformer model uses the RMSProp optimization algorithm for iterative training:

[0070]

[0071]

[0072] and dW represents the accumulated weight gradient momentum during the t-th and t-1-th iterations, respectively; dW is the gradient of the loss function with respect to the weights W; β is the exponent of gradient accumulation; α is the learning rate; and ε is a very small constant to prevent the denominator from being 0.

[0073] Water consumption is predicted using a water consumption prediction model. Real-time collected water consumption data is processed into data samples using the same method as historical water consumption prediction data, and then input into the water consumption prediction model to obtain the final prediction result.

[0074] The Box-Cox transformation is:

[0075]

[0076] Where λ is the transformation parameter, y i This is the water consumption data for the i-th time.

[0077] The standardization process is as follows:

[0078]

[0079] x represents the data before processing, and x' represents the data after processing. min and x max These are the minimum and maximum values ​​before standardization, respectively.

[0080] The output results of the water consumption prediction model test using the test set of data samples still need to undergo Box-Cox inverse transformation and inverse normalization transformation to obtain the water consumption prediction value.

[0081] The predicted water consumption and the actual water consumption in the test set are fitted with an accuracy R. 2 The prediction results of the water consumption prediction model are obtained:

[0082]

[0083] y i This is actual water consumption data; For predicted water consumption data; This is the average value of actual water consumption data.

[0084] In this invention, the Transformer model completely bypasses the sequential constraint in time series prediction problems. Based on positional encoding technology, it can also obtain positional information between sequences. Compared with the shortcomings of existing technologies that require a large amount of different types of basic data for model building and training, the method in this invention does not require processing of data on influencing factors such as weekdays and holidays. The preprocessed water consumption data can be directly input into the model, allowing the model to automatically learn the patterns during training. This improves the model's generalization ability to a certain extent. At the same time, by predicting the water consumption of secondary water supply, it can be applied to the intelligent control of the number of water inlets and the amount of water inlet in secondary water supply.

[0085] The data preprocessing in this invention includes steps such as anomaly detection, missing data imputation, and feature transformation. Since water consumption data is collected by instruments, factors such as instrument installation or maintenance may cause missing data or outliers like "spikes," so these outliers need to be processed first. Considering the correlation between time series data in this invention, moving median filtering is used for imputation, which has a certain degree of resistance to data sensitivity. Feature transformation is used to process the data into a form that is conducive to model training.

[0086] In this invention, historical water consumption data is a one-dimensional structure, while the data used for training the Transformer model is a three-dimensional structure, namely: [sample_size, seq_len, feature], where sample_size is the number of samples, seq_len is the length of the sample sequence, and feature is the feature dimension of the sequence. Therefore, the water consumption data needs to be transformed in dimension before training. The total historical water consumption data is divided into training and testing sets according to a certain ratio to train the model and test the trained model respectively.

[0087] In this invention, the Transformer model essentially performs three processes: adding global information from the sequence samples to each sample point; quantifying the attention each sample point has in the sequence, i.e., its weight; and repeating previously encountered combinations of sample points. For example, for a water consumption sequence, if there is a recent period where water consumption continuously increases and then gradually stabilizes, the probability of a subsequent period of continuous increase followed by stabilization is obviously higher in the Bayesian sense, which is a common time series phenomenon. However, the model is not good at completing the last process, which can significantly reduce its learning efficiency and may even lead to overfitting. Therefore, a sample padding and concatenation process is added. The concatenated data samples introduce prior information into the model's training process, making the sample points before and after the sequence more correlated. This allows the model to obtain this information without learning, thereby accelerating training.

[0088] The historical water consumption data used in this invention may require data transformation to suit the model training due to different distribution patterns. Since the influent turbidity data are all positive, they just meet the conditions for using Box-Cox transformation. At the same time, using Box-Cox transformation can make the water consumption data tend to a normal distribution.

[0089] In this invention, because the water consumption data is relatively large, it is easy for the gradient value to be large during model training, which directly reaches the saturation region of the activation function, making it difficult to update the model. Therefore, the data needs to be standardized, scaled proportionally to fall into a characteristic range, and the data units are removed to obtain dimensionless pure values.

[0090] This invention focuses on water consumption prediction, which falls under the category of time-series regression problems. Existing Transformer models consist of an encoder and a decoder. For the decoder, errors accumulate in regression problems, leading to poor model prediction performance. Therefore, in the improved model design, the decoding part is removed and replaced directly with a linear transformation layer, thereby improving the model's prediction accuracy.

[0091] In this invention, during the training process, considering the model gradient update, the same learning rate is used in areas where the loss function changes significantly or gradually. If the learning rate is too large, it is easy to skip the optimal solution directly in areas where the loss function changes significantly; if the learning rate is too small, learning will be slow and convergence will be difficult in areas where the loss function is relatively flat. Therefore, the RMSProp optimization algorithm is used to ensure that when the loss function changes significantly, the learning rate is automatically changed to a smaller value, and vice versa, thereby accelerating the model's learning speed.

[0092] In this invention, the prediction accuracy and effectiveness of the water consumption prediction model are tested by fitting the accuracy of two sequences: the predicted water consumption value and the actual water consumption value. Since the data samples used for training the model have undergone Box-Cox transformation and standardization, the test results of the water consumption prediction model using the test set need to be inversely transformed to obtain the final water consumption prediction value.

[0093] The embodiments of the present invention first describe the existing Transformer model, such as... Figure 2 The model consists of two main parts: an encoder and a decoder. Internally, it is designed based on a full attention mechanism strategy. This design allows the model to achieve parallel computing without waiting for the processing results of previous related time series during the operation, thus completely eliminating the constraint of sequential dependency in the processing of time series.

[0094] The most important part of the Transformer model is the multi-head attention mechanism, which is implemented using matrix operations. The process is as follows:

[0095] For each word vector X input to the encoder i Create three vectors: Query vector Q i Key vector K i Value vector V i These three vectors are formed by word vectors and W respectively. Q W K W V The product of the three matrices, for a sample of sequence length len, yields:

[0096] Q = XW Q

[0097] K = XW K

[0098] V = XW V

[0099] in, d k For each word's embedding dimension, W Q W K W V It is the parameter matrix to be learned.

[0100] Once the Q, K, and V matrices are obtained, the attention score can be calculated.

[0101]

[0102] QK TThe calculation results in the attention score for each word throughout the entire sentence (including itself). Its purpose is to make the gradient values ​​obtained during backpropagation more stable, preventing them from becoming too large and entering the saturation region of the softmax activation function. Softmax is a multivariate normalization function, and its expression is:

[0103]

[0104] After obtaining the score for each position, it is multiplied by each V. For positions with higher scores, the multiplied value is larger, allowing more attention to be focused on these positions; for positions with lower scores, the multiplied value is smaller, indicating that the information relevance of these positions is low, and the information at these positions can be ignored. This attention mechanism under this definition is called scaled dot product attention. Figure 3 As shown on the left.

[0105] The first two processes constitute a complete attention process, yielding feature information about the sequence data from a specific perspective. To grasp more information about the entire sequence, the first two processes need to be repeated h times. Finally, the h attention results are concatenated, as follows: Figure 3 As shown on the right:

[0106] MultiHead(Q,K,V)=Concat(head1,head2,...,head h )

[0107] Then perform a linear transformation:

[0108] Z = MultiHead(Q, K, V)W o

[0109] head j =Attention(Q) j K j V j )

[0110] W o The weight matrix is ​​denoted as Z; the final matrix Z contains information from all attention heads, and this matrix will be fed into the feedforward neural network FFNN layer.

[0111] Furthermore, the Transformer model includes a positional encoding (PE) mechanism to capture time-series information between input data. The principle is to add sine and cosine functions of different frequencies as "positional codes" to the normalized input sequence, allowing the multi-head attention mechanism in the Transformer model to fully capture the more dimensional positional information between time series. The formula used in the positional encoding technique is as follows:

[0112]

[0113]

[0114] Where pos is the sequence position, d model For the sequence embedding dimension, and d k Same, i is d model The dimension value after dividing by 2. For example, for a length pos = 3, the embedding dimension d model The positional encoding of a sequence of 128 is as follows:

[0115]

[0116] In an embodiment of the present invention, the entire water consumption prediction process is as follows:

[0117] like Figure 4 The image shows a portion of the historical water consumption data. The source dataset is a 5-minute time series from the past two years, where TARGET_VALUE represents the cumulative flow data. Therefore, differential calculation is required to obtain the water consumption within the corresponding time period.

[0118] Utilize Figure 6 The box plot shown detects anomalies in historical water consumption. Data points outside the range of Q1,3±1.70IQR are considered outliers and treated as missing values. Q1,3 are the upper and lower quartile values, and IQR is the upper and lower quartile distance.

[0119] Even after the cumulative traffic data is processed by the box plot, it still contains missing values. In this case, the moving median filter is used to fill in the missing values. Based on the determined sliding window size, the first m and last m data points of the current missing value are taken, resulting in a total of 2m+1 data points. The median of these data points is then used as the filler for the current missing value. If there are other missing values ​​in the sliding window, the missing value positions are excluded, and the median of the remaining data is used as the filler for the current missing value.

[0120]

[0121] As shown in the table above, by taking the three data points before and after the current missing value to form a moving window, we can obtain the data after imputation.

[0122] The cumulative flow is differentially calculated to obtain water consumption data for every 5 minutes. Then, water consumption data for the 30-minute timeframe is obtained through grouping calculations. Subsequent model training and prediction in this embodiment are performed at this timeframe. The water consumption data is first subjected to a Box-Cox transformation and then standardized to obtain the following result: Figure 6 The data preprocessing results are shown.

[0123] The processed historical water consumption data underwent dimensionality transformation, and was divided into training and test sets at a ratio of 8:2, resulting in a training set of 5400 data points and a test set of 1350 data points. For the training set, a sliding window method with an input window size of 12 and an output window size of 1 was used to generate sequence data and target data, both with dimensions [5388, 12, 1]. After the sequence and target data were generated, position encoding was performed. Based on the previously generated sequence dimensions [5388, 12, 1], the sequence length was set to 12 and the embedding dimension to 1. The calculated position encoding array was then directly added to the sequence and target data as input before sample filling. Sample filling and concatenation were then performed, and the concatenated array was copied three times and assigned values ​​to Q, K, and V respectively.

[0124] For sample filling, such as Figure 7 As shown, taking a sequence sample with an array shape and size of [100, 50, 1] as an example, the values ​​filled during the sample filling process are 0. Figure 7 The left side shows the water consumption data before sample filling, denoted as X0. Figure 7 The right side shows the water consumption data after sample filling, denoted as X. t When filling, insert a row of data at the top of the array. After filling, set X... t The first half of X0 is concatenated with the second half of X0 to form the final input sequence data X of the model. The concatenation formula is as follows:

[0125] X = Concat(X) t [:,:,:d k / / 2],X0[:,:,d k / / 2:])

[0126] In the formula, d k X represents the embedding dimension. t [:,:,:d k / / 2] represents the first half of the column after filling, X0[:,:,d k / / 2:] represents the second half of the column before padding. The concatenated sequence data introduces prior information into the model during training, making the sample points before and after the sequence highly correlated. This allows the model to obtain this information without learning, thereby accelerating training.

[0127] like Figure 8 The diagram shows the structure of the improved Transformer model. Since water consumption prediction is a time-series regression problem, the decoder is replaced with a linear transformation layer to build the improved Transformer model for training. Before training, the batch size parameter (i.e., the number of samples in a single training iteration) needs to be provided. In this embodiment, 32, 64, and 128 were considered. Finally, based on the minimum mean squared error (MSE) value, a batch size of 64 was chosen. The MSE expression is:

[0128]

[0129] N is the number of samples, y i and Let be the actual value and the predicted value of the i-th sample, respectively. After calculating the loss function during the training of the improved Transformer model, the RMSProp optimization algorithm is used for iterative training to obtain the water consumption prediction model.

[0130] To test the obtained water consumption prediction model, the test set is partially input into the model, resulting in a dimension of [1338, 12, 1]. In this embodiment, the output window size is 1, so a slice operation is needed on the second dimension (value 12), taking the last value, resulting in an array of size 1338. Then, inverse Box-Cox transformation and inverse normalization transformation are performed to obtain the final water consumption prediction value. If the prediction results obtained from testing the water consumption prediction model do not meet the requirements, the model can be retrained. The predicted and actual water consumption values ​​are then fitted with accuracy to obtain R0. 2 The accuracy rate was 94.28%, indicating that the model's predictive performance was good and its generalization ability was strong. To facilitate visualization of the predicted water consumption, the first 1000 predicted data points and actual data were extracted, as shown below. Figure 9 The comparison diagram shown.

[0131] The above embodiments are further elaborations and descriptions of the present invention to facilitate understanding, and are not intended to limit the present invention in any way. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for predicting secondary water supply consumption based on an improved Transformer model, characterized in that, include: Historical water consumption data was acquired and preprocessed; the historical water consumption data was then subjected to Box-Cox transformation and standardized. The preprocessed historical water consumption data is transformed into a data set that meets the training structure requirements of the Transformer model. Its structure is [number of samples, length of sample sequence, dimensionality features of sequence]. After adding the position encoding, sample padding and splicing are performed to obtain data samples. Sample filling refers to filling the top row of data in the transformed data group with a value of 0. Concatenation refers to combining the filled data groups The first half of the column and the data group before filling The second half of the column is concatenated to form the final input data sample for the improved Transformer model; An improved Transformer model, including a scaling dot product attention module and a multi-head attention module, is established and trained using input data samples to obtain a water consumption prediction model; in the improved Transformer model, the decoder of the Transformer model is replaced with a linear transformation layer. Water consumption is predicted using a water consumption prediction model.

2. The method for predicting secondary water supply consumption based on an improved Transformer model according to claim 1, characterized in that, The data preprocessing process includes: Box plots were used to detect anomalies in historical water consumption data, and outliers were treated as missing values. For missing values ​​in historical water consumption data, a moving median filter is used to fill them in.

3. The method for predicting secondary water supply consumption based on an improved Transformer model according to claim 1, characterized in that, The process of transforming preprocessed historical water consumption data into a dataset that meets the training structure requirements of the Transformer model includes: Historical water consumption data is divided into training and testing sets, and the input window size m and output window size 1 are set. The sequence data and target data are generated by combining the sliding window traversal method, and their dimensions are both [missing information]. T represents the amount of data in the training or test set.

4. The method for predicting secondary water supply consumption based on an improved Transformer model according to claim 2, characterized in that, The Box-Cox transformation is as follows: ， in These are transformation parameters. This represents the water consumption data for the i-th time.

5. A method for predicting secondary water supply consumption based on an improved Transformer model according to claim 2 or 4, characterized in that, The standardization process is as follows: ， x represents the data before processing. For the processed data, and These are the minimum and maximum values ​​before standardization, respectively.

6. A method for predicting secondary water supply consumption based on an improved Transformer model according to claim 1, 2, or 3, characterized in that, The improved Transformer model is trained using the training set of data samples to obtain a water consumption prediction model, and tested using the test set of data samples. During training, the minimum mean squared error (MSE) is used as the evaluation metric to determine the batch size for a single training session.

7. The method for predicting secondary water supply consumption based on an improved Transformer model according to claim 1, characterized in that, The improved Transformer model is iteratively trained using the RMSProp optimization algorithm: ， ， and These are the accumulated weight gradient momentum during the t-th and t-1-th iterations, respectively. Let W be the gradient of the loss function with respect to the weights W. The exponent for gradient accumulation; For learning rate, It is a constant.

8. The method for predicting secondary water supply consumption based on an improved Transformer model according to claim 6, characterized in that, The output results of the water consumption prediction model test using the test set of data samples still need to undergo Box-Cox inverse transformation and inverse normalization transformation to obtain the water consumption prediction value. Accuracy fitting will be performed between the predicted water consumption and the actual water consumption in the test set. The prediction results of the water consumption prediction model are obtained: ， This is actual water consumption data; For predicted water consumption data; This represents the average value of actual water consumption data.

Images