A water affair work order quantity prediction method based on a diffusion model

Abstract

The application provides a water affair work order quantity prediction method based on a diffusion model, and belongs to the field of time series prediction, and comprises the following steps: obtaining aggregated samples from a work order detail dataset; converting a received content field and a work order category field in L aggregated samples into work order detail word embeddings; inputting a work order quantity prediction sequence true value into a diffuser to obtain K noisy work order quantity prediction sequences; taking the noisy work order quantity prediction sequences as current frame work order quantity prediction sequences to obtain noisy hidden layer prediction feature vectors; extracting a historical sequence feature hidden vector by using an encoder of a conditional network, fusing the noisy hidden layer prediction feature vectors after splicing the work order detail word embeddings, and generating a next frame work order quantity prediction sequence; repeating the step to obtain a final restored work order quantity prediction sequence; calculating a mean square error loss function, updating network parameters, and obtaining a next L aggregated sample and a training sample. The application can avoid overfitting and improve the length and accuracy of prediction.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of time series prediction, and particularly relates to a water affair work order quantity prediction method based on a diffusion model. BACKGROUND

[0002] With the popularization of big data application technology, water affair enterprises use various types of historical production and management data to train machine learning or deep learning models, and accordingly make data prediction, and use the data prediction results to guide the work deployment, personnel scheduling and resource allocation of the enterprise, and good economic benefits are obtained.

[0003] However, the existing deep learning method still faces some challenges in application: 1) most enterprises have a short time of digitalization, and the small amount of accumulated historical data makes the model prone to overfitting; 2) in non-stationary long sequence prediction, the length and accuracy of prediction cannot be considered. SUMMARY

[0004] The main purpose of the application is to provide a water affair work order quantity prediction method based on a diffusion model, which can avoid the overfitting phenomenon of the deep learning model under the condition of small sample, and improve the generalization ability of the model; and still provide good prediction length and accuracy in the non-stationary time series processing and emergency response scene.

[0005] The application is implemented by the following technical scheme:

[0006] A water affair work order quantity prediction method based on a diffusion model comprises the following steps:

[0007] Step S1, obtain a work order quantity data set and a work order detail data set, each record in the work order quantity data set has a time period field, and the work order detail data set is grouped and aggregated by hour according to the time period field of each record in the work order quantity data set to obtain a plurality of groups of aggregated samples;

[0008] Step S2, sequentially select L groups of aggregated samples from the work order detail data set, and input the accepted content field and the work order category field in each group of aggregated samples into the Chinese BERT module of the conditional network to convert into work order detail word embedding;

[0009] Step S3, obtain a training sample according to the work order quantity data set, the training sample comprises a work order quantity history sequence, a work order quantity prediction sequence true value, and a temperature, holiday information and global time information sequence, input the work order quantity prediction sequence true value into a diffuser, add noise in K steps until the work order quantity prediction sequence true value becomes Gaussian noise, and obtain a noisy work order quantity prediction sequence after adding noise in the kth step , k = 1, 2, …, K, is the mark of the current training sample;

[0010] Step S4, the noisy work order quantity prediction sequence obtained after adding noise to the k=K step The encoder of the diffusion denoising network inputs the current frame work order quantity prediction sequence to obtain a noisy hidden layer prediction feature vector. The encoder of the conditional network inputs the work order quantity history sequence, the temperature, the holiday information, and the global time information sequence of the training sample to extract a history sequence feature hidden vector. The history sequence feature hidden vector is spliced with the work order detail word embedding. The spliced result is input into the decoder of the diffusion denoising network after being fused with the noisy hidden layer prediction feature vector. The next frame work order quantity prediction sequence is generated by weighting the current frame work order quantity prediction sequence, the decoder output, and a random noise value. The next frame work order quantity prediction sequence is taken as the current frame work order quantity prediction sequence, and the step is repeated to obtain a final restored work order quantity prediction sequence.

[0011] Step S5, the mean square error loss function between the final restored work order quantity prediction sequence and the work order quantity prediction sequence true value is minimized, and the parameters of the diffusion denoising network and the conditional network are updated. Step S2 is entered to select the next L group of aggregated samples. The next training sample is obtained in step S3. This process is repeated until a set number of training samples are used, and a trained prediction model is obtained.

[0012] Further, in step S1, the work order quantity data set is divided into a work order quantity training set, a work order quantity validation set, and a work order quantity test set. In step S3, L+H records are obtained from the work order quantity training set by a sliding window method, and then the training sample is obtained. The first L records in the training sample are taken as the work order quantity history sequence, and the last H records are taken as the work order quantity prediction sequence true value. The historical L temperature, holiday information, and global time information sequences are obtained according to the first L records.

[0013] Further, in step S3, the noise addition process is represented as wherein, is the cumulative noise attenuation coefficient, is the work order quantity prediction sequence true value of the current sample, is the Gaussian noise.

[0014] Further, in step S4, the encoder of the conditional network extracts the history sequence feature hidden vector according to the formula The history sequence feature hidden vector is spliced with the work order detail word embedding according to the formula wherein, represents self-attention, represents the work order quantity history sequence of the current training sample, represents the temperature, holiday information, and global time information sequence of the current training sample, represents a linear fully connected layer,​​ This indicates that backpropagation has stopped. Indicates splicing, This represents the conditional latent vector of the current training sample.

[0015] Furthermore, in step S4, the encoder of the diffusion denoising network is configured according to the formula... Obtain the noisy hidden layer predicted feature vector ,in, , , This represents the normalization layer of the encoder in a diffusion denoising network. For the encoder feedforward layer of the diffusion denoising network, This represents the output of the l-th sublayer of the encoder in the diffusion denoising network. This represents the output of the second sub-layer of the encoder in the diffusion denoising network.

[0016] Furthermore, in step S4, the next frame work order quantity prediction sequence According to the formula Calculate, where, This is the cumulative noise attenuation coefficient. The noise intensity is the value added at step k. , Gaussian noise is added when adding noise in step k. standard deviation This is the predicted sequence of work order volume for the current frame. The next frame prediction sequence is the decoder output of the diffusion denoising network.

[0017] Furthermore, in step S3, the global time information includes hourly, daily, weekly, and monthly codes, and the specific values ​​corresponding to the hourly, daily, weekly, and monthly codes are obtained from the L+H records truncated by the sliding window in the work order training set.

[0018] Furthermore, in step S5, the mean squared error loss function is expressed as: , This is the final predicted sequence of work order volume corresponding to the current sample. This represents the expected mean square loss of each noisy sequence prediction obtained from the current frame work order quantity prediction sequence. This indicates the data distribution that the currently predicted time series samples follow.

[0019] As can be seen from the above description of the present invention, compared with the prior art, the present invention has the following beneficial effects:

[0020] This invention first acquires a work order volume dataset and a work order details dataset. Each record in the work order volume dataset has a time period field. The work order details dataset is then grouped and aggregated by hour according to the time period field of each record in the work order volume dataset, resulting in multiple aggregated samples. Next, L aggregated samples are selected sequentially from the work order details dataset. The acceptance content field and work order category field from each aggregated sample are input into the Chinese BERT module of the conditional network to convert them into work order details word embeddings. Then, training samples are obtained based on the work order volume dataset. The training samples include historical work order volume sequences, actual values ​​of predicted work order volume sequences, and sequences of temperature, holiday information, and global time information. The actual values ​​of the predicted work order volume sequences are input into a diffuser, and noise is added in K steps until the actual value of the predicted work order volume sequences is Gaussian noise. After adding noise in the k-th step, a noisy predicted work order volume sequence is obtained. The noisy work order quantity prediction sequence obtained after adding noise at step k=K is... The current frame's work order quantity prediction sequence is input into the encoder of the denoising network to obtain a noisy hidden layer prediction feature vector. The historical work order quantity sequence, temperature, holiday information, and global time information sequence of the training samples are input into the encoder of the conditional network to extract historical sequence features. These historical sequence features are then concatenated with the work order detail word embeddings. The concatenated result, fused with the noisy hidden layer prediction feature vector, is input into the decoder of the diffusion denoising network. The next frame's work order quantity prediction sequence is generated by weighting the current frame's work order quantity prediction sequence, the decoder output, and random noise values. This next frame's work order quantity prediction sequence is used as the current frame's work order quantity prediction sequence, and this step is repeated to obtain the final restored work order quantity prediction sequence. Finally, the final restored work order quantity prediction sequence is constructed. The mean squared error loss function between the predicted work order quantity sequence and the actual value of the predicted work order quantity sequence is used to update the parameters of the denoising network and the conditional network to minimize the mean squared error loss function, thereby achieving the prediction of waterworks work order quantity. In the process, noise is gradually added to the actual value of the predicted work order quantity sequence to cover the entire space of historical sequence data distribution, and then the predicted sequence data distribution is fitted by reverse denoising. This enables a more accurate fit to global trends and non-stationary local drastic fluctuations under conditions with few samples, effectively reducing the risk of overfitting. Furthermore, the use of conditional constraints enhances the initiative and accuracy of the diffusion model prediction, enabling simultaneous improvement in prediction length and prediction accuracy, especially in the response to emergencies. Attached Figure Description

[0021] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0022] Figure 1 This is a flowchart of the present invention.

[0023] Figure 2 This is a fragment example of the work order quantity dataset of the present invention.

[0024] Figure 3 This is a fragment example of the work order details dataset of the present invention.

[0025] Figure 4 This is an example of a training sample for the present invention.

[0026] Figure 5 This is a schematic diagram of the overall structure of the present invention.

[0027] Figure 6 Figure 5 Enlarged view of section A.

[0028] Figure 7 for Figure 5 Enlarged view of section B.

[0029] Figure 8 for Figure 5 Enlarged view of section C.

[0030] Figure 9 for Figure 5 Enlarged view of section D.

[0031] Figure 10 This is a comparison chart of the prediction performance of the present invention and the Diffusion-TS method.

[0032] Figure 11 This invention compares its mean squared error (MSE) and mean absolute error (MAE) with five existing methods on four public datasets (Weather, Traffic, Electricity, and ETTh1) and the private datasets (Wqdata and Wqdata+Wddata).

[0033] Figure 12 This paper compares the invention with five existing methods on four public datasets (Weather, Traffic, Electricity, and ETTh1) and the private datasets (Wqdata and Wqdata+Wddata) in terms of context-Fréchet inception distance (Context-FID) and predictive score. Detailed Implementation

[0034] The present invention will be further described below through specific embodiments.

[0035] like Figure 1 As shown, the waterworks work order volume prediction method based on the diffusion model includes the following steps:

[0036] Step S1, obtain a work order quantity dataset and a work order detail dataset, each record in the work order quantity dataset has a time period field, and the work order detail dataset is grouped and aggregated by hour according to the time period field of each record in the work order quantity dataset, to obtain a plurality of groups of aggregated samples;

[0037] As shown in Figure 2 is a fragment instance of the work order quantity dataset Wqdata. The work order quantity dataset Wqdata is divided into a work order quantity training set, a work order quantity validation set and a work order quantity test set according to a ratio of 7:2:1. After the division is completed, the sequence length L input to the condition network CN encoder and the sequence length H input to the diffuser of the diffusion denoising network DiDeN are set according to the requirements, and generally, L = H; a group of training data is cut off in the work order quantity training set using a window of L+H width, and then the window is slid by one record to obtain the next group of training data; for the work order quantity data of the work order quantity training set, the work order quantity validation set and the work order quantity test set, each batch of 32 groups, each group of L+H records is batched.

[0038] A fragment instance of the aggregated work order detail dataset Wddata is shown in Figure 3 After the aggregation processing, the requirements of the model of the application for the data format can be met.

[0039] Step S2, sequentially select L groups of aggregated samples from the work order detail dataset, and input the accepted content field and the work order category field in each group of aggregated samples to the Chinese BERT module of the condition network to convert them into work order detail word embeddings to obtain work order event feature information, in particular, emergency information, wherein, is the hidden layer dimension of the BERT module;

[0040] Step S3, obtain a training sample according to the work order quantity dataset, the training sample contains a work order quantity historical sequence, a work order quantity prediction sequence true value, and a temperature, holiday information and global time information sequence, input the work order quantity prediction sequence true value to the diffuser, add noise in K steps until the work order quantity prediction sequence true value becomes Gaussian noise, and obtain a noisy work order quantity prediction sequence , k = 1, 2, …, K, is the current training sample flag;

[0041] L+H records are obtained from the work order quantity training set by sliding window method, and then the training sample is obtained, the first L records in the training sample are used as the work order quantity historical sequence, and the last H records are used as the work order quantity prediction sequence true value, and the historical L temperature, holiday information and global time information sequences are obtained according to the first L records. As shown in Figure 4The global time information includes hour encoding, day encoding, week encoding and month encoding, and the specific values corresponding to the hour encoding, day encoding, week encoding and month encoding are obtained according to the L+H records of the work order volume training set. In Figure 4 In the above, the sequence filled with 0~23 represents the hour encoding, the sequence filled with 0~6 represents the day encoding, the sequence filled with 0~3 represents the week encoding, and the sequence filled with 0~11 represents the month encoding. V1, V7 and V8 represent holiday encoding.

[0042] The process of adding noise in the kth step is represented as The noise adding process conforms to Markov property, wherein, is the cumulative noise attenuation coefficient, used to control the mixing ratio of the original sequence and the noise, is the real value of the work order volume prediction sequence of the current sample, is the Gaussian noise.

[0043] In step S4, the noisy work order volume prediction sequence obtained after adding noise in the k=Kth step is The encoding of the diffusion denoising network is input into the encoder of the current frame work order volume prediction sequence to obtain the noisy hidden layer prediction feature vector. The work order volume history sequence, temperature, holiday information and global time information sequence of the training sample are input into the encoder of the conditional network to extract the historical sequence feature hidden vector. The historical sequence feature hidden vector and the work order detail word embedding are spliced, and the splicing result is input into the decoder of the diffusion denoising network after being fused with the noisy hidden layer prediction feature vector. The next frame work order volume prediction sequence is generated by weighting the current frame work order volume prediction sequence, the decoder output and the noise random value. The next frame work order volume prediction sequence is taken as the current frame work order volume prediction sequence, and the step is repeated to obtain the final restored work order volume prediction sequence.

[0044] As Figures 5 to 9 is the overall structure schematic diagram and the enlarged view of each part. The encoder of the conditional network extracts the historical sequence feature hidden vector according to the formula The historical sequence feature hidden vector is spliced with the work order detail word embedding according to the formula , wherein, represents self-attention, represents the work order volume history sequence of the current training sample, represents the temperature, holiday information and global time information sequence of the current training sample, represents a linear fully connected layer, represents stop backpropagation, represents splicing, represents the conditional hidden vector of the current training sample.

[0045] ​​The encoder of the diffusion denoising network DiDeN uses self-attention to sample the noisy sequence to extract distribution features, specifically according to the formula to obtain a noisy hidden layer prediction feature vector wherein , , denotes a normalization layer of the encoder of the diffusion denoising network, denotes a feedforward layer of the encoder of the diffusion denoising network, denotes the output of the first sublayer of the lth layer of the encoder of the diffusion denoising network, denotes the output of the second sublayer of the lth layer of the encoder of the diffusion denoising network, denotes the first step noisy work order quantity prediction sequence obtained after adding noise.

[0046] The decoder of the diffusion denoising network generates a next frame prediction sequence according to the formula to calculate the next frame work order quantity prediction sequence wherein is a cumulative noise attenuation coefficient, i = 1, 2, …, k, is the noise intensity when adding noise at the kth step, , is the standard deviation of the Gaussian noise added when adding noise at the kth step, , is the current frame work order quantity prediction sequence, more specifically, denotes the decoder output corresponding to the input , denotes the historical event sequence conditional information of the current frame, which is a known quantity.

[0047] Step S5, minimize the mean square error loss function between the final restored work order quantity prediction sequence and the true value of the work order quantity prediction sequence, update the parameters of the diffusion denoising network and the conditional network; enter Step S2 to select the next L group of aggregated samples, obtain the next training sample in Step S3, until a set number of training samples are used, and obtain a trained prediction model;

[0048] Specifically, the mean square error loss function is represented as After calculating the mean square error loss function, the Adam optimizer is used for back propagation to minimize the mean square error loss function and update the network parameters, is the final restored work order quantity prediction sequence corresponding to the current sample, denotes the prediction mean square loss expectation of each noisy sequence obtained from the current frame work order quantity prediction sequence, represents a data distribution to which a currently predicted time series sample is subject.

[0049] In step S6, the prediction model is verified by using the ticket volume verification set, the mean square error loss based on the verification set is obtained, and steps S2 to S5 are repeated until the mean square error based on the verification set no longer decreases, and the training process ends.

[0050] From Figure 10 It can be seen that the present application (Wo-Diff) can better predict the change trend of ticket volume in the whole process of the event at the initial stage of the event; the comparative scheme (Diffusion-TS) relies on historical statistical rules, and due to the sparsity of the event, the prediction value can only passively follow the original trend, and cannot accurately predict the overall change trend of the ticket volume.

[0051] Figure 11 In the above table, the experimental results of the best scheme under each experimental condition are indicated in bold, and the experimental results of the suboptimal scheme are indicated in underlined, it can be seen that the performance of the present application is better than the other five comparative schemes, such as the comparative Diffusion-TS, the present application reduces the mean square error (MSE) by 71.68%, 51.54%, 58.18%, 9.44%, 62.24% and 65.99% respectively on the six data sets, and the average decrease is 53.20%.

[0052] Figure 12 It is shown that under the same experimental conditions, the experimental results of the present application, Diffusion-TS, TimeDiff, TTS-GAN, Dlinear, FEDformer on six kinds of data sets such as Weather, Traffic, Electricity, ETTh1, Wqdata and Wqdata+Wddata are shown, and the measurement indexes are Context-Fréchet inception distance (Context-FID) and predictive score (Predictive Score); Context-FID not only measures the distribution similarity between the generated sequence and the real sequence, but also considers the matching degree of the conditional information, and a low Context-FID value indicates that the prediction sequence quality is high and meets the conditions; Predictive Score is used to evaluate whether the model retains the statistical characteristics and semantic information of the real data, and a low value is better; the present application has an average improvement of 11.06% in Context-FID value and an average improvement of 38.60% in Predictive Score compared with the suboptimal scheme.

[0053] In summary, compared with the method of only relying on historical data statistical characteristics, the present application can effectively process non-stationary data under the assistance of background conditions, and the present application can adjust the generated trajectory in the early stage by explicitly modeling the impact of the sudden event through the conditional mechanism, so as to realize the prediction more in line with the actual scene.

[0054] In the present application, the terms "first", "second", "third" and the like are only used to distinguish similar objects, and do not have to be used to describe a specific order or sequence, and cannot be understood as indicating or implying relative importance. In the description, the directions or positional relationships indicated by "up", "down", "left", "right", "front" and "back" are based on the directions or positional relationships shown in the drawings, and are only for the convenience of describing the present application, and do not indicate or imply that the devices referred to must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as limiting the scope of protection of the present application. For those skilled in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances.

[0055] In addition, in the description of the present application, "a plurality of" means two or more, unless otherwise specified. The association relationship between the associated objects described by "and / or" indicates that there can be three relationships, for example, A and / or B can represent the three cases of A existing alone, A and B existing together, and B existing alone. The character " / " generally represents an "or" relationship between the associated objects before and after it.

[0056] The above is only a specific embodiment of the present application, but the design concept of the present application is not limited thereto, and any non-essential modification of the present application using this concept shall be regarded as an act of infringing the protection scope of the present application.

Claims

1. A water affair work order quantity prediction method based on a diffusion model, characterized in that: The method comprises the following steps: Step S1, obtaining a work order quantity dataset and a work order detail dataset, each record in the work order quantity dataset has a time period field, and the work order detail dataset is grouped and aggregated by hour according to the time period field of each record in the work order quantity dataset to obtain a plurality of groups of aggregated samples; Step S2, sequentially selecting L groups of aggregated samples from the work order detail dataset, and inputting the accepted content field and the work order category field in each group of aggregated samples into the Chinese BERT module of the conditional network to convert into work order detail word embeddings; Step S3, obtaining a training sample according to the work order quantity data set, the training sample containing a work order quantity history sequence, a work order quantity prediction sequence true value, and a temperature, holiday information and global time information sequence, inputting the work order quantity prediction sequence true value into a diffuser, adding noise in K steps until the work order quantity prediction sequence true value becomes Gaussian noise, obtaining a noisy work order quantity prediction sequence after adding noise in the kth step , k = 1, 2, …, K, is the current training sample flag; Step S4, the noisy work order quantity prediction sequence obtained after adding noise to the k=K step The noisy hidden layer prediction feature vector is obtained by inputting the current frame work order quantity prediction sequence into the encoder of the diffusion denoising network; the work order quantity history sequence, the air temperature, the holiday information and the global time information sequence of the training sample are input into the encoder of the conditional network to extract the historical sequence feature hidden vector; the historical sequence feature hidden vector is spliced with the work order detail word embedding; the splicing result is input into the decoder of the diffusion denoising network after being fused with the noisy hidden layer prediction feature vector; the next frame work order quantity prediction sequence is generated by weighting the current frame work order quantity prediction sequence, the decoder output and the noise random value; the next frame work order quantity prediction sequence is taken as the current frame work order quantity prediction sequence, and the step is repeated to obtain the final restored work order quantity prediction sequence. Step S5, minimizing the mean square error loss function between the finally restored work order quantity prediction sequence and the true value of the work order quantity prediction sequence, updating the parameters of the diffusion denoising network and the conditional network; enter step S2 to select the next L group of aggregated samples, obtain the next training sample in step S3, and until a set number of training samples are used, a trained prediction model is obtained. 2.The water affair work order quantity prediction method based on diffusion model according to claim 1, characterized in that: In step S1, the work order quantity dataset is divided into a work order quantity training set, a work order quantity validation set and a work order quantity test set, in step S3, L+H records are obtained from the work order quantity training set by a sliding window method, and then the training sample is obtained, the first L records in the training sample are used as the work order quantity history sequence, and the last H records are used as the true value of the work order quantity prediction sequence, and the history L temperature, holiday information and global time information sequences are obtained according to the first L records.

3. The water affair work order quantity prediction method based on diffusion model according to claim 2, characterized in that: The step S3 is represented as wherein, is the cumulative noise attenuation coefficient, is the work order quantity prediction sequence true value of the current sample, is the Gaussian noise.

4. The water affair work order quantity prediction method based on diffusion model according to claim 1 or 2 or 3, characterized in that: In the step S4, the encoder of the conditional network is according to the formula extracts the historical sequence feature hidden vector , according to the formula splices the historical sequence feature hidden vector with the ticket detail word embedding, wherein, represents self-attention, represents the ticket volume historical sequence of the current training sample, represents the air temperature, holiday information and global time information sequence of the current training sample, represents a linear fully connected layer, represents stop back propagation, represents splicing, represents the conditional hidden vector of the current training sample.

5. The water affair work order quantity prediction method based on diffusion model according to claim 1 or 2 or 3, characterized in that: In the step S4, the encoder of the diffusion denoising network obtains the noisy hidden layer prediction feature vector wherein, represents the normalization layer of the encoder of the diffusion denoising network, is the feedforward layer of the encoder of the diffusion denoising network, represents the first sub-layer output of the lth layer of the encoder of the diffusion denoising network, represents the second sub-layer output of the lth layer of the encoder of the diffusion denoising network.​​​ 6. The water affair work order quantity prediction method based on diffusion model according to claim 1 or 2 or 3, characterized in that: In the step S4, the next frame work order quantity prediction sequence According to the formula is calculated, wherein, is the cumulative noise attenuation coefficient, is the noise intensity added at the kth step, , is the Gaussian noise added at the kth step standard deviation, is the current frame work order quantity prediction sequence, is the next frame prediction sequence output by the decoder of the diffusion denoising network.

7. The water affair work order quantity prediction method based on diffusion model according to claim 2 or 3, characterized in that: In step S3, the global time information includes hour encoding, day encoding, week encoding and month encoding, and the specific values corresponding to the hour encoding, day encoding, week encoding and month encoding are obtained according to the L+H records in the work order quantity training set that are intercepted by the sliding window.

8. The water affair work order quantity prediction method based on diffusion model according to claim 2 or 3, characterized in that: In the step S5, the mean square error loss function is expressed as , is the final restored order quantity prediction sequence corresponding to the current sample, represents the prediction mean square error loss expectation of each noisy sequence prediction obtained by the current frame order quantity prediction sequence, represents the data distribution to which the current predicted time series sample is subjected.

Images