A deep learning work order quantity prediction method based on wavelet multi-resolution decomposition

By using a deep learning method based on wavelet multi-resolution decomposition, the problems of inaccurate feature extraction and high time complexity in traditional time series prediction methods are solved, and accurate prediction and precision improvement are achieved under sudden events.

CN118657255BActive Publication Date: 2024-11-22HUAQIAO UNIVERSITY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411113958.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-14
Publication Date
2024-11-22
Estimated Expiration
2044-08-14

AI Technical Summary

Technical Problem

Traditional time series forecasting methods struggle to extract complex temporal features from long series completely and accurately. Inaccurate modeling and large cumulative errors during inference lead to unsatisfactory forecasting results for long series. Furthermore, Transformer-based methods have the risk of high time complexity and narrow time windows that sever correlations, making it impossible to balance the ability to fit and predict local drastic fluctuations and unable to cope with sudden events.

Method used

We employ a deep learning method based on wavelet multi-resolution decomposition, using the ExWaveformer model for event detection and feature extraction. By combining wavelet enhancement layers and cross-attention modules, we reduce time complexity, extract non-stationary features, and integrate global trends and local drastic fluctuations to improve prediction accuracy.

Benefits of technology

It enables accurate prediction in the event of emergencies, reduces time complexity, improves prediction accuracy, and can completely and accurately extract the sequence features of work orders, adapting to the needs of practical application scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118657255B_ABST
    Figure CN118657255B_ABST
Patent Text Reader

Abstract

The present invention provides a deep learning work order quantity prediction method based on wavelet multi-resolution decomposition, which belongs to the field of time series prediction, including: inputting each group of work order samples into the event detector of the ExWaveformer model to extract the work order type features; obtaining the work order quantity historical input sequence and the prediction input sequence and inputting them into the encoder and decoder respectively; decomposing the work order quantity prediction input sequence into a trend component and a periodic random component, extracting the frequency domain features of the periodic random component and reconstructing it back to the time domain, using self-attention to extract the associated features in the time domain and encoding them as latent vectors in the encoder; calculating the time domain cross-attention to obtain the reconstructed periodic random latent vector, fusing it with the trend component to obtain the final generated work order quantity prediction sequence; step S6, updating parameters, and finally obtaining a trained prediction model. The present invention can reduce time complexity, improve prediction accuracy, and can still accurately predict when emergencies occur in actual application scenarios.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of time series prediction, and in particular to a deep learning work order quantity prediction method based on wavelet multi-resolution decomposition. Background Art

[0002] As cities expand and populations become denser, the requirements for water companies to efficiently and reliably provide high-quality products and services continue to increase in order to ensure the orderly conduct of various economic activities. Companies are eager to use the main historical business data accumulated on the water service platform to predict data changes over a long period of time in the future, providing a basis for daily management, quality improvement and efficiency enhancement.

[0003] However, traditional time series prediction methods have difficulty in completely and accurately extracting complex time series features from long sequences, and the modeling is inaccurate. The accumulated error during inference is large, resulting in unsatisfactory prediction results for long sequences. In recent years, the Transformer method based on the attention mechanism has made great progress in the problem of extracting associations in long sequences, but its use of windowed inputs poses a risk of the associations in the sequence being severed. The point-to-point attention calculation mechanism has a high time complexity, which places high demands on computing power when the prediction method is deployed and applied, restricting the increase in prediction length, thus affecting the widespread application of long sequence prediction.

[0004] Therefore, more and more models have improved the problems of Transformer, but some problems still exist: 1) The higher time complexity restricts the width of the input time window, and the narrow time window has the risk of severing the correlation relationship; 2) When reducing the time complexity, it is impossible to take into account the fitting prediction ability of local drastic fluctuations, and the prediction accuracy still has room for improvement; 3) The attention calculation based on probability weights cannot cope with the problem of emergency prediction in actual application scenarios. Summary of the invention

[0005] The main purpose of this invention is to propose a deep learning work order quantity prediction method based on wavelet multi-resolution decomposition, which can reduce time complexity, improve prediction accuracy, and still be able to accurately predict when emergencies occur in actual application scenarios.

[0006] The present invention is achieved through the following technical solutions:

[0007] A deep learning work order quantity prediction method based on wavelet multi-resolution decomposition includes the following steps:

[0008] Step S1, split the work order data set into a work order training set, a work order verification set and a work order test set, and split the work order quantity data set into a work order quantity training set, a work order quantity verification set and a work order quantity test set;

[0009] Step S2, randomly select N1 groups of training samples from the work order training set, input the work order acceptance content field and the work order category field contained in each group of training samples into the event detector of the ExWaveformer model, and fine-tune the event detector based on Chinese BERT with the constructed prompt template, wherein the ExWaveformer model is based on a codec architecture and consists of an event detector, an encoder, and a decoder;

[0010] Step S3, randomly selecting N2 groups of training samples from the work order quantity training set, combining the historical work order quantity sequence, temperature sequence, holiday information sequence and global time information contained in each group of training samples for event type detection encoding to input into the encoder as the historical input sequence, and replacing half of the data of the historical input sequence with subsequent predicted data in a sliding window manner, and then input into the decoder as the predicted input sequence, wherein the global time information includes year, month and day information;

[0011] Step S4: extract frequency domain features from the historical input sequence through the wavelet enhancement layer of the encoder and reconstruct it back to the time domain, and use self-attention in the time domain to extract the associated features of the result reconstructed back to the time domain and encode it into a latent vector;

[0012] Step S5: Decompose the prediction input sequence into trend components and periodic random components in the decoder, extract frequency domain features from the periodic random components through the wavelet enhancement layer of the decoder, and reconstruct them back to the time domain. Perform a linear transformation on the latent vector obtained in step S4 to obtain a key value vector and a numerical vector. Use the sparse attention query vector, the key value vector, and the numerical vector in the cross-attention module of the decoder to calculate the time domain cross-attention to obtain the reconstructed periodic random latent vector, and fuse it with the trend component to obtain the final generated work order quantity prediction sequence.

[0013] Step S6, calculate the mean square error and mean absolute error between the final work order quantity prediction sequence and the actual sequence to update the parameters, and finally obtain a trained prediction model, use the validation set to verify the prediction model, obtain the mean square error based on the validation set, and repeat steps S2 to S6 until the mean square error based on the validation set no longer decreases.

[0014] Furthermore, in step S2, the event detector uses the Chinese BERT pre-trained model as the core module, uses the constructed prompt template during fine-tuning, uses the work order acceptance content field as the input text, and the work order category field as the classification label. During prediction, Chinese BERT is used to extract word features and word features, and then the event category is determined through named entity recognition and entity pair relationship type classification. Finally, each work order is statistically analyzed by category and hour to form an event type code.

[0015] Furthermore, in step S3, the length of the historical input sequence input to the encoder is , the length of the predicted input sequence input to the decoder is , a sliding window is used to obtain the data of the sequence input to the encoder and decoder each time, where the length of the historical input sequence is The sequence and length are The sequences are all historical sequences. In the prediction input sequence, the length is The sequence is a historical sequence, with a length of The sequence is the predicted sequence.

[0016] In step S3, the input part of the encoder uses a multi-time window buffer to expand the receptive field, and the multi-time window buffer is calculated by the formula To achieve, SG () indicates stopping the gradient back propagation. Indicates the encoder τ Time window l The layer hidden vector, is the splicing symbol, Indicates τ Time window l Layer expansion hidden vector, backward expansion n time window.

[0017] Furthermore, in step S4, the wavelet enhancement layer implementation process includes:

[0018] In the subspace system composed of orthogonal scaling functions and wavelet functions, the Mallat algorithm is used to perform wavelet orthogonal multi-resolution analysis on the periodic random component. The relationship between the decomposition and reconstruction coefficients of the approximate sequence and the detail sequence in each subspace is:

[0019] ,

[0020] ,

[0021] ,

[0022] ,

[0023] in, , are the scale coefficient and wavelet coefficient in wavelet multi-resolution decomposition, respectively. are the wavelet multi-resolution reconstruction coefficients, H and G is the coefficient matrix of the low-pass filter and high-pass filter composed of the selected scaling function and wavelet function, represents the low-pass filter coefficient matrix, represents the high-pass filter coefficient matrix, represents the correction coefficient matrix, , , For each frequency component, a learnable weight parameter is used in a learnable weighted gating unit, and the weight of each component is learned using back propagation, , , , , , Represents the Sigmoid() function, W z represents the learnable cross-correlation parameter matrix of adjacent wavelet components, W h represents the learnable wavelet component autocorrelation parameter matrix, k Indicates the number of detail components of wavelet decomposition.

[0024] Furthermore, in step S4, the encoder uses self-attention in the time domain to extract relevant features and uses relative position encoding to obtain a latent vector.

[0025] Furthermore, in step S5, the formula and The first l Decomposition of the layer input sequence to obtain trend components and periodic random components ,in, Indicates the decoder l The input sequence of the layer, Indicates the decoder l Tier L The input sequence of time windows is is the mean pooling function, The symbol for splicing.

[0026] Furthermore, in step S5, the cross attention module is based on the formula To achieve, , , , Indicates l The periodic random component of the first decomposition of the layer decoder, , Indicates l The first cycle random component of the layer decoder j time slice key-value vector, Indicates l The learnable weight matrix for encoding the relative position of the random component in the first cycle of the layer decoder, express i, j The difference between the sinusoidal position codes of two periodic random components, Indicates l The first cycle random component of the layer decoder i Time slice for j time slice query vector, Indicates l The first cycle random component of the layer decoder i Time slice for j The query vector encoded by the time slice position, Indicates l The layer decoder query vector can learn the weight matrix, Indicates l The layer decoder key-value learnable weight matrix, Indicates l The layer decoder value vector can learn the weight matrix, Indicates l The first sparse attention query vector of the layer τ The fragment corresponding to the time window is represented by the query vector Sparsely obtained, Indicates the decoder l Tier τ The attention normalized value matrix of the random component of the first cycle of the time window, Indicates that the decoder l Tier τ The attention score matrix of the random component of the first cycle of the time window, Indicates the decoder l Tier τ The numerical matrix of the random components of the first period of the time window, express Matrix i OK j Column elements, Indicates the decoder l Tier τ The first key-value matrix of the time window, express No. j Column elements, Indicates the encoder l Layer output.

[0027] Furthermore, in step S6, the mean square error is calculated according to the formula Calculate the mean absolute error according to the formula Calculate, where m is the length of the predicted sequence, and Respectively λ The predicted and true values ​​of samples.

[0028] It can be seen from the above description of the present invention that, compared with the prior art, the present invention has the following beneficial effects:

[0029] The present invention adopts the work order quantity sequence, temperature sequence, holiday information sequence and global time information to combine event type detection code as the historical input sequence, and uses the global time information including year, month and day to effectively solve the requirement of accurately and completely extracting features after the receptive field is expanded; uses the BERT-based event detector to encode the work order acceptance content field as the event type detection code, and then embeds and splices it with the work order quantity so that the model obtains the characteristic information of the emergency event; based on the convenience of wavelet time-frequency domain analysis of non-stationary change characteristics, adopts the wavelet multi-resolution decomposition / reconstruction module, removes the trend of the input work order sequence and then performs multi-resolution decomposition, uses the weighted gating unit to determine the weight of each frequency component to extract the frequency domain pattern characteristics in the non-stationary change process, then uses attention to extract the associated features of the reconstructed work order quantity time series, and finally integrates the time domain trend features, so as to have the ability to more accurately fit the global trend and local violent fluctuations, reduce the time complexity, and improve the prediction accuracy, so that accurate prediction can still be made when emergencies occur in actual application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0031] Figure 1 It is a flow chart of the present invention.

[0032] Figure 2 It is a schematic diagram of the overall structure of the present invention.

[0033] Figure 3 This is an example of a fragment of the work order data set of the present invention.

[0034] Figure 4 This is an example of a fragment of the work order quantity data set of the present invention.

[0035] Figure 5 This is the input data structure diagram of the present invention.

[0036] Figure 6 Schematic diagram of the wavelet enhancement layer of the present invention.

[0037] Figure 7 This is a time complexity comparison chart between the present invention and Transformer.

[0038] Figure 8 This is a comparison chart of the prediction accuracy between the present invention and Informer.

[0039] Fig. 9The present invention is compared with four existing methods in terms of mean square error (MSE) and mean absolute error (MAE) on four public data sets, namely Weather, Traffic, Electricity, and ETTh1, and one private data set, namely WOdata. DETAILED DESCRIPTION

[0040] like Figure 1 As shown in FIG. 1 , the deep learning work order quantity prediction method based on wavelet multi-resolution decomposition includes the following steps:

[0041] Step S1: Divide the work order dataset into a work order training set, a work order verification set, and a work order test set. An example of a fragment of the work order dataset is as follows: Figure 3 Specifically, the work order data set is divided into a work order training set, a work order verification set, and a work order test set in a ratio of 7:2:1. The work order data in the work order training set, the work order verification set, and the work order test set are grouped and split to obtain multiple groups of samples.

[0042] The work order quantity dataset WOdata is divided into a work order quantity training set, a work order quantity verification set, and a work order quantity test set. An example of a fragment of the work order quantity dataset is as follows: Figure 4 ; Specifically, the work order data set is divided into a work order training set, a work order verification set, and a work order test set in a ratio of 7:2:1. The work order data in the work order training set, the work order verification set, and the work order test set are grouped and segmented to obtain multiple groups of samples to meet the requirements of the ExWaveformer model for data format. After grouping, set the sequence length of each encoder input according to the requirements and the length of the sequence input to the decoder each time , for the encoder input sequence, and Respectively represent the length of the front segment and the back segment after the historical input sequence of the input encoder is divided into two segments, Indicates that the sequence of input decoder is of length The first part of is the historical input sequence, and its length is The latter part is the prediction sequence. .

[0043] Step S2, randomly select N1=32 groups of training samples from the work order training set, input the work order acceptance content field and work order category field contained in each group of training samples into the event detector of the ExWaveformer model, and use the constructed prompt template to fine-tune the Chinese BERT-based event detector, where the ExWaveformer model is based on a codec architecture and consists of an event detector, an encoder, and a decoder.

[0044] Specifically, the prompt template is constructed by splicing the prompt template after the input work order acceptance content field, such as: the user reported [mask] situation; the user complained about [mask] problem, etc.; [mask] is the input mask mark. The work order category field is used as the true value of the label and the event detector label prediction value to calculate the loss. The loss function uses the cross entropy loss to measure the prediction error and fine-tune the event detector accordingly;

[0045] Specifically, the cross entropy loss is calculated as follows:

[0046]

[0047] in, is the set of predicted work order types, Predict probability values ​​for predefined ticket event types, Predict the probability value for the predicted work order event type. The total number of predefined ticket event types.

[0048] The trained event extractor is used to detect event types from the acceptance content field, so that the ExWaveformer model can obtain event feature information, especially emergency event information. Specifically, during prediction, Chinese BERT is used to extract word features and word-to-word features, and then the event category is determined through named entity recognition and entity pair relationship type classification. Finally, each work order is statistically analyzed by category and hour to form an event type code.

[0049] Step S3, randomly select N2=32 groups of training samples from the work order quantity training set, and combine the historical work order quantity sequence, temperature sequence, holiday information sequence and global time information contained in each group of training samples into event encoding to input the historical input sequence into the encoder, and replace half of the data of the historical input sequence with the subsequent predicted data in a sliding window manner, and input it into the decoder as the predicted input sequence, wherein the global time information includes year, month and day information, such as Figure 4 In the example, directly fill the sequence with 0~23 to represent the hour code, fill the sequence with 0~6 to represent the day code, fill the sequence with 0~3 to represent the week code, and fill the sequence with 0~11 to represent the month code;

[0050] like Figure 2 As shown in the overall structural diagram, the input layer is responsible for preliminary processing of the original data to form the data structure required by the prediction model. Figure 5 This is the input data structure diagram of the present invention.

[0051] The initial values ​​of the historical input sequence and the predicted input sequence are zero, and a sliding window is used to obtain the data of the historical input sequence and the predicted input sequence input to the encoder and decoder each time. The size of the sliding window is the same as the set prediction duration.

[0052] Step S4: extract frequency domain features from the historical input sequence through the wavelet enhancement layer of the encoder and reconstruct it back to the time domain, and use self-attention in the time domain to extract the associated features of the result reconstructed back to the time domain and encode it into a latent vector;

[0053] Specifically, the input part of the encoder uses a multi-time window cache to expand the receptive field. The multi-time window cache is implemented by the formula ,in, SG () indicates stopping the gradient back propagation. Indicates the encoder τ Time window l The layer hidden vector, WEL () represents the wavelet enhancement layer, is the splicing symbol, Indicates τ Time window l Layer expansion hidden vector, backward expansion n time window.

[0054] The wavelet enhancement layer is Figure 6 As shown in the figure, the wavelet multi-resolution decomposition algorithm is used to decompose the historical input sequence in the frequency domain, extract effective features such as local violent and subtle fluctuations, and then reconstruct the time domain sequence. The residual structure is used to eliminate the model degradation tendency and normalize it to reduce the model divergence risk. Then, the trend component of the time domain reconstructed sequence is removed and the dimension is transformed through the feedforward network (FFN). The sequence is again regularized by residual and normalization to remove the trend component and enter the next layer of encoder. The specific implementation process includes:

[0055] First, we construct an orthogonal subspace for the piecewise polynomial function we want to fit; , subspace The dimension is ;;The following relationship exists among the scale function subspaces: ; The scale subspace and wavelet subspace at each scale are orthogonal to each other and have the following relationship: ; Then there exists the following spatial decomposition relationship: ;

[0056] In the subspace system composed of orthogonal scaling functions and wavelet functions, the Mallat algorithm is used to perform wavelet orthogonal multi-resolution analysis on the periodic random component. The relationship between the decomposition and reconstruction coefficients of the approximate sequence and the detail sequence in each subspace is:

[0057] ;

[0058] ;

[0059] ;

[0060] ;

[0061] in, , are the scale coefficient and wavelet coefficient in wavelet multi-resolution decomposition, respectively. are the wavelet multi-resolution reconstruction coefficients, H and G The low-pass filter and high-pass filter are composed of the selected scaling function and wavelet function. represents the low-pass filter coefficient matrix, represents the high-pass filter coefficient matrix, represents the correction coefficient matrix, , , In the learnable weighted gating unit, each frequency component is determined using a learnable weight parameter, and the weight of each component is learned using back propagation. The weighted gating unit formula is:

[0062] , , , , , Represents the Sigmoid() function, W z represents the learnable cross-correlation parameter matrix of adjacent wavelet components, W h represents the learnable wavelet component autocorrelation parameter matrix, k Indicates the number of detail components of wavelet decomposition.

[0063] In order to adapt to the feature position marking after the receptive field is expanded to multiple windows, self-attention is used in the time domain to extract related features and relative position encoding is used to obtain the latent vector. The specific implementation method is as follows:

[0064] ;

[0065] ;

[0066] ,in, Indicates l The first sparse attention query vector of the layer τ The fragment corresponding to the time window is represented by The query vector is obtained by sparseness.

[0067] Step S5: Decompose the predicted input sequence into trend component and periodic random component in the decoder, extract frequency domain features from the periodic random component through the wavelet enhancement layer of the decoder and reconstruct it back to the time domain. Perform linear transformation on the latent vector obtained in step S4 to obtain the key value vector and the numerical vector. Use the sparse attention query vector in the cross attention module of the decoder. , key-value vector and a numeric vector Calculate the time domain cross attention to obtain the reconstructed periodic random latent vector, and fuse it with the trend component to obtain the final generated work order quantity prediction sequence;

[0068] The STL decomposition algorithm is used in the decoder to decompose the forecast input sequence into trend components and periodic random components. l The input of the layer is calculated using the formula and The first l Decomposition of the layer input sequence to obtain trend components and periodic random components ,in, Indicates the decoder l The input sequence of the layer, Indicates the decoder l Tier L The input sequence of time windows is is the mean pooling function, It is a concatenation symbol used to connect different sequence blocks.

[0069] In addition to the feature extraction function of the encoder, the decoder adds a cross-attention module to perform sequence reasoning, which enables the use of local current time series fragments to infer the complete prediction sequence. The reasoning process is as follows:

[0070] ,in, , Respectively represent l Tier Decomposed periodic random component and trend component; Indicates the extracted The weight of the trend component; the final forecast value is the weighted sum of the two components .

[0071] The cross-attention module enables the historical sequence features learned by the ExWaveformer model to be integrated into the predicted sequence in a cross-attention manner. The residual structure is used to eliminate the model degradation tendency and then normalize it to reduce the risk of model divergence. Then, the trend component of the time domain reconstructed sequence is removed and the dimension is transformed through a feedforward network (FFN). The sequence is again residually and normalized to remove the trend component before entering the next layer of the decoder.

[0072] The cross attention module is based on the formula To achieve, , , , Indicates l The key vector of the jth time slice of the first period random component of the layer decoder, Indicates l The learnable weight matrix for encoding the relative position of the random component in the first cycle of the layer decoder, express i, j The difference between the sinusoidal position codes of two periodic random components, Indicates l The first cycle random component of the layer decoder i Time slice for j A time slice query vector, Indicates l The first cycle random component of the layer decoder i Time slice for j The query vector encoded with the time slice position, Indicates l The query vector of the layer decoder can learn the weight matrix, Indicates l The layer decoder key vector can learn the weight matrix, Indicates l The layer decoder value vector can learn the weight matrix, Indicates l The first sparse attention query vector of the layer τ The fragment corresponding to the time window is represented by the query vector Sparsely obtained, Indicates the decoder l Tier τ The attention normalized value matrix of the random component of the first cycle of the time window, Indicates that the decoder l Tier τ The attention score matrix of the random component of the first cycle of the time window, Indicates the decoderl Tier τ The numerical matrix of the random components of the first period of the time window, express Matrix i OK j Column elements, Indicates the decoder l Tier τ The first key-value matrix of the time window, express No. j Column elements, Indicates the encoder l Layer output.

[0073] For sparse attention query vectors , is to reduce the time complexity of the model. The process of obtaining this quantity is:

[0074] (1) K Random sampling from a vector collection vector, denoted as ;

[0075] (2) Calculation Vector collection each q Attention scores, select the one with the highest score indivual q ,the remaining q Fill with the mean value, denoted as ;

[0076] The sparse attention calculation method is as follows: .

[0077] like Figure 7 The figure shows the time complexity comparison between the sparse model of the present invention and the standard model Trasformer on the Wodata dataset. It can be seen that the sparsity of attention can effectively reduce the time complexity of the model.

[0078] Step S6, calculating the mean square error and mean absolute error between the final work order quantity prediction sequence and the actual sequence to update the parameters, and finally obtaining a trained prediction model, using the validation set to verify the prediction model, obtaining the mean square error based on the validation set, and repeating steps S2 to S6 until the mean square error based on the validation set no longer decreases;

[0079] Specifically, the mean square error and mean absolute error between the final work order quantity prediction sequence and the actual sequence are calculated, and then back-propagation is performed through the Adam optimizer to update the network parameters and finally obtain a trained prediction model; the prediction model is verified using the validation set to obtain a prediction sequence generated based on the validation set, and the mean square error (MSE) between the prediction sequence generated based on the validation data set and the actual prediction sequence is calculated. The mean of the mean square error of all groups of data is the MSE error based on the validation data set; when the MSE error based on the validation data set no longer decreases, it means that the network parameters have been updated, the prediction model has achieved the optimal fitting effect, and the model training ends; the input sequence given by the prediction task is input into the trained ExWaveformer model, and the final prediction sequence is output to complete the prediction.

[0080] The mean square error is calculated according to the formula Calculate the mean absolute error according to the formula Calculate, where m is the length of the predicted sequence, and Respectively λ The predicted and true values ​​of samples.

[0081] Figure 8 It shows the comparison and changes in prediction accuracy between the method of the present invention (ExWaveformer) and Informer when an emergency occurs under the same experimental conditions. It can be seen that the method of the present invention (ExWaveformer) has a greater advantage in fitting accuracy, with the prediction accuracy improved by 46.32%.

[0082] Fig. 9 The table shows the experimental results of five methods, ExWaveformer, LSTM, FEDformer, Prophet, and Informer, on five datasets, Weather, Traffic, Electricity, ETTh1, and WOdata, under the same experimental conditions. The measurement criteria are mean square error (MSE) and square absolute error (MAE). Under each experimental condition, the experimental results of the best performing model are bolded in the table. Figure 7 It can be seen that the method of the present invention (ExWaveformer) has a significant improvement over the other four methods. For example, compared with the FEDformer method, the method of the present invention (ExWaveformer) reduces the mean square error (MSE) by 27.31%, 35.27%, 23.28%, 3.44% and 42.62% on five data sets, respectively, with an average decrease of 26.39%.

[0083] In the present invention, the terms "first", "second", "third", etc. are only used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence, nor can they be understood as indicating or implying relative importance. In the description, the directions or positional relationships indicated by "upper", "lower", "left", "right", "front", and "back" are based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention, and are not intended to indicate or imply that the device referred to must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as limiting the scope of protection of the present invention. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0084] In addition, in the description of this application, unless otherwise specified, "plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0085] The above is only a specific implementation of the present invention, but the design concept of the present invention is not limited thereto. Any non-substantial changes to the present invention using this concept shall be deemed as an infringement of the protection scope of the present invention.

Claims

1. A deep learning work order quantity prediction method based on wavelet multi-resolution decomposition, characterized by: The steps include: Step S1, split the work order data set into a work order training set, a work order verification set and a work order test set, and split the work order quantity data set into a work order quantity training set, a work order quantity verification set and a work order quantity test set; Step S2: Randomly select N1 groups of training samples from the work order training set, input the work order acceptance content field and work order category field contained in each group of training samples into the event detector of the ExWaveformer model, and fine-tune the Chinese BERT-based event detector with the constructed prompt template, wherein the ExWaveformer model is based on a codec architecture, and is composed of an event detector, an encoder, and a decoder. The prompt template is constructed by splicing the prompt template after the input work order acceptance content field, and the work order category field is used as the true value of the label and the predicted value of the event detector label to calculate the loss. The loss function adopts the cross entropy loss. to measure the prediction error and fine-tune the event detector accordingly; Step S3, randomly selecting N2 groups of training samples from the work order quantity training set, combining the historical work order quantity sequence, temperature sequence, holiday information sequence and global time information contained in each group of training samples for event type detection encoding to input into the encoder as the historical input sequence, and replacing half of the data of the historical input sequence with subsequent predicted data in a sliding window manner, and then input into the decoder as the predicted input sequence, wherein the global time information includes year, month and day information; Step S4: extract frequency domain features from the historical input sequence through the wavelet enhancement layer of the encoder and reconstruct it back to the time domain, and use self-attention in the time domain to extract the associated features of the result reconstructed back to the time domain and encode it into a latent vector; Step S5: Decompose the prediction input sequence into trend components and periodic random components in the decoder, extract frequency domain features from the periodic random components through the wavelet enhancement layer of the decoder, and reconstruct them back to the time domain. Perform a linear transformation on the latent vector obtained in step S4 to obtain a key value vector and a numerical vector. Use the sparse attention query vector, the key value vector, and the numerical vector in the cross-attention module of the decoder to calculate the time domain cross-attention to obtain the reconstructed periodic random latent vector, and fuse it with the trend component to obtain the final generated work order quantity prediction sequence. Step S6, calculate the mean square error and mean absolute error between the final work order quantity prediction sequence and the actual sequence to update the parameters, and finally obtain a trained prediction model, use the validation set to verify the prediction model, obtain the mean square error based on the validation set, and repeat steps S2 to S6 until the mean square error based on the validation set no longer decreases.

2. According to claim 1, a deep learning work order quantity prediction method based on wavelet multi-resolution decomposition is characterized in that: In step S2, the event detector uses the Chinese BERT pre-trained model as the core module, uses the constructed prompt template during fine-tuning, uses the work order acceptance content field as the input text, and the work order category field as the classification label. During prediction, Chinese BERT is used to extract word features and word-to-word features, and then the event category is determined through named entity recognition and entity pair relationship type classification. Finally, each work order is statistically analyzed by category and hour to form an event type code.

3. According to claim 1, a deep learning work order quantity prediction method based on wavelet multi-resolution decomposition is characterized in that: In step S3, the length of the historical input sequence input to the encoder is , the length of the predicted input sequence input to the decoder is , a sliding window is used to obtain the data of the sequence input to the encoder and decoder each time, where the length of the historical input sequence is The sequence and length are The sequences are all historical sequences. In the prediction input sequence, the length is The sequence is a historical sequence, with a length of The sequence is the predicted sequence.

4. A deep learning work order quantity prediction method based on wavelet multi-resolution decomposition according to claim 1, 2 or 3, characterized in that: In step S3, the input part of the encoder uses a multi-time window buffer to expand the receptive field. The multi-time window buffer is calculated by the formula To achieve, SG () indicates stopping the gradient back propagation. Indicates the encoder τ Time window l Layer hidden vector, is the splicing symbol, Indicates τ Time window l Layer expansion hidden vector, backward expansion n time window.

5. A deep learning work order quantity prediction method based on wavelet multi-resolution decomposition according to claim 1, 2 or 3, characterized in that: In step S4, the wavelet enhancement layer implementation process includes: In the subspace system composed of orthogonal scaling functions and wavelet functions, the Mallat algorithm is used to perform wavelet orthogonal multi-resolution analysis on the periodic random component. The relationship between the decomposition and reconstruction coefficients of the approximate sequence and the detail sequence in each subspace is: , , , ,in, , are the scale coefficient and wavelet coefficient in wavelet multi-resolution decomposition, respectively. , They are the upper and lower half-band coefficients of wavelet multi-resolution reconstruction, H and G is the coefficient matrix of the low-pass filter and high-pass filter composed of the selected scaling function and wavelet function, represents the low-pass filter coefficient matrix, represents the high-pass filter coefficient matrix, represents the correction coefficient matrix, , , For each frequency component, a learnable weight parameter is used in a learnable weighted gating unit, and the weight of each component is learned using back propagation, , , , , , Represents the Sigmoid() function, W z represents the learnable cross-correlation parameter matrix of adjacent wavelet components, W h represents the learnable wavelet component autocorrelation parameter matrix, k' Indicates the number of detail components of wavelet decomposition.

6. The method for predicting work order quantity based on deep learning of wavelet multi-resolution decomposition according to claim 5 is characterized in that: In step S4, the encoder uses self-attention in the time domain to extract relevant features and uses relative position encoding to obtain a latent vector.

7. A deep learning work order quantity prediction method based on wavelet multi-resolution decomposition according to claim 1, 2 or 3, characterized in that: In step S5, the formula and The first l Decomposition of the layer input sequence to obtain trend components and periodic random components ,in, Indicates the decoder l The input sequence of the layer, Indicates the decoder l Tier L The input sequence of time windows is is the mean pooling function, The symbol for splicing.

8. A deep learning work order quantity prediction method based on wavelet multi-resolution decomposition according to claim 1, 2 or 3, characterized in that: In step S5, the cross attention module is based on the formula To achieve, , , , Indicates l The first cycle random component of the layer decoder, , Indicates l The second cycle random component of the layer encoder, Indicates l The first cycle random component of the layer decoder j time slice key-value vector, Indicates l The learnable weight matrix for encoding the relative position of the random component in the first cycle of the layer decoder, express i, j The difference between the sinusoidal position codes of the random components of two time slices, Indicates l The first cycle random component of the layer decoder i Time slice for j time slice query vector, Indicates l The first cycle random component of the layer decoder i Time slice for j The query vector encoded by the time slice position, Indicates l The layer decoder query vector can learn the weight matrix, Indicates l The layer decoder key-value vector can learn the weight matrix, Indicates l The layer decoder numerical vector can learn the weight matrix, Indicates l The first sparse attention query vector of the layer τ The fragment corresponding to the time window is represented by the query vector Sparsely obtained, express No. i element, which is the same as the l The first cycle random component of the layer decoder i Time slices correspond to Indicates the decoder l Tier τ The attention normalized value matrix of the random component of the first cycle of the time window, Indicates the decoder l Tier τ The attention score matrix of the random component of the first cycle of the time window, Indicates the decoder l Tier τ The numerical matrix of the random components of the first period of the time window, express Matrix i Line j Column element, which is the l The random component of the first cycle of the layer encoder i time slice and l The first cycle random component of the layer decoder j The attention score between time slices, Indicates the decoder l Tier τ The first key-value matrix of the time window, express No. j Column element, which is the same as the l The first cycle random component of the layer decoder j Time slices correspond to Indicates the encoder l Layer output.

9. A deep learning work order quantity prediction method based on wavelet multi-resolution decomposition according to claim 1, 2 or 3, characterized in that: In step S6, the mean square error is calculated according to the formula Calculate the mean absolute error according to the formula Calculate, where m is the length of the predicted sequence, and Respectively λ The predicted and true values ​​of samples.