Space sequence data long-term prediction method based on fusion of CNN and RNN

Abstract

The invention relates to the technical field of spatial sequence data prediction, in particular to a spatial sequence data long-term prediction method based on fusion of CNN and RNN, and the method comprises the steps: constructing a plurality of simple feature generators through an original sequence to form a multi-dimensional feature generator, and achieving the data generation thought of inputting a time step serial number and outputting a corresponding time step multi-dimensional feature sequence. Therefore, the monitoring sequence is prevented from being directly input into the prediction model, and the multi-dimensional feature sequence of the monitoring sequence is adopted as prediction input data. For spatial sequence data, a CNN module is added into a prediction model to construct spatial dimension feature extraction and a time dimension feature extractor based on RNN to construct a time dependency relationship, and a single-model one-time prediction output sequence is simplified into a single-model multi-time prediction combination which outputs a result of one time step each time to form a sequence.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of spatial sequence data prediction, in particular to a long-term prediction method for spatial sequence data based on fusion of CNN and RNN. BACKGROUND

[0002] Sequence data prediction is a core research direction in the field of machine learning, aiming to infer the trend and law of future data from historical observation values. Sequence data prediction can be classified into single-step prediction and multi-step prediction according to the length of the predicted output sequence. Single-step prediction refers to predicting a single sequence data at the next time point based on historical sequence, and multi-step prediction refers to predicting sequence data at multiple consecutive time points based on historical sequence. Existing machine learning methods can well handle single-step prediction problems, but have many serious defects when dealing with multi-step prediction problems.

[0003] There are two strategies for traditional sequence data multi-step prediction methods, namely multi-model single-output strategy and single-model multi-output strategy. Based on these two strategies, five specific sequence data multi-step prediction models can be subdivided, namely direct multi-step prediction, direct-recursion hybrid prediction, recursive multi-step prediction, multi-output prediction, and Seq2seq prediction. The first two belong to the multi-model single-output strategy, and the last three belong to the single-model multi-output strategy.

[0004] A serious defect of the multi-model single-output strategy modeling is that the number of prediction models is equal to the length of the predicted sequence, and training these models requires a large amount of computing resources. In addition, it is generally believed that the distance between two time steps increases or decreases, and the correlation between the two data weakens, so a defect of direct multi-step prediction is that the sequence correlation law learned by the model for predicting data at the farthest time in the future is very weak, and the prediction effect is poor. Direct-recursion hybrid prediction can solve this problem, but another serious problem arises after adding a recursive mode to the model, i.e. the error accumulation problem caused by using the current predicted data as input data for the next step prediction.

[0005] Different methods of single model multi-output strategy correspond to different defects. The defect of recursive multi-step prediction is the same as that of direct-recursive hybrid prediction, and error accumulation problem occurs in the prediction process; the multi-output prediction model can output multiple future multiple time steps at the same time, and the output sequence of each time step is independent of each other, which is contrary to the time dependent relationship of sequence data; Seq2seq prediction can solve the disadvantages of multi-output prediction, considering the time dependent relationship of the prediction output sequence, but when the required prediction output sequence length is greater than the historical sequence, this method is invalid, because it is impossible to construct a training data set, and there may be such a problem in actual engineering, for example, the structure is required to be predicted for the whole life cycle in engineering structure design or operation and maintenance, and the service life of the structure is 50 years or 100 years, and the length of the existing historical monitoring sequence data is only one year or two years, so the method cannot be used for prediction.

[0006] The defects of the above sequence data prediction methods ultimately come from the input of the original sequence, and the defects can be solved by changing the form of the input data and bypassing the direct input of the original sequence, and how to generate independent input data based on the original data needs further research.

[0007] In most engineering scenarios, structure condition monitoring is often carried out by multiple sensors working together to collect data, and such sequence data has spatial characteristics, and the traditional sequence prediction method cannot fully consider the spatial characteristics of the spatial sequence data, so it is necessary to further study. SUMMARY

[0008] In view of the problems in the prior art, the application provides a long-term prediction method for spatial sequence data based on fusion of CNN and RNN.

[0009] The application is realized by the following technical solutions: A long-term prediction method for spatial sequence data based on fusion of CNN and RNN, comprising: Step 1, constructing a multi-dimensional feature generator and a deep learning network model structure based on fusion of CNN and RNN, wherein the multi-dimensional feature generator is constructed according to the feature components of the input sequence; The deep learning network model structure based on fusion of CNN and RNN comprises a spatial dimension feature extractor based on CNN and a time dimension feature extractor based on RNN; Step 2, the multi-dimensional feature generator generates a spatial multi-dimensional feature sequence according to the time step; Step 3, the spatial multi-dimensional feature sequence passes through a sliding window and a data iterator, and a batch of spatial multi-dimensional feature sample sequences are generated; Step 4, taking the spatial multi-dimensional feature sample sequence as input, the hidden spatial feature sequence of the spatial multi-dimensional feature sample sequence is obtained through the spatial dimension feature extractor based on CNN; Step 5, taking the hidden space feature sequence as input, obtaining the time dependence of the hidden space feature sequence through the RNN-based time dimension feature extractor; Step 6, the last layer of the RNN-based time dimension feature extractor combines the adaptive attention mechanism to generate the time information vector as the prediction result.

[0010] Preferably, the loss function of the deep learning network model structure based on the fusion of CNN and RNN is L2, and the optimization algorithm is Adam or SGD.

[0011] Preferably, the temperature sequence is taken as the input sequence, and the feature components of the temperature sequence include periodic features, fluctuation features and trend features. The multi-dimensional feature generator includes a periodic feature generator, a fluctuation feature generator and a trend feature generator. Preferably, in step 2, the temperature sequence is taken as the input sequence, and the multi-dimensional feature generator generates a spatial multi-dimensional feature sequence according to the time step, including: The periodic feature generator generates periodic feature values by fitting a multi-parameter Fourier function; The fluctuation feature generator obtains a fluctuation feature sequence by combining statistics and sampling; The trend feature generator performs a polynomial fitting on the feature values of the spatial sequence on a monthly scale to generate a monthly trend feature, and generates a trend feature for all time steps according to the monthly trend feature; The periodic feature values, the fluctuation feature sequence and the trend feature are spliced to obtain a 3D spatial multi-dimensional feature (m, n, f), wherein m, n and f are respectively the total number of time steps, the number of monitoring points and the number of feature components of the monitoring values.

[0012] Preferably, the generation process of the fluctuation feature sequence is: The periodic features in the temperature sequence are removed to obtain a fluctuation sequence that fluctuates around 0; The time distance between adjacent peak values and valley values in the fluctuation sequence is counted to calculate the average distance between adjacent peak values and valley values; The frequency histogram of the fluctuation sequence is counted, and then sampled; The sampling structure is rearranged in descending order; The sampling is performed multiple times, and the sampling results are spliced in the sampling order to obtain the fluctuation feature sequence.

[0013] Preferably, in step 3, the data of the spatial multi-dimensional feature sequence is divided into m-k small samples in the dimension represented by m, and each small sample is a small spatial sample sequence, forming a 4D form (m-k, k, n, f).

[0014] Preferably, in step 5, the data size of the hidden space feature sequence (batchsize, k, 1, f) is converted into (batchsize, k, f) by dimension reduction and transposition, and then input into the RNN-based time dimension feature extractor.

[0015] Preferably, in step 6, the last layer of the RNN-based time dimension feature extractor combines an adaptive attention mechanism to generate a time information vector as a prediction result, and the calculation formula is:

[0016]

[0017]

[0018] wherein, is the attention weight of the output hidden state of the jth time step of the lth layer, is the transition quantity for calculating the attention weight, is a coefficient updated in the training process.

[0019] An electronic device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the steps of the method when executing the computer program.

[0020] A storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the steps of the method.

[0021] Compared with the prior art, the present application has the following beneficial effects: The spatial sequence data long-term prediction method based on fusion of CNN and RNN of the present application can output multi-dimensional sequence features (including periodic, fluctuation, trend and other features) of the corresponding time step by constructing a multi-dimensional feature generator and inputting the serial number of the corresponding time step of the to-be-input feature, and can avoid using the predicted data of the current step as the input data for the next step prediction, thereby avoiding error accumulation. The multi-output prediction model has the defect that each step prediction considers the time dependence of the sequence data, and avoids outputting the prediction results of multiple time steps at the same time. The seq2seq has the defect that each prediction only predicts the result of one time step, and multiple predictions are performed to achieve the effect of multi-step prediction, so that a training data set can be constructed to train the model regardless of the length of the predicted sequence.

[0022] In the structural monitoring scenario in engineering, the structure condition is often monitored by multiple sensors in cooperation, at this time the obtained sequence data is spatial sequence data, the spatial sequence data contains spatial information of the data, and it is also important to consider these spatial information in the long-term prediction process of sequence data, therefore, the innovation point of the present application is to add a CNN module in the neural sequence data prediction model to construct a spatial dimension feature extractor, which can fully extract the hidden spatial features of the spatial multi-dimensional feature sequence, and provide sufficient feature basis for spatial sequence prediction. At the same time, based on the time dimension feature extractor of RNN, the idea of constructing time dependence relationship is used to simplify the single model prediction output sequence into a single model multiple prediction combination, and the result of each output time step forms a sequence. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 is a flow chart of a spatial sequence data long-term prediction method based on fusion of CNN and RNN according to the present application; Figure 2 is a network framework diagram of a spatial sequence data long-term prediction method based on fusion of CNN and RNN according to the present application; Figure 3 is the change of the loss function in the training process in the embodiment of the present application; Figure 4 is the periodic feature fitting result in the embodiment of the present application; Figure 5 is the fluctuation sequence in the embodiment of the present application; Figure 6 is the histogram statistics of the fluctuation sequence in the embodiment of the present application; Figure 7 is the fluctuation feature sequence display in the embodiment of the present application; Figure 8 is the monthly trend feature fitting result in the embodiment of the present application; Figure 9 is the initial diagram of spatial sequence data long-term prediction training in the embodiment of the present application; Figure 10 is the late diagram of spatial sequence data long-term prediction training in the embodiment of the present application. DETAILED DESCRIPTION

[0024] The embodiments of the present application are described below by specific specific embodiments, and those skilled in the art can easily understand other advantages and effects of the present application from the content disclosed in the present specification.

[0025] The present application will now be described with reference to the attached drawings, which are presented for the purpose of illustration and description and are not intended to limit the scope of the application. The application can be implemented in numerous ways, including, but not limited to, the embodiments described herein. These embodiments are presented for the purpose of illustration and description and are not intended to limit the scope of the application. The terminology used in the description presented herein is not intended to limit the scope of the application. Rather, the terminology is used solely by way of reference to assist in understanding the present application. In the drawings, like reference numerals are used to indicate like elements throughout the various figures.

[0026] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

[0027] The application will now be described in further detail with reference to the specific examples, which are intended to explain and not to limit the application.

[0028] Since a large amount of computing resources is required for long-term prediction of sequence data by using a multi-model single-output strategy for modeling, the long-term prediction strategy of the present application is a single-model multi-output. The recursive multi-step prediction method in the existing single-model multi-output method takes the prediction result of the current step as one of the input data for the next step prediction, which has an error accumulation problem. The multi-output prediction model simultaneously outputs the prediction results of all time steps during prediction, and the prediction results of different time steps are independent of each other, which is contrary to the existing time-dependent relationship of sequence data. The seq2seq continuously outputs the prediction results of adjacent time points each time, and the prediction can consider the time-dependent relationship, but when the prediction time step is too long, it is impossible to construct a training data set to train the model. It is found that the defects of these methods are affected by the input or output form of the data.

[0029] The application discloses a long-term prediction method for spatial sequence data based on fusion of CNN and RNN, referring to Figure 1 , 2 , comprising the following steps: Step 1, constructing a multi-dimensional feature generator and a deep learning network model structure based on fusion of CNN and RNN, wherein the multi-dimensional feature generator is constructed according to the feature components of the input sequence; The deep learning network model structure based on fusion of CNN and RNN includes a spatial dimension feature extractor based on CNN and a time dimension feature extractor based on RNN. The spatial dimension feature extractor based on CNN is composed of multiple convolution layers and is used for extracting spatial features of the spatial multi-dimensional feature sequence. The time dimension feature extractor based on RNN is composed of multiple RNN layers and is used for extracting the time-dependent relationship of the spatial multi-dimensional feature sequence.

[0030] Each RNN layer of the RNN-based time-dimension feature extractor includes two subparts, an encoder and a decoder, and the constituent units of the encoder and the decoder are LSTM or GRU, and an adaptive attention mechanism is added when generating the information vector.

[0031] The loss function of the deep learning network model structure based on the fusion of CNN and RNN is L2, and the optimization algorithm is Adam or SGD. The feature set used for training is constructed by a spatial multi-dimensional feature sequence, and the target set is constructed by spatial sequence data. Wherein, the input of the multi-dimensional feature generator is the time step sequence (1, 2, 3, 4…), and the spatial multi-dimensional feature sequence can be output.

[0032] In order to make the data shape adapt to the input requirements of the spatial-dimension feature extractor and the time-dimension feature extractor, data operations (sliding window) and data operations (dimension reduction and transposition) are performed to change the input shape of the data. Figure 2 In the formula, n represents the number of measuring points in the space, m represents the length of the space sequence, f represents the number of feature components that a single measuring point sequence can contain, and batchsize is the amount of data in a batch during the training process. is the hidden state of the recurrent neural network, is the feature of the kth time step of the sample input into the first RNN layer.

[0033] Step 2, the multi-dimensional feature generator generates a spatial multi-dimensional feature sequence according to the time step. The temperature sequence is inputted as the sequence to be inputted, and the feature components of the temperature sequence include periodic feature, fluctuation feature and trend feature. The multi-dimensional feature generator includes a periodic feature generator, a fluctuation feature generator and a trend feature generator.

[0034] (1) The periodic feature generator generates periodic feature values by fitting a multi-parameter Fourier function.

[0035] The Fourier function is used to fit the monitoring sequence corresponding to each measuring point in the sampling space respectively. The Fourier function that has completed fitting is the periodic feature generator, and the input time step can output the corresponding Fourier function value, which is used as the periodic feature of this time step. The formula of the Fourier function is as follows:

[0036] Wherein is the coefficient to be fitted of the Fourier function, t is the time step, m represents the number of sine and cosine pairs, and the larger the m is, the more details the fitted periodicity shows.

[0037] (2) The fluctuation feature generator obtains the fluctuation feature sequence by combining statistics and sampling. Specifically: The periodic characteristics in the temperature sequence are removed by monitoring the spatial sequence, and a fluctuation sequence fluctuating around 0 is obtained; The time distance between adjacent peak values and valley values in the fluctuation sequence is counted, and the average distance between adjacent peak values and valley values is calculated; The frequency histogram of the fluctuation sequence is counted, and then sampling is performed; the number of samples in a single sampling is determined by the average distance between adjacent peak values and valley values, and the sampling results are reordered from large to small, the maximum value represents the peak value of sampling, and the minimum value represents the valley value of sampling; Multiple sampling is performed, and the sampling results are spliced in the sampling order to obtain a fluctuation feature sequence.

[0038] (3) The trend feature generator performs a polynomial fitting on the feature values of the spatial sequence at a monthly scale to generate a monthly trend feature, and generates a trend feature for all time steps according to the monthly trend feature. Specifically: The spatial sequence with the periodic component removed is scaled by month, and the mean, minimum, 25th percentile, 50th percentile, 75th percentile, and maximum of the sequence in a single month are counted. The means of all months are fitted by a polynomial regression, and the fitted polynomial is the trend feature generator 1. The input count month number, and the output polynomial value represents the trend feature of the month. It is assumed that the trend features of all time steps in the same month are the same, so the trend features of all time steps in the same month are assigned. The least squares method is used to fit the polynomial , is the undetermined coefficient.

[0039] The mathematical expression of the least squares method for fitting a polynomial is as follows.

[0040]

[0041] where (·, ·) represents the inner product, is a polynomial determined by the least squares method, y is the observed data, and the solution is as follows: When formula (2-19) takes the minimum value, there exists

[0042] such that is established.

[0043] where:

[0044]

[0045]

[0046] ​

[0047]

[0048]

[0049] Thus the solution is :

[0050] Similarly, the minimum value, 25% quantile value, 50% quantile value, 75% quantile value and maximum value can be fitted by a polynomial regression to construct different trend feature generators.

[0051] Take a month as the observation time span, and count the mean, minimum value, 25% quantile value, 50% quantile value, 75% quantile value and maximum value of all feature values in the month. All these values of all months form a monthly sequence.

[0052] For each feature value of the monthly sequence, a function regression fitting is performed, and the monthly sequence number is input to output the trend feature of the corresponding month.

[0053] (4) The periodic feature value, fluctuation feature sequence and trend feature are spliced to obtain a 3D form of spatial multi-dimensional feature (m, n, f), wherein m, n and f are the total time step number, the number of measuring points and the number of feature components of the monitoring value, respectively.

[0054] Step 3, the spatial multi-dimensional feature sequence is batched to generate a spatial multi-dimensional feature sample sequence through a sliding window and a data iterator, specifically: Step 3.1, data operation is performed on the data layer represented by the spatial multi-dimensional feature data m, and the data of the spatial multi-dimensional feature sequence is divided into m-k small samples, each small sample being a small spatial sample sequence, forming a 4D form (m-k, k, n, f).

[0055] Step 3.2, batched sample data is generated through a data iterator, and the data amount of each batch is batchsize.

[0056] Step 4, taking the spatial multi-dimensional feature sample sequence as input, the hidden space feature sequence of the spatial multi-dimensional feature sample sequence is obtained through a CNN-based spatial dimension feature extractor; specifically: The batched sample data is input into the spatial dimension feature extractor. Each sample is a 3D data, which is a sequence of length k, and each time step of the sequence contains n spatial points, and each spatial point contains f sequence features. A spatial multi-dimensional feature sample sequence is input, and the correlation of all channels on a certain sequence feature is represented by a spatial feature value through convolution operation. Since there are f sequence features, there are f spatial feature values representing the correlation of the spatial sequence. The output result is defined as the hidden space feature sequence of the spatial sequence.

[0057] The input data shape of the spatial feature extractor is (batchsize, k, n, f), and the output data shape is required to be (batchsize, k, 1, f). According to the characteristics of the input and output data shapes and the complexity of the prediction problem, the convolution layer and the pooling layer are flexibly matched, and batch normalization processing can also be performed after the convolution layer.

[0058] The design of the convolution layer and the pooling layer is very flexible and can be adjusted according to actual needs. For example, assuming that k = 10, n = 10, and f = 8, the output feature map size is (k, 1, f), and the parameter settings of the corresponding CNN module of each convolution layer and pooling layer are as follows: Table 1 Parameter settings of each layer of the spatial dimension feature extractor

[0059] Step 5: The data size of the hidden space feature sequence (batchsize, k, 1, f) is converted to (batchsize, k, f) through dimension reduction and transposition, so that it meets the data input requirements of the time dimension feature extractor. Then it is input into the RNN-based time dimension feature extractor. Through the RNN-based time dimension feature extractor, the number of RNN layers in the time dimension feature extractor is set according to the complexity of the problem, Figure 2 All superscripts of symbols are the serial numbers of RNN layers, and RNN units can be LSTM or GRU, etc. The calculation process of the hidden space feature sequence input into the time dimension feature extractor from the spatial dimension feature extractor is as shown in Figure 2 to obtain the time dependence of the hidden space feature sequence.

[0060] Step 6: The last layer of the RNN-based time dimension feature extractor combines the adaptive attention mechanism to generate a time information vector as the prediction result, and the calculation formula is:

[0061]

[0062]

[0063] wherein, is the attention weight of the output hidden state of the lth layer at the jth time step, is the transition amount for calculating the attention weight, is a coefficient updated by the training process.

[0064] The application further discloses an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method when executing the computer program.

[0065] The application further discloses a storage medium having a computer program stored thereon, and the computer program implements the steps of the method when executed by a processor.

[0066] The application constructs multiple simple feature generator combinations into a multi-dimensional feature generator through an original sequence, implements a data generation idea of inputting a time step number and outputting a corresponding time step multi-dimensional feature sequence, thereby avoiding directly inputting a monitoring sequence into a prediction model, and adopting a multi-dimensional feature sequence of the monitoring sequence as input data for prediction. For spatial sequence data, a CNN module is added in the prediction model to construct a spatial dimension feature extractor. A single model one-time prediction output sequence is simplified to a single model multiple-time prediction combination result of each time step to form a sequence.

[0067] Embodiment Step 1, constructing a multi-dimensional feature generator and a deep learning network model structure based on fusion of CNN and RNN.

[0068] The specific training strategy of the deep learning network model structure based on fusion of CNN and RNN is as follows: (1) encapsulating temperature feature sequence samples and target sequence samples by using TensorDataset, and realizing batch data iteration by using DataLoader. The batch size is set to 64 samples / batch, and a random shuffling (shuffle=True) mechanism is activated to ensure randomness of sample distribution in the training process, thereby effectively avoiding overfitting of the model to the input order. (2) adopting a mean square error loss function (MSE) as an optimization target. An adaptive moment estimation (Adam) is selected as an optimizer, and the initial learning rate is set to (3) Gradient clipping technique (torch.nn.utils.clip_grad_norm_) was implemented to set the parameter gradient norm threshold to 0.5, which effectively suppressed the gradient explosion phenomenon. (4) Before training, the parameters of the convolutional layer and the fully connected layer were initialized using Xavier, which controlled the initial variance of the weight matrix to ensure the stability of the signal and gradient in the forward propagation and back propagation process of the neural network, thereby alleviating the common problems of gradient vanishing or gradient explosion in deep model training.

[0069] Figure 3 The model optimization process is shown. When the training round is 0-200, the optimization speed of the model performance is very fast; when the training round is 200-375, the optimization speed of the model performance gradually slows down, but the model performance is already quite good; from the 375th training round, the loss value of the model training process starts to converge, and finally tends to 0.

[0070] Step 2: Spatial sequence as input, multi-dimensional feature generator generates spatial multi-dimensional feature sequence according to time step.

[0071] With a certain temperature sequence as input, the periodic feature generator uses a 5-parameter Fourier function as the periodic feature generator, and the Fourier function value output by the input time step number as the periodic feature of the corresponding time step. The fitting result is shown in Table 2 and Figure 4 Table 2 Fourier function parameter fitting results

[0072] Wave feature generator First step: Spatial sequence deperiodization, wave sequence is obtained by monitoring spatial sequence minus periodic feature, the value of wave sequence fluctuates around 0, as shown by the orange dashed line in Figure 5 Second step: Frequency histogram statistics of wave sequence, as shown in Figure 6 Third step: Sampling according to the probability distribution represented by the frequency histogram statistics, the number of temperature points sampled each time can be determined according to the average time step length of adjacent wave peaks and wave troughs. In this case, it is assumed that the average time step length is 1 day, corresponding to 48 time steps. Each time 48 temperatures are sampled, and then the temperatures are reordered. The number of sampling times = total time step length / average time step length. Part of the sampling results are shown in Figure 7 Figure 7 ​​​​The middle red dotted line is the generated fluctuation feature, and its trend is basically consistent with the fluctuation trend of the monitoring data.

[0073] Trend feature generator First step: take the month as the observation time span, and count the mean, minimum, 25% quantile, 50% quantile, 75% quantile, and maximum of all monitoring values in the month. All these values of all months form a monthly sequence.

[0074] Second step: perform a function regression fitting on the monthly sequence of each feature value, input the monthly sequence number, and output the trend feature of the corresponding month. The fitting results are shown in Table 3: Table 3 Temperature trend feature fitting results

[0075] Third step: assuming that the trend features of different time steps in the same month are equal to the monthly trend feature, generate all time step trend features according to the monthly trend feature as shown in Figure 9

[0076] Fourth step: construct a spatial multi-dimensional feature sequence with size (m, n, f) = (m, 10, 8), where the symbols are consistent with the previous text, m, n, and f are the total number of time steps, the number of measurement points, and the number of feature components of the monitoring value, respectively.

[0077] Step 3, the spatial multi-dimensional feature sequence passes through a sliding window and a data iterator to batch generate a spatial multi-dimensional feature sample sequence.

[0078] Set the sliding window k = 10, operate in the dimension represented by m, and divide the spatial sequence data into m-k small samples. Each small sample is a small spatial sample sequence, forming a data set with shape (m-k, 10, 10, 8).

[0079] Construct a data iterator to input batch data sets to the model. The shape of this batch data set is (batchsize, 10, 10, 8).

[0080] Step 4, take the spatial multi-dimensional feature sample sequence as input, and obtain the hidden space feature sequence of the spatial multi-dimensional feature sample sequence through the CNN-based spatial dimension feature extractor.

[0081] Step 4.1, construct this spatial dimension feature extractor according to the case parameters mentioned in the previous text (Table 1 Spatial Dimension Feature Extractor Layer Parameter Settings); The input data shape is (batchsize, 10, 10, 8), and the output data shape is (batchsize, 10, 1, 8).

[0082] ​The data dimension reduction obtains data with a shape of (batchsize, 10, 8).

[0083] In step 4.2, according to k=10, a recurrent neural network shown in programming implementation 2 is implemented, and in this embodiment, the number of layers of the recurrent neural network is 4, and the number of hidden neurons is 64.

[0084] The input data has a shape of (batchsize, 10, 8), and the output data has a shape of (batchsize, n)=(batchsize, 10).

[0085] The dimension represented by the output data n has a total of 10 values, which respectively represent the predicted values of the 10 measuring points.

[0086] In step 5, the hidden space feature sequence is taken as input, and the time dependence of the hidden space feature sequence is obtained through the RNN-based time dimension feature extractor. In step 6, the last layer of the RNN-based time dimension feature extractor combines the adaptive attention mechanism to generate a time information vector as a prediction result, and the result is as shown in Figure 9 、 10 .

[0087] The long-term prediction method for spatial sequence data based on the fusion of CNN and RNN firstly constructs a multi-dimensional feature generator, directly outputs multi-dimensional sequence features through input time step numbers, replaces the original sequence with the multi-dimensional sequence features, and makes the data input into the deep learning network model have independence; meanwhile, the spatial dimension feature extractor in the network can extract hidden spatial information of the input data; the hidden spatial information sequence is input into the time dimension feature extractor, and the time dependence is also extracted, based on which the next time sequence can be effectively predicted, and the results of multiple predictions are combined to be the results of multi-step prediction, so that long-term prediction of sequence data can be realized. In the test stage, the multi-dimensional feature generator is input with the prediction time step sequence (assuming that the length of the historical sequence is L, the prediction time step sequence is L+1, L+2, …), and the corresponding spatial multi-dimensional feature sequence is output, and then the spatial multi-dimensional feature sequence is input into the deep learning network, so that prediction of any sequence length can be realized.

[0088] The above only describes the preferred embodiments of the present application, and does not use any limitation on the technical solutions of the present application, and those skilled in the art should understand that the technical solutions can be modified and replaced in several simple ways without departing from the spirit and principles of the present application, and these modifications and replacements also belong to the protection scope covered by the claims.

Claims

1. A long-term prediction method for spatial sequence data based on a fusion of CNN and RNN, characterized in that, include: Step 1: Construct a multi-dimensional feature generator and a deep learning network model structure based on the fusion of CNN and RNN. The multi-dimensional feature generator is constructed based on the feature components of the input sequence. The deep learning network model structure based on the fusion of CNN and RNN includes a spatial dimension feature extractor based on CNN and a temporal dimension feature extractor based on RNN. Step 2: The multidimensional feature generator generates a spatial multidimensional feature sequence based on the time step of the input sequence; Step 3: Spatial multidimensional feature sequences are batch-generated into spatial multidimensional feature sample sequences through a sliding window and a data iterator; Step 4: Using the spatial multidimensional feature sample sequence as input, obtain the hidden spatial feature sequence of the spatial multidimensional feature sample sequence through a CNN-based spatial dimension feature extractor. Step 5: Using the hidden space feature sequence as input, obtain the temporal dependency of the hidden space feature sequence through an RNN-based temporal dimension feature extractor; Step 6: The last layer of the RNN-based temporal feature extractor combines an adaptive attention mechanism to generate a temporal information vector as the prediction result.

2. The long-term prediction method for spatial sequence data based on the fusion of CNN and RNN as described in claim 1, characterized in that, The loss function for deep learning network models based on the fusion of CNN and RNN is L2, and the optimization algorithm is Adam or SGD.

3. The long-term prediction method for spatial sequence data based on the fusion of CNN and RNN as described in claim 1, characterized in that, The temperature sequence is used as the input sequence. The feature components of the temperature sequence include periodic features, fluctuation features and trend features. The multidimensional feature generator includes a periodic feature generator, a fluctuation feature generator and a trend feature generator.

4. The long-term prediction method for spatial sequence data based on the fusion of CNN and RNN as described in claim 3, characterized in that, In step 2, using the temperature sequence as the input sequence, the multidimensional feature generator generates a spatial multidimensional feature sequence based on the time step, including: The periodic feature generator generates periodic feature values ​​by fitting a multi-parameter Fourier function; The volatility feature generator uses a combination of statistical and sampling methods to obtain volatility feature sequences. The trend feature generator performs a first polynomial fit on the feature values ​​of the spatial sequence on a monthly scale to generate monthly trend features, and generates trend features for all time steps based on the monthly trend features. Periodic feature values, fluctuation feature sequences, and trend features are concatenated to obtain 3D spatial multidimensional features (m, n, f), where m, n, and f are the total number of time steps, the number of measurement points, and the number of feature components of the monitored values, respectively.

5. The long-term prediction method for spatial sequence data based on the fusion of CNN and RNN as described in claim 3, characterized in that, The process of generating the wave characteristic sequence is as follows: By removing periodic features from the temperature sequence, a fluctuation sequence that fluctuates around 0 is obtained. Calculate the time distance between adjacent peaks and troughs in a statistical fluctuation sequence, and then calculate the average distance between adjacent peaks and troughs. Analyze the frequency histogram of the fluctuation series, and then sample; The sampling structure is rearranged in descending order; Multiple samplings are performed, and the sampling results are spliced ​​together in the sampling order to obtain the fluctuation characteristic sequence.

6. The long-term prediction method for spatial sequence data based on the fusion of CNN and RNN as described in claim 3, characterized in that, In step 3, the operation is performed on the dimension represented by m, dividing the data of the spatial multidimensional feature sequence into mk small samples. Each small sample is a small spatial sample sequence, forming a 4D form (mk, k, n, f).

7. The long-term prediction method for spatial sequence data based on the fusion of CNN and RNN as described in claim 1, characterized in that, In step 5, the data size (batchsize, k, 1, f) of the hidden space feature sequence is converted to (batchsize, k, f) through dimensionality reduction and transpose, and then input into the RNN-based time dimension feature extractor. Here, f represents the number of feature components that a single measurement point sequence can contain, and batchsize is the amount of data in a batch during training.

8. The long-term prediction method for spatial sequence data based on the fusion of CNN and RNN as described in claim 1, characterized in that, In step 6, the last layer of the RNN-based temporal feature extractor combines an adaptive attention mechanism to generate a temporal information vector as the prediction result. The calculation formula is as follows: in, Output the attention weights of the hidden state at the j-th time step of the l-th layer. To calculate the transition amount of attention weights, These are coefficients that are updated automatically during the training process.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

10. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

Images