Data processing method, device and electronic equipment

Abstract

The present invention provides a data processing method, device and electronic device, the method comprising: obtaining first time series data corresponding to a first time series, the first time series comprising N time nodes, the first time series data comprising N time nodes and M indicator data values ​​corresponding to M time nodes among the N time nodes; M being a positive integer less than N; generating a first image based on the first time series data, wherein the first image is used to represent the N time nodes, the M indicator data values ​​and the corresponding relationship between the M indicator data values ​​and the M time nodes; performing denoising processing on the first image using a denoising autoencoder to obtain a second image; generating second time series data based on the second image, the second time series data comprising N indicator data values ​​corresponding to the N time nodes; generating target time series data based on the second time series data and a time series prediction model. The embodiment of the present invention can improve the accuracy of time series prediction.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of computer technology, and in particular to a data processing method, device and electronic device. Background Art

[0002] Time series data forecasting is to conduct statistical analysis on past time series data to infer the development trend in the future. It aims to understand the development trend of things and add certain interventions to regulate according to the predicted trend to achieve the expected value of the development of things.

[0003] Current time series forecasting methods often use traditional statistical learning methods and machine learning algorithms to complete time series forecasting. Common algorithms include ARIMA, Xgboost, and LSTM. In the forecasting process, missing values ​​and outliers have a great impact on the forecast results. Most current methods use simple operations such as deletion and averaging to handle missing values ​​and outliers, which cannot provide effective and accurate missing data, resulting in low accuracy of time series forecasting. Summary of the invention

[0004] Embodiments of the present invention provide a data processing method, device and electronic device to improve the accuracy of time series prediction.

[0005] To solve the above problems, the present invention is achieved as follows:

[0006] In a first aspect, an embodiment of the present invention provides a data processing method, the method comprising: obtaining first time series data corresponding to a first time series, the first time series comprising N time nodes, the first time series data comprising the N time nodes and M indicator data values ​​corresponding to M time nodes among the N time nodes; N is a positive integer, and M is a positive integer less than N;

[0007] Generate a first image based on the first time series data, wherein the first image is used to represent the N time nodes, the M indicator data values, and the corresponding relationship between the M indicator data values ​​and the M time nodes;

[0008] De-noising the first image using a denoising autoencoder DAE to obtain a second image;

[0009] Based on the second image, second time series data is generated, where the second time series data includes N indicator data values ​​corresponding to the N time nodes, and the N indicator data values ​​include the M indicator data values;

[0010] Based on the second time series data and the time series prediction model, target time series data is generated.

[0011] In a second aspect, an embodiment of the present invention further provides a data processing device, including:

[0012] A first acquisition module is used to acquire first time series data corresponding to a first time series, wherein the first time series includes N time nodes, and the first time series data includes the N time nodes and M indicator data values ​​corresponding to M time nodes among the N time nodes; N is a positive integer, and M is a positive integer less than N;

[0013] A first generating module, configured to generate a first image based on the first time series data, wherein the first image is used to represent the N time nodes, the M indicator data values, and the corresponding relationship between the M indicator data values ​​and the M time nodes;

[0014] A denoising module, configured to perform denoising processing on the first image using a denoising autoencoder DAE to obtain a second image;

[0015] A second generating module, configured to generate second time series data based on the second image, wherein the second time series data includes N indicator data values ​​corresponding to the N time nodes, and the N indicator data values ​​include the M indicator data values;

[0016] The third generation module is used to generate target time series data based on the second time series data and the time series prediction model.

[0017] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising: a memory, a processor, and a program stored in the memory and executable on the processor; characterized in that the processor is used to read the program in the memory to implement the steps in the method described in the first aspect above.

[0018] In a fourth aspect, an embodiment of the present invention further provides a readable storage medium for storing a program, wherein the program, when executed by a processor, implements the steps in the method described in the first aspect.

[0019] In an embodiment of the present invention, the first time series data to be processed is converted into a first image, and then the first image is processed by a denoising autoencoder to obtain a second image, and then the second image is inversely converted into second time series data, so that the missing indicator data values ​​of NM time nodes in the first time series data are filled in by image processing, so that the prediction result based on the second time series data and the time series prediction model is more accurate, that is, the accuracy of the time series prediction is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0021] Figure 1 is a flow chart of a data processing method provided by an embodiment of the present invention;

[0022] Figure 2 It is a schematic diagram of a flow chart of a generative adversarial network execution provided by an embodiment of the present invention;

[0023] Figure 3 is a schematic diagram of a flow chart of an image completion layer execution provided by an embodiment of the present invention;

[0024] Figure 4 is a possible system architecture diagram provided by an embodiment of the present invention;

[0025] Figure 5 is a structural diagram of a data processing device provided by the present invention;

[0026] Figure 6 It is a structural diagram of an electronic device provided by the implementation of the present invention. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] The terms "first", "second" etc. in the embodiments of the present invention are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. In addition, the terms "include" and "have" and any variation thereof are intended to cover non-exclusive inclusions, for example, the process, method, system, product or equipment comprising a series of steps or units need not be limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment. In addition, "and / or" is used in the present application to represent at least one of the connected objects, such as A and / or B and / or C, which represents 7 situations including single A, single B, single C, and A and B all exist, B and C all exist, A and C all exist, and A, B and C all exist.

[0029] See also Figure 1 , Figure 1 is a flow chart of a data processing method provided by an embodiment of the present invention, such as Figure 1 As shown, the above method includes:

[0030] Step 101: Obtain first time series data corresponding to a first time series, where the first time series includes N time nodes, and the first time series data includes the N time nodes and M indicator data values ​​corresponding to M time nodes among the N time nodes; N is a positive integer, and M is a positive integer less than N.

[0031] For time series prediction, it is usually necessary to predict time series data in the future through historical time series data corresponding to the historical time series. Therefore, the first time series can be a continuous time period in the past for the current time point, and the time period can include N time nodes set at intervals. The time nodes can be set according to certain rules, such as minutes, hours, days, weeks, months or years, which are not limited here.

[0032] It should be noted that due to external factors or human factors, data at certain time points in the historical time series data may be missing. At this time, the above data processing method can be used to first complete the data at the above time points, and then make predictions based on the completed time series data.

[0033] In such a case, the first time series data may include the N time nodes and M indicator data values ​​corresponding to M time nodes among the N time nodes.

[0034] Taking urban residents' electricity consumption as an example, if you need to predict the daily urban residents' electricity consumption data in the next five days, you can first obtain the daily urban residents' electricity consumption data in the past 100 days. The past 100 days are the first time series, and each day is a time node, and N is equal to 100. If there are missing electricity consumption data for 10 days in the past 100 days, M is equal to 90. The dates of the 100 days and the urban residents' electricity consumption data corresponding to 90 of them can be included in the above first time series data.

[0035] In an embodiment of the present invention, the first time series data may include an indicator data value corresponding to a time node. In this case, if multiple indicator data values ​​need to be predicted, different time series data corresponding to different indicator data values ​​may be generated according to the first time series. Of course, in other optional embodiments, the first time series data may also include multiple indicator data values ​​corresponding to a time.

[0036] It should be noted that for each time node, feature data of each time node may also be included, and the feature data corresponds one-to-one with the first time series data, so that the electronic device can generate a more accurate prediction result based on the features of the time node. The feature data of each time node may indicate the features that affect the index data value at the time node, such as whether it is a working day, whether it is a rainy day, or whether it is winter.

[0037] Continuing with the above example, illustratively, since factors such as weather conditions, seasonal conditions, and working day conditions will affect the electricity consumption of urban residents, the above first time series data may also include daily weather conditions, seasonal conditions, and working day conditions, etc., and may be stored in the form of digital identifiers "0" and "1", etc., and may establish a dependency relationship with the daily urban residents' electricity consumption data, so as to facilitate subsequent prediction using the time series prediction model.

[0038] Step 102: Generate a first image based on the first time series data, wherein the first image is used to represent the N time nodes, the M indicator data values, and the corresponding relationship between the M indicator data values ​​and the M time nodes.

[0039] In the above step 102, the above first image can be transformed from the above first time series data. Specifically, the above first time series data can be presented in the form of an image by using coordinate transformation and trigonometric function transformation, wherein the index data value corresponding to each time node has a corresponding image block in the first image, so that after the image is processed and changed, the corresponding changed index data value and the time node corresponding to the index data value can be obtained by reverse transformation. For example, the above first image can be specifically a Gramian Angular Field (GAF) diagram, etc. When performing image conversion processing, since the index data values ​​of NM time nodes in the above first time series data are missing, the index data values ​​of the above NM time nodes in the above first time series data can be set to preset values ​​first, and then the N time nodes and the corresponding N index data values ​​in the first time series data can be converted into the first image.

[0040] It should be understood that in some embodiments, the electronic device may also pre-process the above-mentioned first time series data, such as correcting obviously erroneous data or modifying data in a special period, and then generate the above-mentioned first image based on the pre-processed time series data.

[0041] Step 103: Use a denoising autoencoder DAE to denoise the first image to obtain a second image.

[0042] In the above step 103, the denoising auto encoder (DAE) is a type of auto encoder that accepts damaged data as input and is trained to predict the original undamaged data as input. It can be further known from the above content that for the above first time series data, there are no corresponding indicator data values ​​for the above NM time nodes. Therefore, the step of setting the indicator data values ​​corresponding to the NM time nodes in the above first time series data to preset values ​​can be regarded as adding noise to the first image. The noise in the above first image can be removed by the denoising auto encoder to obtain a denoised second image.

[0043] Step 104: Generate second time series data based on the second image, where the second time series data includes N indicator data values ​​corresponding to the N time nodes, and the N indicator data values ​​include the M indicator data values.

[0044] In the above step 104, it can be seen from the above content that the time nodes and corresponding indicator data values ​​in the above first time series data correspond one-to-one to the image blocks in the first image. Since the above second image is de-noised by the denoising autoencoder, some image blocks in the above first image will change, so when the above second image is converted back to time series data to obtain the above second time series data, the indicator data values ​​corresponding to the above NM time nodes of the above second time series data will change from the preset value to the interpolated value.

[0045] In this way, the missing indicator data values ​​of NM time nodes in the first time series data can be filled by image processing. In other words, in the second time series data, the N time nodes all have corresponding indicator data values.

[0046] Step 105: Generate target time series data based on the second time series data and the time series prediction model.

[0047] In the above step 105, after the first time series data is processed by image processing to obtain the above second time series data, the above second time series data can be input into a time series prediction model to perform time series prediction to obtain target time series data.

[0048] Continuing with the previous example, after obtaining the urban residents' electricity consumption data for the past 100 days, the urban residents' electricity consumption data for the next 5 days can be predicted based on the urban residents' electricity consumption data for the past 100 days. Since the urban residents' electricity consumption data for the past 100 days have been processed and filled with the urban residents' electricity consumption index data values ​​corresponding to 10 days, the prediction results obtained by time series prediction based on this data are more accurate.

[0049] The time series prediction model can be a neural network model, specifically a convolutional neural network (CNN), a recurrent neural network (RNN), or a convolutional recurrent neural network (CRNN). The convolutional recurrent neural network is a combination of a convolutional neural network and a recurrent neural network. Since the convolutional neural network can classify and predict images, and the recurrent neural network can capture the dependency of the sequence, the embodiment of the present invention can use the convolutional neural network to first extract the feature information of each related multiple time series, and then connect the output of the convolutional neural network to the recurrent neural network to form a convolutional recurrent neural network.

[0050] In an embodiment of the present invention, the first time series data to be processed is converted into a first image, and then the first image is processed by a denoising autoencoder to obtain a second image, and then the second image is inversely converted into second time series data, so that the missing indicator data values ​​of NM time nodes in the first time series data are filled in by image processing, so that the prediction result based on the second time series data and the time series prediction model is more accurate, that is, the accuracy of the time series prediction is improved.

[0051] Optionally, the step of generating a first image based on the first time series data includes:

[0052] Normalizing the M indicator data values ​​corresponding to the M time nodes;

[0053] Setting the indicator data values ​​corresponding to NM time nodes other than the M time nodes among the N time nodes to the first preset value;

[0054] Convert the N time nodes and the N indicator data values ​​corresponding to the N time nodes into a polar coordinate system;

[0055] Based on the trigonometric function transformation of the polar coordinate system coordinates, the first image is generated, and the first image is a Gram angular field (GAF) diagram.

[0056] In an embodiment of the present invention, in order to facilitate subsequent image conversion, the above-mentioned first time series data can be normalized first, and the M indicator data values ​​corresponding to the above-mentioned M time nodes can be scaled to the interval range of [-1,1] or [0,1]. At this time, the above-mentioned first time series data is presented in a Cartesian coordinate system.

[0057] Furthermore, in order to improve the denoising effect, the data uniform values ​​of the NM time nodes whose corresponding indicator data values ​​are missing among the N time nodes may be set to a first preset value, and the first preset value is within the interval of [-1, 1] or [0, 1], and may specifically be 0. The NM time nodes and the NM first preset values, when converted to the first image, are the noise of the first image.

[0058] After that, since the coordinates in the polar coordinate system are only related to the angle, the polar coordinate system can retain the absolute time relationship relative to the Cartesian coordinate system. The embodiment of the present invention can convert the horizontal and vertical coordinates corresponding to the time nodes and the index data values ​​in the Cartesian coordinate system to the polar coordinate system, and generate the above-mentioned first image through the coordinate trigonometric transformation in the polar coordinate system. Specifically, the Gramian Angular Summation Field (GASF) diagram can be obtained through the cosine function of the sum of the two angles, or the Gramian Angular Difference Field (GADF) diagram can be obtained through the cosine function of the difference of the two angles. The specific process can be implemented by the encapsulated software tool, which will not be repeated here.

[0059] The embodiment of the present invention converts the above-mentioned first time series data from a Cartesian coordinate system to a polar coordinate system and then to a Gram angular field diagram through coordinate conversion, so that the image can be processed using currently mature image processing technology, and finally the time series data filled with indicator data values ​​is obtained through inverse transformation.

[0060] Optionally, before step 103, the method may further include:

[0061] Acquire at least one training time series data from the historical time series data;

[0062] Setting the indicator data value corresponding to at least one time node in each of the training time series data to a second preset value to obtain at least one third time series data;

[0063] The denoising autoencoder is trained based on the at least one training time series data and the at least one third time series data.

[0064] In an embodiment of the present invention, the above-mentioned historical time series data may be the training data of the above-mentioned denoising autoencoder. As training data, in order to improve the training effect, the data volume is usually large. For example, if the above-mentioned first time series data is the data of the past 60 days, the above-mentioned historical time series data may be the data of the past two years. Generally speaking, the above-mentioned historical time series data is independent of the above-mentioned first time series data, that is, the above-mentioned historical time series data does not overlap with the above-mentioned first time series data. The electronic device may obtain at least one training time series data from the above-mentioned historical time series data by means of cross-validation, and each training time series data may be used as a group of training samples. It should be understood that the above-mentioned training time series data may be selected from the historical time series data, and the time series data with no missing indicator data values, and the time series length corresponding to each training time series data may be the same or different, and training may be performed by using multiple groups of low-noise training samples, so as to improve the training effect.

[0065] The electronic device can set the index data value corresponding to at least one time node in each group of the above training time series data to a second preset value. The above second preset value is similar to the above first preset value and is within the above interval of [-1,1] or [0,1], and can specifically be 0. By setting at least one time node in the above training time series data to the second preset value, at least one third time series data is obtained, so that the situation where the index data value is missing in the time series data can be simulated. The time series data generated after denoising by the denoising autoencoder can be compared with the training time series data, and an error function can be established to gradually optimize the denoising autoencoder, so that the error of the denoising autoencoder is reduced, thereby realizing the training of the denoising autoencoder.

[0066] Furthermore, the step of training the denoising autoencoder based on the at least one training time series data and the at least one third time series data includes:

[0067] Based on the at least one training time series data, correspondingly generating at least one third image;

[0068] Based on the at least one third time series data, correspondingly generating at least one fourth image;

[0069] Performing denoising processing on the at least one fourth image using the denoising autoencoder to obtain at least one fifth image;

[0070] Generate a loss function based on a mean square error between the third image and the fifth image in one-to-one correspondence;

[0071] Based on the loss function, the denoising autoencoder is trained.

[0072] In an embodiment of the present invention, in the above-mentioned training time series data, the indicator data values ​​corresponding to each time node are all real values. Therefore, based on the above-mentioned at least one training time series data, the corresponding at least one third image generated can be understood as an image generated by real data. The purpose of the above-mentioned denoising autoencoder is to denoise the fourth image generated by the above-mentioned third time series data, so as to generate a fifth image that is closest to the above-mentioned third image.

[0073] Therefore, the mean square error between the fifth image and the third image corresponding to can be calculated, and the denoising autoencoder can be optimized through the loss function, thereby realizing the training of the denoising autoencoder.

[0074] It should be understood that since the third time series data is obtained by processing the training time series data, the one-to-one corresponding third image and fifth image are respectively generated by the training time series data before processing and the third time series data after processing.

[0075] Optionally, the denoising autoencoder may adopt a Sigmoid function, which may help to learn the nonlinear characteristics of the hidden layer.

[0076] Optionally, the above step 102 may specifically include:

[0077] Inputting the first time series data into a generative adversarial network GAN to obtain fourth time series data;

[0078] Setting the indicator data value corresponding to at least one time node in the fourth time series data to a third preset value to obtain fifth time series data;

[0079] Based on the fifth time series data, the first image is generated.

[0080] In an embodiment of the present invention, a generative adversarial network (GAN) may be used to preprocess the first time series data to obtain fourth time series data, and then generate the first image based on the fourth time series data.

[0081] Specifically, refer to Figure 2 The generative adversarial network is a generative model that can generate new samples that obey the distribution of the original data set. It mainly consists of a generator and a discriminator. It can output complete time series data, that is, the fourth time series data, by inputting the first time series data into the generative adversarial network GAN. At this time, the missing indicator data values ​​in the first time series data are initially filled by the generative adversarial network GAN.

[0082] Furthermore, the indicator data value corresponding to at least one time node of the fourth time series data may be set to a preset value to obtain fifth time series data, and then the first image may be generated based on the fifth time series data to perform the above denoising process.

[0083] Preferably, the indicator data value corresponding to at least one of the M time nodes in the fourth time series data that previously lacked the indicator data value can be set to the third preset value to obtain the fifth time series data, and the first image is generated based on the fifth time series data, so that in the subsequent process of processing the first image using the denoising autoencoder, the processing process can be accelerated due to the reduction of the time nodes lacking the indicator data value. At the same time, the indicator data value initially filled by the GAN can be optimized, and a more accurate indicator data value can be obtained by combining the GAN algorithm and the image denoising processing scheme.

[0084] The third preset value is similar to the first preset value and the second preset value, and will not be described in detail herein.

[0085] For example, if the above-mentioned first time series includes 100 time nodes, 90 time nodes have corresponding indicator data values, and 10 time nodes have missing indicator data values. In an embodiment of the present invention, the time series data corresponding to the above-mentioned first time series can be first input into the above-mentioned generative adversarial network for preliminary data filling to obtain fourth time series data, wherein all 100 time nodes in the fourth time series data have corresponding indicator data values. Then, one of the 10 time nodes that were previously missing indicator data values ​​can be selected, and its corresponding indicator data value can be set to a third preset value to obtain the fifth time series data. At this time, the noise of the first image generated based on the fifth time series data is relatively small, and the denoising process of the denoising autoencoder can obtain a better denoising effect.

[0086] It should be noted that the above-mentioned step of setting the indicator data value corresponding to at least one time node of the above-mentioned fourth time series data to a preset value can be performed multiple times to obtain multiple fifth time series data, wherein in each fifth time series data, the time node at which the corresponding indicator data value is set to the third preset value is not exactly the same.

[0087] Continuing with the above example, the indicator data values ​​corresponding to the 10 time nodes where the indicator data values ​​were previously missing in the above-mentioned fourth time series data can be set to the third preset values ​​in sequence to generate 10 fifth time series data, and then processed by subsequent denoising autoencoder and the like to obtain 10 time series data, so that the indicator data values ​​corresponding to the time nodes where the indicator data values ​​were previously set to the third preset values ​​can be found from the 10 time series data, and then the second time series data can be generated according to the indicator data values ​​corresponding to these 10 time nodes.

[0088] In this way, the embodiment of the present invention processes the first time series data with missing indicator data values ​​multiple times, so that the filled indicator data values ​​in the second time series data are more real and reliable, thereby further improving the accuracy of the target time series data generated based on the prediction of the second time series data.

[0089] Optionally, the above step 105 may specifically include:

[0090] Inputting the second time series data into a generative adversarial network GAN to obtain sixth time series data;

[0091] The sixth time series data is used as input of the time series prediction model to obtain the target time series data.

[0092] In the embodiment of the present invention, there may still be some noise in the second image denoised by the denoising autoencoder. Therefore, there may be a small number of index data values ​​corresponding to time nodes in the second time series data that are the first preset values, which affects the accuracy of the final target time series data. Therefore, the second time series data can be input into the generative adversarial network to obtain the sixth time series data, and the sixth time series data can be used as the input of the time series prediction model to obtain the target time series data.

[0093] In this way, the embodiment of the present invention further optimizes the second time series data through a generative adversarial network, so that the indicator data value corresponding to each time node in the second time series data is more real and reliable, thereby improving the accuracy of the predicted target time series data.

[0094] Optionally, the above step 102 may specifically include:

[0095] Get the first time node t in the first time series data and the indicator data value x corresponding to the first time node t ', the first time node is the tth time node in the first time series;

[0096] According to the first time node t, the first time series is divided into preset periods T, wherein one preset period T is a time period formed by L adjacent time nodes, and before the period where the first time node t is located, the first time series further includes n1 periods, and after the period where the first time node t is located, the first time series further includes n2 periods;

[0097] In each cycle, a time period formed by the first c time nodes of the first time node is obtained to obtain a forward time window, where c is a preset constant;

[0098] In each cycle, a time period formed by the c time nodes after the first time node is obtained to obtain a backward time window;

[0099] pass k=1,2,…c, get the forward window coefficient α k ;λ is a preset constant;

[0100] pass k=1,2,…c, get the backward window coefficient β k ;

[0101] pass Correct the index data value corresponding to the first time node in the first time series data to x t , the sixth time series data is obtained; δ is a preset constant;

[0102] Based on the sixth time series data, the first image is generated.

[0103] In the embodiment of the present invention, since the indicator data value corresponding to the time node in the first time series data may be obviously wrong, or the time node is relatively special, resulting in the corresponding indicator data value having no reference value, the data value x corresponding to the first time node t in the first time series data may be t 'Use an algorithm to perform correction. The first time node may be one or more, and may be a time node in the first time series data at which the corresponding indicator data value exceeds a preset range.

[0104] For example, if the first time series data includes the dates of the past 100 days and the urban residents' electricity consumption corresponding to each date, if a large-scale power outage occurs in the city on one of the days, the indicator data value corresponding to the time node is lower than the preset threshold, which can be determined as the first time node. In this case, the preset period can be divided according to the first time node. For example, in some embodiments, one week, that is, seven days, can be a period. A total of 14 periods can be divided into 100 days. If the first time node is in the 7th week, then n1=6, n2=7.

[0105] Furthermore, in the cycle where the first time node is located, a time period consisting of c adjacent and continuous time nodes can be selected forward as a forward time window, and a time period consisting of c adjacent and continuous time nodes can be selected backward as a backward time window. Continuing with the above example, if the above-mentioned first time node is Thursday, the preset cycle can be Monday to Sunday as a cycle, then the above-mentioned forward time window can be Monday to Wednesday in each cycle, and the backward time window can be Friday to Sunday in each cycle. If the above-mentioned first time node is Wednesday, the preset cycle can be Sunday to Saturday as a cycle, then the above-mentioned forward time window can be Sunday to Tuesday in each cycle, and the backward time window can be Thursday to Saturday in each cycle. According to the above-mentioned first time node, the cycle division is specifically set, so that the forward and backward time windows containing enough time nodes can be obtained within the preset cycle, thereby improving the reliability of the correction value finally generated.

[0106] Furthermore, according to the indicator data values ​​corresponding to each time node in the forward time window in each cycle, k=1,2,…c, determine the forward window coefficient, which is the time window attenuation factor, indicating that the longer the time from the first time node, the smaller the impact of the index data value corresponding to the time node on the forward window coefficient. Since the index data value corresponding to the time node far away from the first time node is usually not closely related to the index data value corresponding to the first time node, the reliability of the final correction can be improved by setting the time window attenuation factor. Among them, λ is a preset value, which can be set according to actual needs and is not limited here.

[0107] Similarly, one can k=1,2,…c, determines the backward window coefficient, which will not be described here.

[0108] Furthermore, according to Calculate the correction value x corresponding to the first time node above t , and obtain the sixth time series data. Among them, is a periodic attenuation factor, indicating that the longer the time from the first time node, the smaller the impact of the indicator data value corresponding to the time node on the final correction value. δ is a preset value, which can be set according to actual needs and is not limited here.

[0109] In the embodiment of the present invention, the forward and backward window coefficients can be obtained according to the first time node and the indicator data values ​​corresponding to each time node, and then the indicator data value corresponding to the first time node in the first time series data is corrected to x according to the forward and backward window coefficients and the indicator data values ​​corresponding to each time node. t, thereby achieving the correction of obviously erroneous data values ​​or data values ​​with little reference value in the above-mentioned first time series data, and improving the accuracy of the target time series data generated by the final prediction.

[0110] At the same time, by setting the time window attenuation factor and the period attenuation factor, the data values ​​corresponding to the time nodes far away from the first time node can be reduced, and the final correction value x t The influence of t reliability.

[0111] Of course, in other optional embodiments, the indicator data value corresponding to the first time node may be regarded as a missing value, and the data may be filled using image processing.

[0112] The various optional implementation modes introduced in the embodiments of the present invention may be implemented in combination with each other or may be implemented separately if they do not conflict with each other, and the embodiments of the present invention are not limited to this.

[0113] Reference Figure 4 , Figure 4 This is a possible architecture diagram of an embodiment of the present invention, which may include a data access layer, a data processing layer, an image completion layer and a timing prediction layer.

[0114] Based on this architecture, the data processing method provided by the embodiment of the present invention may include the following steps:

[0115] Step 201 executed by the data access layer takes historical time series data and feature data (the feature data may be empty) as input data.

[0116] Among them, historical time series data mainly refers to the historical data of the indicators to be predicted, including two columns of data: time and indicator value; there is also feature data related to the predicted indicators, including time and feature value data corresponding to each feature, and feature data refers to data that can obtain feature values ​​of future time periods in advance.

[0117] Step 202 executed by the data processing layer is to pre-process the input data, mainly including data interpolation and data correction.

[0118] Among them, data interpolation is to fill in the missing values ​​of time series data based on the generative adversarial network. Specifically, the generative adversarial network is a generative model that can generate new samples that obey the distribution of the original data set. It is mainly composed of a generator and a discriminator. The process of using the generative adversarial network for data interpolation is mainly:

[0119] (1) Input a random low-dimensional vector through the generator and output the complete time series data;

[0120] (2) The input of the discriminator is divided into two parts: one is the complete time series data generated by the generator, and the other is the real and accurate time series data in the original data set. The output is the probability value of the two parts of samples being "true", indicating the possibility that the discriminator input data is real time series data.

[0121] To handle missing time series data, both the generator and the discriminator consist of padded recurrent neural networks.

[0122] By training the generative adversarial network, for each missing time series data, we try to find an optimal input vector so that the time series data generated by the generator is most similar to the original time series data, thereby filling in the missing values ​​of the original time series data.

[0123] Data correction is to use algorithms to calculate and correct the fluctuation data and abnormal data on periodic special dates. Specifically, the correction method is as follows:

[0124] For example, the data at time t can be obtained by calculating the proportional relationship between the data of the first n1 and last n2 periods and the indicator data values ​​corresponding to the time nodes in the previous and next c time windows (the number of previous and next periods n1 and n2 needs to be determined based on the position of time t).

[0125] Calculate the forward window coefficient α based on the data in the forward window of the first n1 and the last n2 cycles k ; Calculate the backward window coefficient β based on the data in the backward window of the first n1 and the last n2 cycles k ,in

[0126] k=1,2,…c;

[0127] k=1,2,…c;

[0128]

[0129] Then according to the front and back window coefficients α k and β k , and the indicator data values ​​of the time nodes of the previous and next windows, calculate the value x at the current time t t ,in is the period attenuation factor of period i, is the time window attenuation factor of window c, which means that the earlier the period or window is, the smaller the influence coefficient is.

[0130] Reference Figure 5 , Figure 5 The process executed by the image completion layer may mainly include steps 203 to 205 .

[0131] Specifically, step 203 performed by the image completion layer converts the processed time series data into an image.

[0132] The main method is to use the Gramian Angular Field (GAF) to encode historical time series data into images, so that mature image processing technology can be used to interpolate time series to improve prediction results. The Gramian Angular Field converts the one-dimensional time series in the Cartesian coordinate system into a polar coordinate system, and then uses trigonometric functions to generate the GAF matrix.

[0133] Specifically, it may include the following steps:

[0134] Numerical scaling: scale the time series in the Cartesian coordinate system to the interval [0,1] or [-1,1];

[0135] Polar coordinate conversion: Use the coordinate transformation formula to convert the Cartesian coordinate system sequence into a polar coordinate system time series. As time increases, the corresponding values ​​​​are distorted between different corners of the span circle, just like water ripples. Given a time series, after polar coordinate system transformation, there is only one result and a unique inverse mapping. This property satisfies the decoding image prediction time series value and lays the foundation. The original time series in Cartesian coordinates can be restored through the main diagonal of the GAF matrix.

[0136] Trigonometric transformations of angle sums / differences: If the cosine function of the angle sum is used, GASF is obtained; if the cosine function of the angle difference is used, GADF is obtained.

[0137] Step 204 performed by the image completion layer is to perform denoising on the image.

[0138] The main process is to convert data into images and repair the missing noise of the image. The processing flow is as follows:

[0139] First, missing noise is manually added to the original time series (i.e., the values ​​of the missing points are randomly set to 0); then, the data is converted into GASF images; and then the denoising autoencoder is trained as a generative model to reconstruct the GASF image; the Sigmoid function can help learn the nonlinear characteristics of the hidden layer, and the mean square error (MSE) between the original GASF image and the "missing" GASF image is calculated as the loss function to evaluate the fitting performance.

[0140] Furthermore, the time series restored by the image denoising algorithm can be compared with the interpolation result in the above step 202. If the deviation is not large, it proves that the missing value filling effect is good. It can be used as the generative adversarial network model for filling the data optimization step 202, and looped until the model is stable. Combined with the image denoising strategy and the GAN generation algorithm, optimized interpolation data is obtained. The technical solution of combining the image denoising strategy and the GAN generation algorithm can refer to the description in the above embodiment and will not be repeated here.

[0141] Step 205 performed by the image completion layer converts the processed image back into temporal data.

[0142] Among them, the trained denoising autoencoder can be used to help restore the image, and the main diagonal lines can be extracted from the image to reconstruct the restored time series.

[0143] Step 206 performed by the time series prediction layer inputs the processed time series data into the time series prediction model to obtain a prediction result.

[0144] Among them, the time series prediction model can be a convolutional recurrent neural network, which uses a combination of convolutional neural networks and recurrent neural networks to predict one or more groups of related time series. The experience of convolutional neural networks being successfully used for image classification and prediction and the ability of recurrent neural networks to capture the dependencies of sequences can be used to predict future values. Therefore, the convolutional neural network is used to first extract the distinctive features of each of the multiple related time series, and then the top of the convolutional neural network combined output is connected to the recurrent neural network to form a convolutional recurrent neural network.

[0145] One or more time series are taken as input, where each input time series contains one prediction indicator, and distinct features are extracted through the convolution layer; the most representative features of the time series are captured through the pooling layer; after integrating the pooling layer, the multidimensional data set is connected into an n-dimensional vector and input into the recurrent neural network for prediction to obtain the prediction result.

[0146] Optionally, the embodiment of the present invention may further include a data application layer to depict and display the prediction results through a trend chart, understand the subsequent indicator change trend, and provide a basis for subsequent strategy research.

[0147] The time series data prediction method based on image interpolation in the embodiment of the present invention can use mature image processing technology to complete image data, thereby obtaining complete feature data and historical data, which helps to improve the accuracy of subsequent time series prediction.

[0148] At the same time, the embodiment of the present invention can correct abnormal data through a periodic data correction method, improve the data accuracy of the time series itself and the integrity of the feature data, reduce the impact of abnormal data on time series prediction, and improve the accuracy of time series prediction.

[0149] See also Figure 5 , Figure 5 FIG. 1 is one of the structural diagrams of the data processing device provided by the embodiment of the present invention. Figure 5 As shown, the data processing device 500 includes:

[0150] The first acquisition module 501 is used to acquire first time series data corresponding to a first time series, where the first time series includes N time nodes, and the first time series data includes the N time nodes and M indicator data values ​​corresponding to M time nodes among the N time nodes; N is a positive integer, and M is a positive integer less than N;

[0151] A first generating module 502 is used to generate a first image based on the first time series data, wherein the first image is used to represent the N time nodes, the M indicator data values, and the corresponding relationship between the M indicator data values ​​and the M time nodes;

[0152] A denoising module 503, configured to perform denoising processing on the first image using a denoising autoencoder DAE to obtain a second image;

[0153] A second generating module 504 is configured to generate second time series data based on the second image, where the second time series data includes N indicator data values ​​corresponding to the N time nodes, where the N indicator data values ​​include the M indicator data values;

[0154] The third generating module 505 is used to generate target time series data based on the second time series data and the time series prediction model.

[0155] Optionally, the first generating module 502 includes:

[0156] Normalizing the M indicator data values ​​corresponding to the M time nodes;

[0157] Setting the indicator data values ​​corresponding to NM time nodes other than the M time nodes among the N time nodes to the first preset value;

[0158] Convert the N time nodes and the N indicator data values ​​corresponding to the N time nodes into a polar coordinate system;

[0159] Based on the trigonometric function transformation of the polar coordinate system coordinates, the first image is generated, and the first image is a Gram angular field (GAF) diagram.

[0160] Optionally, the data processing device 500 further includes:

[0161] A second acquisition module is used to acquire at least one training time series data from the historical time series data;

[0162] A setting module, used to set the indicator data value corresponding to at least one time node in each of the training time series data to a second preset value, to obtain at least one third time series data;

[0163] A training module is used to train the denoising autoencoder based on the at least one training time series data and the at least one third time series data.

[0164] Optionally, the training module includes:

[0165] A first generating unit, configured to generate a third image based on the training time series data;

[0166] A second generating unit, configured to generate at least one fourth image correspondingly based on the at least one third time series data;

[0167] a denoising unit, configured to perform denoising processing on the at least one fourth image using the denoising autoencoder to obtain at least one fifth image;

[0168] A third generating unit, configured to generate a loss function based on a mean square error between the third image and the at least one fifth image;

[0169] A training unit is used to train the denoising autoencoder based on the loss function.

[0170] Optionally, the first generating module 502 includes:

[0171] A first input unit, configured to input the first time series data into a generative adversarial network GAN to obtain fourth time series data;

[0172] A setting unit, configured to set an indicator data value corresponding to at least one time node in the fourth time series data to a third preset value, to obtain fifth time series data;

[0173] The fourth generating unit is used to generate the first image based on the fifth time series data.

[0174] Optionally, the third generating module 505 includes:

[0175] A second input unit, used to input the second time series data into a generative adversarial network GAN to obtain sixth time series data;

[0176] The third input unit is used to use the sixth time series data as the input of the time series prediction model to obtain the target time series data.

[0177] Optionally, the first generating module 502 includes:

[0178] A first acquisition unit is used to acquire a first time node t in the first time series data and an indicator data value x corresponding to the first time node t ', the first time node is the tth time node in the first time series;

[0179] a dividing unit, configured to divide the first time series into preset periods T according to the first time node t, wherein one preset period T is a time period formed by L adjacent time nodes, and before the period where the first time node t is located, the first time series further includes n1 periods, and after the period where the first time node t is located, the first time series further includes n2 periods;

[0180] A second acquisition unit is used to acquire, in each cycle, a time period formed by the first c time nodes of the first time node to obtain a forward time window, where c is a preset constant;

[0181] A third acquisition unit is used to acquire, in each cycle, a time period formed by c time nodes after the first time node to obtain a backward time window;

[0182] The first computing unit is used to

[0183] k=1,2,…c, get the forward window coefficient α k ;λ is a preset constant;

[0184] The second computing unit is used to

[0185] k=1,2,…c, get the backward window coefficient β k ;

[0186] Correction unit for Correct the index data value corresponding to the first time node in the first time series data to x t , the sixth time series data is obtained; δ is a preset constant;

[0187] The fifth generating unit is used to generate the first image based on the sixth time series data.

[0188] The data processing device 500 can implement the embodiment of the present invention Figure 1The various processes of the method embodiment and the same beneficial effects are achieved, and will not be described again here to avoid repetition.

[0189] The embodiment of the present invention also provides an electronic device. Figure 6 , the electronic device may include a processor 601, a memory 602, and a program 6021 stored in the memory 602 and executable on the processor 601.

[0190] When the program 6021 in the electronic device is executed by the processor 601, the following can be achieved: Figure 1 Any steps in the corresponding method embodiments and achieving the same beneficial effects will not be repeated here.

[0191] A person skilled in the art can understand that all or part of the steps of implementing the above-mentioned embodiment method can be completed by hardware related to program instructions, and the program can be stored in a readable medium. The embodiment of the present invention also provides a readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned Figure 1 Any steps in the corresponding method embodiments can achieve the same technical effect and will not be repeated here to avoid repetition.

[0192] The storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0193] The above is a preferred implementation of the embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A data processing method, characterized in that: The method comprises: Acquire first time series data corresponding to a first time series, the first time series includes N time nodes, the first time series data includes the N time nodes and M indicator data values ​​corresponding to M time nodes among the N time nodes, the indicator data value is power consumption data; N is a positive integer, and M is a positive integer less than N; Generate a first image based on the first time series data, wherein the first image is used to represent the N time nodes, the M indicator data values, and the corresponding relationship between the M indicator data values ​​and the M time nodes; De-noising the first image using a denoising autoencoder DAE to obtain a second image; Based on the second image, second time series data is generated, where the second time series data includes N indicator data values ​​corresponding to the N time nodes, and the N indicator data values ​​include the M indicator data values; Generate target time series data based on the second time series data and the time series prediction model; The step of generating a first image based on the first time series data comprises: Get the first time node t in the first time series data and the indicator data value x corresponding to the first time node t ', the first time node is the tth time node in the first time series; According to the first time node t, the first time series is divided into preset periods T, wherein one preset period T is a time period formed by L adjacent time nodes, and before the period where the first time node t is located, the first time series further includes n1 periods, and after the period where the first time node t is located, the first time series further includes n2 periods; In each cycle, a time period formed by the first c time nodes of the first time node is obtained to obtain a forward time window, where c is a preset constant; In each cycle, a time period formed by the c time nodes after the first time node is obtained to obtain a backward time window; pass Get the forward window coefficient α k ;λ is a preset constant; is the time window attenuation factor; pass Get the backward window coefficient β k ; pass Correct the index data value corresponding to the first time node in the first time series data to x t , the sixth time series data is obtained; δ is a preset constant; is the period attenuation factor; Based on the sixth time series data, the first image is generated.

2. The data processing method according to claim 1, characterized in that: The step of generating a first image based on the first time series data comprises: Normalizing the M indicator data values ​​corresponding to the M time nodes; Setting the indicator data values ​​corresponding to NM time nodes other than the M time nodes among the N time nodes to the first preset value; Convert the N time nodes and the N indicator data values ​​corresponding to the N time nodes into a polar coordinate system; Based on the trigonometric function transformation of the polar coordinate system coordinates, the first image is generated, and the first image is a Gram angular field (GAF) diagram.

3. The data processing method according to claim 1, characterized in that: Before the step of performing denoising processing on the first image by using a denoising autoencoder DAE to obtain a second image, the method further includes: Acquire at least one training time series data from the historical time series data; Setting the indicator data value corresponding to at least one time node in each of the training time series data to a second preset value to obtain at least one third time series data; The denoising autoencoder is trained based on the at least one training time series data and the at least one third time series data.

4. The data processing method according to claim 3, characterized in that: The step of training the denoising autoencoder based on the at least one training time series data and the at least one third time series data comprises: Based on the at least one training time series data, correspondingly generating at least one third image; Based on the at least one third time series data, correspondingly generating at least one fourth image; Performing denoising processing on the at least one fourth image using the denoising autoencoder to obtain at least one fifth image; Generate a loss function based on a mean square error between the third image and the fifth image in one-to-one correspondence; Based on the loss function, the denoising autoencoder is trained.

5. The data processing method according to claim 1, characterized in that: The step of generating a first image based on the first time series data comprises: Inputting the first time series data into a generative adversarial network GAN to obtain fourth time series data; Setting the indicator data value corresponding to at least one time node in the fourth time series data to a third preset value to obtain fifth time series data; Based on the fifth time series data, the first image is generated.

6. The data processing method according to claim 1, characterized in that: The step of generating target time series data based on the second time series data and the time series prediction model comprises: Inputting the second time series data into a generative adversarial network GAN to obtain sixth time series data; The sixth time series data is used as input of the time series prediction model to obtain the target time series data.

7. A data processing device, characterized in that: include: A first acquisition module is used to acquire first time series data corresponding to a first time series, wherein the first time series includes N time nodes, and the first time series data includes the N time nodes and M indicator data values ​​corresponding to M time nodes among the N time nodes, wherein the indicator data value is power consumption data; N is a positive integer, and M is a positive integer less than N; A first generating module, configured to generate a first image based on the first time series data, wherein the first image is used to represent the N time nodes, the M indicator data values, and the corresponding relationship between the M indicator data values ​​and the M time nodes; A denoising module, configured to perform denoising processing on the first image using a denoising autoencoder DAE to obtain a second image; A second generating module, configured to generate second time series data based on the second image, wherein the second time series data includes N indicator data values ​​corresponding to the N time nodes, and the N indicator data values ​​include the M indicator data values; A third generating module, configured to generate target time series data based on the second time series data and the time series prediction model; The first generation module comprises: A first acquisition unit is used to acquire a first time node t in the first time series data and an indicator data value x corresponding to the first time node t ', the first time node is the tth time node in the first time series; a dividing unit, configured to divide the first time series into preset periods T according to the first time node t, wherein one preset period T is a time period formed by L adjacent time nodes, and before the period where the first time node t is located, the first time series further includes n1 periods, and after the period where the first time node t is located, the first time series further includes n2 periods; A second acquisition unit is used to acquire, in each cycle, a time period formed by the first c time nodes of the first time node to obtain a forward time window, where c is a preset constant; A third acquisition unit is used to acquire, in each cycle, a time period formed by c time nodes after the first time node to obtain a backward time window; The first computing unit is used to Get the forward window coefficient α k ;λ is a preset constant; is the time window attenuation factor; The second computing unit is used to Get the backward window coefficient β k ; Correction unit for Correct the index data value corresponding to the first time node in the first time series data to x t , the sixth time series data is obtained; δ is a preset constant; is the period attenuation factor; The fifth generating unit is used to generate the first image based on the sixth time series data.

8. An electronic device, comprising: A memory, a processor, and a program stored in the memory and executable on the processor; wherein the processor is used to read the program in the memory to implement the steps in the data processing method as described in any one of claims 1 to 6.

9. A readable storage medium for storing a program, characterized in that: When the program is executed by a processor, the steps of the data processing method according to any one of claims 1 to 6 are implemented.

Images