Time series data interpolation method and device based on variable self-attention

Abstract

The invention discloses a variable self-attention-based time series data interpolation method and device, and belongs to the technical field of data interpolation. The variable self-attention-based time series data interpolation method comprises the steps of processing multiple groups of acquired historical monitoring data, and acquiring at least one group of to-be-interpolated data corresponding to monitoring equipment; inputting the at least one group of to-be-interpolated data into an embedding layer of the data interpolation model to obtain an embedding vector corresponding to each target object included in the to-be-interpolated data output by the embedding layer; inputting each embedded vector into a sub-neural network unit of a data interpolation model in sequence to obtain a target vector corresponding to each target object output by the sub-neural network unit; inputting the target vector into a linear layer of the data interpolation model to obtain initial monitoring data output by the linear layer; and training a data interpolation model based on the initial monitoring data. According to the variable self-attention-based time series data interpolation method provided by the invention, the precision and accuracy of interpolation are improved.

Description

Technical Field

[0001] The present application belongs to the technical field of data interpolation, and in particular, relates to a method and device for time series data interpolation based on variable self-attention. Background Art

[0002] In the fields of finance, meteorology or transportation, it is necessary to analyze and predict trends, cyclical changes and seasonal changes based on the collected monitoring data. However, due to the instability or interference of the data acquisition equipment, the collected monitoring data is often missing, which brings difficulties to the analysis modeling and practical application of the data. In the related art, there are methods of interpolating monitoring data through traditional time series interpolation technology (such as nearest interpolation, linear interpolation or special value filling, etc.), but this method cannot make full use of the time series characteristics of the data, resulting in the interpolated value obtained. There may be a large deviation from the original data, and the accuracy of the traditional interpolation method is low, which cannot meet the needs of high-precision analysis. Summary of the invention

[0003] The present application aims to solve at least one of the technical problems existing in the related art. To this end, the present application proposes a method and device for time series data interpolation based on variable self-attention, which reduces the deviation between the interpolated value and the original data, improves the precision and accuracy of the interpolation, and can meet the needs of high-precision analysis.

[0004] In a first aspect, the present application provides a time series data interpolation method based on variable self-attention, the method comprising:

[0005] Processing multiple sets of historical monitoring data of the monitoring device acquired within a historical period to obtain at least one set of data to be interpolated corresponding to the monitoring device;

[0006] Inputting the at least one set of data to be interpolated into an embedding layer of a data interpolation model to obtain an embedding vector corresponding to each target object included in the data to be interpolated output by the embedding layer;

[0007] Input each of the embedding vectors into the sub-neural network unit of the data interpolation model in sequence to obtain a target vector corresponding to each of the target objects output by the sub-neural network unit;

[0008] The target vector is input into the linear layer of the data interpolation model to obtain the initial monitoring data output by the linear layer and corresponding to the data to be interpolated; the data interpolation model is used to perform interpolation processing on the historical monitoring data;

[0009] The data interpolation model is trained based on the initial monitoring data.

[0010] According to the time series data interpolation method based on variable self-attention provided in the embodiment of the present application, by constructing a data interpolation model to interpolate the data to be interpolated, independent time series are used as variable labels, the time series characteristics of the data can be fully utilized, and the data to be interpolated is interpolated by obtaining the correlation between each target object in the data to be interpolated, thereby reducing the deviation between the interpolated value and the original data, improving the precision and accuracy of the interpolation, and being able to meet high-precision analysis needs.

[0011] In a time series data interpolation method based on variable self-attention according to an embodiment of the present application, the sub-neural network unit includes a plurality of sub-neural network modules connected in sequence; the sub-neural network unit sequentially inputs each of the embedding vectors into the sub-neural network unit of the data interpolation model to obtain a target vector corresponding to each of the target objects output by the sub-neural network unit, including:

[0012] Input each of the embedding vectors into a first sub-neural network module among the plurality of sub-neural network modules in sequence, and obtain a plurality of first vectors output by the first sub-neural network module;

[0013] The multiple first vectors are input to a next sub-neural network module connected to the first sub-neural network module among the multiple sub-neural network modules, and the multiple target vectors output by the next sub-neural network module are obtained.

[0014] In a time series data interpolation method based on variable self-attention according to an embodiment of the present application, the sub-neural network includes a self-attention layer and a feedforward neural network layer connected in sequence; the embedding vectors are sequentially input into a first sub-neural network module among the plurality of sub-neural network modules, and a plurality of first vectors output by the first sub-neural network module are obtained, including:

[0015] Inputting the plurality of embedding vectors into the self-attention layer in sequence, and obtaining a second vector corresponding to the plurality of embedding vectors output by the self-attention layer;

[0016] Input multiple second vectors into the feedforward neural network layer, and obtain multiple first vectors output by the feedforward neural network layer.

[0017] In a time series data interpolation method based on variable self-attention according to an embodiment of the present application, the multiple embedding vectors are sequentially input into the self-attention layer, and a second vector corresponding to the multiple embedding vectors output by the self-attention layer is obtained, including:

[0018] Performing a linear transformation on the embedding matrices corresponding to the plurality of embedding vectors to obtain a query vector, a key vector, and a value vector corresponding to the embedding matrix;

[0019] Performing a dot product operation on the query vector and the key vector to obtain a correlation weight between the query vector and the key vector;

[0020] The relevance weight and the value vector are weightedly summed to obtain the second vector.

[0021] In an embodiment of the present application, a time series data interpolation method based on variable self-attention is provided, wherein the data interpolation model is trained based on the initial monitoring data, including:

[0022] Constructing a loss function based on the initial monitoring data and the data to be interpolated;

[0023] The data interpolation model is trained based on the loss function.

[0024] In a method for time series data interpolation based on variable self-attention according to an embodiment of the present application, after training the data interpolation model based on the initial monitoring data, the method further includes:

[0025] Acquire multiple groups of actual monitoring data of the monitoring device within a target period;

[0026] The multiple groups of actual monitoring data are input into the trained data interpolation model to obtain the target monitoring data corresponding to the monitoring device output by the trained data interpolation model.

[0027] In a second aspect, the present application provides a time series data interpolation device based on variable self-attention, comprising:

[0028] A first processing module is used to process multiple sets of historical monitoring data acquired from the monitoring device within a historical period to obtain at least one set of data to be interpolated corresponding to the monitoring device;

[0029] A second processing module is used to input the at least one set of data to be interpolated into an embedding layer of a data interpolation model to obtain an embedding vector corresponding to each target object included in the data to be interpolated output by the embedding layer;

[0030] A third processing module is used to input each of the embedding vectors into the sub-neural network unit of the data interpolation model in sequence to obtain a target vector corresponding to each of the target objects output by the sub-neural network unit;

[0031] A fourth processing module is used to input the target vector into the linear layer of the data interpolation model to obtain the initial monitoring data output by the linear layer and corresponding to the data to be interpolated; the data interpolation model is used to perform interpolation processing on the historical monitoring data;

[0032] A fifth processing module is used to train the data interpolation model based on the initial monitoring data.

[0033] According to the time series data interpolation device based on variable self-attention provided by the embodiment of the present application, by constructing a data interpolation model to interpolate the data to be interpolated, independent time series are used as variable labels, the time series characteristics of the data can be fully utilized, and the data to be interpolated is interpolated by obtaining the correlation between each target object in the data to be interpolated, thereby reducing the deviation between the interpolated value and the original data, improving the precision and accuracy of the interpolation, and being able to meet high-precision analysis needs.

[0034] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for time series data interpolation based on variable self-attention as described in the first aspect above is implemented.

[0035] In a fourth aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the time series data interpolation method based on variable self-attention as described in the first aspect above.

[0036] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the time series data interpolation method based on variable self-attention as described in the first aspect above.

[0037] The above one or more technical solutions in the embodiments of the present application have at least one of the following technical effects:

[0038] By constructing a data interpolation model to interpolate the data to be interpolated and using independent time series as variable markers, the time series characteristics of the data can be fully utilized. By obtaining the correlation between the target objects in the data to be interpolated, the data to be interpolated is interpolated, which reduces the deviation between the interpolated value and the original data, improves the precision and accuracy of the interpolation, and can meet the needs of high-precision analysis.

[0039] Furthermore, by setting up multiple sequentially connected sub-neural network modules to process the embedded vector, higher-level feature representations can be extracted to improve the accuracy and reliability of interpolation. In addition, by performing nonlinear transformation on the input data through the feedforward neural network layer, the data interpolation model can better adapt to the actual distribution of the data to improve the accuracy of interpolation. The application of attention mechanism and feedforward neural network in the variable dimension reduces the consumption of computing resources and can better process data with long-term backtracking.

[0040] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0042] Figure 1 It is one of the flow charts of the time series data interpolation method based on variable self-attention provided in the embodiment of the present application;

[0043] Figure 2 This is the second flow chart of the time series data interpolation method based on variable self-attention provided in the embodiment of the present application;

[0044] Figure 3 This is the third flow chart of the time series data interpolation method based on variable self-attention provided in the embodiment of the present application;

[0045] Figure 4 This is the fourth flow chart of the time series data interpolation method based on variable self-attention provided in the embodiment of the present application;

[0046] Figure 5 It is a structural schematic diagram of a time series data interpolation device based on variable self-attention provided in an embodiment of the present application;

[0047] Figure 6 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0049] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0050] In combination with the accompanying drawings, the time series data interpolation method based on variable self-attention, the time series data interpolation device based on variable self-attention, the electronic device and the readable storage medium provided in the embodiments of the present application are described in detail through specific embodiments and their application scenarios.

[0051] Among them, the time series data interpolation method based on variable self-attention can be applied to the terminal, and can be specifically executed by hardware or software in the terminal.

[0052] The terminal includes, but is not limited to, a portable communication device such as a mobile phone or tablet computer with a touch-sensitive surface (e.g., a touch screen display and / or a touch pad). It should also be understood that in some embodiments, the terminal may not be a portable communication device, but a desktop computer with a touch-sensitive surface (e.g., a touch screen display and / or a touch pad).

[0053] In the following various embodiments, a terminal including a display and a touch-sensitive surface is described. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, a mouse and a joystick.

[0054] The embodiment of the present application provides a method for time series data interpolation based on variable self-attention. The execution subject of the method for time series data interpolation based on variable self-attention can be an electronic device or a functional module or functional entity in the electronic device that can implement the method for time series data interpolation based on variable self-attention. The electronic devices mentioned in the embodiment of the present application include but are not limited to mobile phones, tablet computers, computers, cameras, wearable devices, etc. The time series data interpolation method based on variable self-attention provided in the embodiment of the present application is explained below by taking the electronic device as the execution subject as an example.

[0055] like Figure 1 As shown, the time series data interpolation method based on variable self-attention includes: step 110, step 120, step 130, step 140 and step 150.

[0056] It should be noted that the time series data interpolation method based on variable self-attention can be applied to fields such as finance, meteorology or transportation, and can be used to analyze and predict trends, cyclical changes and seasonal changes based on monitoring data.

[0057] Step 110: Process multiple sets of historical monitoring data of the monitoring device obtained within the historical period to obtain at least one set of data to be interpolated corresponding to the monitoring device;

[0058] In this step, the monitoring equipment may be different depending on the application field.

[0059] Monitoring equipment may include meteorological monitoring systems, financial monitoring systems or continuous emissions monitoring systems, etc.

[0060] For example, historical monitoring data of the CEMS (Continuous Emission Monitoring System) of a thermal power unit can be obtained.

[0061] Based on the historical monitoring data of thermal power units, environmental management and pollution control can be carried out on thermal power plants.

[0062] The historical period can be a series of months or years before the current collection time.

[0063] Historical monitoring data may include observations at multiple time points, each time point corresponding to one or more monitoring indicators. For example, for thermal power units, historical monitoring data may include sulfur dioxide concentration or nitrogen oxide concentration corresponding to each time point.

[0064] For example, City A includes 4 thermal power plants with a total of 8 thermal power units. The historical monitoring data of each thermal power unit can be collected through the CEMS of multiple thermal power units. The historical monitoring data of each thermal power unit can include the record of pollutant emissions every hour from B month C day of year A to E month F day of year D, which can specifically include the measured values ​​of SO2, NO x There are 10 attributes in total, including measured values ​​and smoke emissions.

[0065] The data to be interpolated are the data that need to be supplemented with missing values.

[0066] In the actual implementation process, data with fewer missing values ​​(eg, less than 30%) can be selected from historical monitoring data as a training data set, and then the mean interpolation and forward filling methods can be used to simply interpolate the data to make it non-missing data.

[0067] The interpolated data can then be masked so that the data randomly lacks multiple groups of continuous data, wherein a masking rate can be preset so that the amount of masked data to the total amount of training data meets the preset masking rate.

[0068] The dataset can be divided into training set, test set and validation set in a ratio of 7:2:1.

[0069] The training set can then be used as the data to be imputed.

[0070] Step 120: input at least one set of data to be interpolated into an embedding layer of a data interpolation model to obtain an embedding vector corresponding to each target object included in the data to be interpolated output by the embedding layer;

[0071] In this step, the data interpolation model is used to interpolate the historical monitoring data.

[0072] The data interpolation model may be a neural network model, for example, the data interpolation model may be a variable self-attention model.

[0073] The data to be interpolated may include monitoring indicators of multiple target objects corresponding to multiple time points.

[0074] The target object is a variable, for example, the target object may be sulfur dioxide gas or nitrogen oxides.

[0075] By performing variable time series extraction on the interpolation data, the time series data corresponding to each target object can be obtained.

[0076] The time series data corresponding to the target object includes the monitoring data of the target object arranged in chronological order during the historical period.

[0077] Time series data are continuous observations or records over time.

[0078] The variable self-attention model can be combined with the self-attention mechanism to process time series data containing multiple variables or features.

[0079] Among them, the self-attention mechanism can take into account the relationship between each element in the sequence and other elements when processing time series data.

[0080] The data interpolation model can include an embedding layer to embed time series data.

[0081] In the actual execution process, the entire time series data of each target object can be extracted and used as an independent unit (token). For example, when there are N target objects and T time steps, the time series data for the i-th target object can be expressed as: X :,i ={x 1i ,x 2i ,...,x Ti}, where X :,i is the i-th column of the matrix X, i.e., the monitoring value of all time steps of the i-th target object.

[0082] A multilayer perceptron (MLP) may be provided in the embedding layer.

[0083] The time series data of each target object is embedded using a multi-layer perceptron (MLP), which can map the time series data into a high-dimensional space.

[0084] Time series data can be embedded based on the following formula:

[0085] h i =Embedding(X :,i )

[0086] Among them, h i is the embedding vector corresponding to the i-th target object, and D is the dimension of the embedding vector, X :,i For time series data.

[0087] During the research and development process, the inventors discovered that in the related art there is a method of performing data interpolation according to time steps in a Transformer model. Multiple variables at each time point are embedded in a time marker, but the time marker mixes information of different variables. These variables may include different physical meanings and measurement units, resulting in unclear representation. In addition, when processing data in a longer historical window, the amount of calculation and complexity are large.

[0088] In the present application, data interpolation is performed by using the time series data corresponding to the variable (target object) as an independent unit to replace the time tag. The time series in which the variable is located is taken as an independent unit and the variable tag sequence is followed. The relative independence of each variable can be maintained, so that the data corresponding to each variable can be clearly represented, and the amount of calculation and complexity are reduced when processing data in a longer historical window.

[0089] Step 130: input each embedding vector into the sub-neural network unit of the data interpolation model in sequence to obtain the target vector corresponding to each target object output by the sub-neural network unit;

[0090] In this step, the data interpolation model may include a sub-neural network unit, and the input end of the sub-neural network unit is connected to the output end of the embedding layer.

[0091] The sub-neural network unit is used to obtain the high-dimensional vector corresponding to the target object.

[0092] The dimension of the target vector is larger than that of the embedding vector.

[0093] like Figure 3 As shown, in some embodiments, step 130 may include:

[0094] Input each embedding vector into a first sub-neural network module among the multiple sub-neural network modules in sequence, and obtain multiple first vectors output by the first sub-neural network module;

[0095] Input multiple first vectors to a next sub-neural network module connected to the first sub-neural network module among multiple sub-neural network modules, and obtain multiple target vectors output by the next sub-neural network module.

[0096] In this embodiment, the sub-neural network unit includes a plurality of sub-neural network modules connected sequentially.

[0097] The sub-neural network module can be a Transformer module.

[0098] Multiple sub-neural network modules are stacked and arranged, and each embedding vector can be input into the first sub-neural network module in sequence to obtain multiple first vectors output by the first sub-neural network module.

[0099] The multiple first vectors are passed as input to the next sub-neural network module sequentially connected to the first sub-neural network module, and the next sub-neural network module can extract deeper or more abstract features to output multiple target vectors.

[0100] The above process may be repeated multiple times, that is, multiple sub-neural network modules process the input data in turn, and each processing will make the data representation closer to the final interpolation target.

[0101] In the actual implementation process, Figure 2 As shown, the sub-neural network unit may include L stacked Transformer modules, for example, the sub-neural network unit may include 3 stacked Transformer modules.

[0102] By inputting each embedding vector into the first Transformer module, multiple first vectors output by the first Transformer module can be obtained.

[0103] By inputting multiple first vectors into the second Transformer module, multiple vectors output by the second Transformer module can be obtained.

[0104] Then, multiple vectors are input into the third Transformer module to obtain multiple target vectors output by the third Transformer module.

[0105] like Figure 2 As shown, in some embodiments, inputting each embedding vector sequentially into a first sub-neural network module among multiple sub-neural network modules, and obtaining multiple first vectors output by the first sub-neural network module may include:

[0106] Input multiple embedding vectors into the self-attention layer in sequence, and obtain a second vector corresponding to the multiple embedding vectors output by the self-attention layer;

[0107] The plurality of second vectors are input into the feedforward neural network layer, and the plurality of first vectors output by the feedforward neural network layer are obtained.

[0108] In this embodiment, the sub-neural network may include a self-attention layer and a feed-forward neural network layer connected sequentially.

[0109] The self-attention layer is used to obtain the correlation between multiple target objects.

[0110] Feed-forward neural network layers are used to learn global representations of time series data.

[0111] The embedding vector of each target object is used as the input of the self-attention layer. By calculating the correlation weight matrix between multiple target objects, the relationship between multiple target objects can be obtained.

[0112] In the self-attention layer, each embedding vector interacts with other embedding vectors to calculate the attention weight between it and other vectors. Based on the self-attention weight, the self-attention layer can generate a corresponding second vector for each embedding vector.

[0113] The second vector includes not only the information of the original embedding vector, but also the information of other embedding vectors.

[0114] The multiple second vectors output from the self-attention layer are input into the feed-forward neural network layer to further process the input data.

[0115] The feedforward neural network layer can include multiple fully connected layers, which can perform nonlinear transformation and feature combination on the input data.

[0116] The feedforward neural network layer can output multiple first vectors based on multiple second vectors.

[0117] In some embodiments, inputting multiple embedding vectors into the self-attention layer in sequence, and obtaining a second vector corresponding to the multiple embedding vectors output by the self-attention layer may include:

[0118] Performing linear transformation on the embedding matrices corresponding to the multiple embedding vectors to obtain the query vector, key vector, and value vector corresponding to the embedding matrix;

[0119] Perform a dot product operation on the query vector and the key vector to obtain the correlation weight between the query vector and the key vector;

[0120] The relevance weight and the value vector are weighted summed to obtain a second vector.

[0121] In this embodiment, an embedding matrix may be obtained based on multiple embedding vectors, and each row in the embedding matrix may represent an embedding vector.

[0122] The embedding matrix can be transformed three different times linearly through different weight matrices and bias items to obtain three new matrices. For example, the query vector matrix, key vector matrix and value vector matrix can be obtained. Each row of these three matrices can correspond to the query vector, key vector and value vector of each row in the original embedding matrix.

[0123] For each query vector in the query vector matrix, a dot product operation is performed with each key vector in the key vector matrix to obtain a scalar representing the similarity or correlation between the query vector and the key vector.

[0124] The dot product result can then be divided by a scaling factor, and a softmax operation can be performed on the scaled result to obtain the correlation weights between the query vector and all key vectors.

[0125] For each query vector, its corresponding relevance weight may be multiplied by the value vector in the value vector matrix, and then these products are added together to obtain a second vector corresponding to the query vector.

[0126] In actual implementation, multiple embedding vectors can be sequentially input into L stacked Transformer modules. The self-attention layer of each Transformer module can be used to obtain the correlation between multiple target objects, and the layer normalization and feedforward neural network layer can be used to learn the local and global representation of the sequence.

[0127] H l+1 =TrmBlock(H l ),l=0,…,L-1

[0128] in Contains N embedding vectors with dimension D.

[0129] The query Q, key K and value V vectors can be obtained through linear transformation:

[0130] Q=W Q H, K = W K H, V = W V H

[0131] Among them, W Q ,W K ,W V is the weight matrix with dimension , H is the embedding matrix of all variables.

[0132] Compute the similarity between the query vector and the key vector using the dot product and divide the result by the scaling factor , we can get the second vector:

[0133]

[0134] After the self-attention layer, each second vector is further processed by a feed-forward neural network layer, which can be applied independently on each target object to learn more complex nonlinear representations. The calculation of the feed-forward neural network layer can be expressed as:

[0135] FFN(h i )=ReLU(W1h i +b1)W2+b2

[0136] Among them, W1, W2 are weight matrices, b1, b2 are bias terms, and h i is the embedding vector corresponding to the i-th target object.

[0137] In some embodiments, layer normalization can be used between the self-attention layer and the feedforward neural network layer, so the above calculation steps can be formally expressed as:

[0138] H l =LayerNorm(H l-1 +Attention(H l-1 ))

[0139] H l =LayerNorm(H l +FFN(H l ))

[0140] For example, a residual connection can be made between the output of the self-attention layer and the original input, and then layer normalization can be performed on the result.

[0141] A residual connection can be made between the output of the feedforward neural network layer and the original input, and then the result is layer normalized.

[0142] According to the time series data interpolation method based on variable self-attention provided in the embodiment of the present application, by setting multiple sequentially connected sub-neural network modules to process the embedded vector, a higher-level feature representation can be extracted to improve the accuracy and reliability of interpolation; in addition, the input data is nonlinearly transformed through the feedforward neural network layer, so that the data interpolation model can better adapt to the actual distribution of the data to improve the accuracy of interpolation, and the application of attention mechanism and feedforward neural network in the variable dimension reduces the consumption of computing resources and can better process data with long-term backtracking.

[0143] Step 140: Input the target vector to the linear layer of the data interpolation model to obtain initial monitoring data output by the linear layer corresponding to the data to be interpolated.

[0144] In this step, the data interpolation model may include a linear layer, and the input end of the linear layer is connected to the output end of the sub-neural network unit.

[0145] By inputting the target vector into the linear layer, multiple interpolated values ​​can be obtained. After passing through the Transformer module, the representation of each target object can be converted into a high-dimensional vector. Then, after multiple high-dimensional vectors pass through the linear layer, an interpolated sequence can be generated, that is, the initial monitoring data.

[0146] Among them, the initial monitoring data can be expressed based on the following formula:

[0147] in, For the initial monitoring data, Projection is a linear layer that maps the high-dimensional vector to the interpolated sequence.

[0148] Step 150: training a data interpolation model based on the initial monitoring data.

[0149] In this step, the initial monitoring data is the predicted value of the data interpolation model, and the data interpolation model can be trained based on the difference between the predicted value and the actual value.

[0150] For example, a loss function can be constructed by the difference between the predicted value and the actual value, and then the parameters in the data interpolation model can be updated by the loss function to obtain a trained data interpolation model; or the parameters in the data interpolation model can be updated based on other methods, which can be selected based on user needs and is not limited in this application.

[0151] According to the time series data interpolation method based on variable self-attention provided in the embodiment of the present application, by constructing a data interpolation model to interpolate the data to be interpolated, independent time series are used as variable labels, the time series characteristics of the data can be fully utilized, and the data to be interpolated is interpolated by obtaining the correlation between each target object in the data to be interpolated, thereby reducing the deviation between the interpolated value and the original data, improving the precision and accuracy of the interpolation, and being able to meet high-precision analysis needs.

[0152] like Figure 4 As shown, in some embodiments, step 150 may include:

[0153] Construct a loss function based on the initial monitoring data and the data to be interpolated;

[0154] Train a data interpolation model based on a loss function.

[0155] In this embodiment, the initial monitoring data is data after the data to be interpolated is interpolated.

[0156] The loss function is used to measure the difference between the predicted values ​​of the data imputation model and the actual values.

[0157] Loss functions can include mean squared error, mean absolute error, or root mean square error, etc.

[0158] For example, the initial monitoring data after interpolation can be compared with the data to be interpolated, the training loss value can be calculated using the mean square error MSE, and then the parameters of the data interpolation model can be updated using the training loss value.

[0159] Continue to refer Figure 4 In some embodiments, the trained data interpolation model can be verified based on the data to be interpolated in the validation set.

[0160] In this embodiment, the data to be interpolated in the validation set can be used as the input of the trained data interpolation model, and the interpolation processing is performed on the data to be interpolated according to steps 110 to 140, and the mean square error between the interpolated data and the original data is calculated, and then it is determined whether the loss does not decrease for multiple consecutive times.

[0161] If the loss does not decrease for several consecutive times, the iteration ends and the network with the smallest loss value is selected as the trained data interpolation model.

[0162] If the loss fails to decrease for multiple consecutive times, determine whether the number of iterations has reached the set maximum number of iterations.

[0163] When the number of iterations reaches the set maximum number of iterations, the iteration ends and the network with the smallest loss value in the iteration is used as the trained data interpolation model.

[0164] If the number of iterations does not reach the set maximum number of iterations, continue iterating.

[0165] In some embodiments, the trained data interpolation model may also be applied to a test set to check the accuracy and rationality of the interpolated data.

[0166] In some embodiments, after step 150, the time series data interpolation method based on variable self-attention may further include:

[0167] Obtain multiple sets of actual monitoring data of the monitoring equipment within the target period;

[0168] Multiple groups of actual monitoring data are input into the trained data interpolation model to obtain the target monitoring data corresponding to the monitoring equipment output by the trained data interpolation model.

[0169] In this embodiment, the target period may be a few days, weeks, or months before the current collection time.

[0170] The actual monitoring data may include different physical quantities such as temperature, humidity, pressure, flow or concentration, and may be determined based on the type and purpose of the monitoring equipment.

[0171] In some embodiments, the actual monitoring data may be preprocessed. For example, the actual monitoring data may be checked to remove abnormal values ​​in the actual monitoring data.

[0172] The actual monitoring data may also be preliminarily interpolated. For example, missing values ​​in the actual monitoring data may be filled with 0 as data to be interpolated.

[0173] The preprocessed actual monitoring data is used as the input of the trained data interpolation model. The data interpolation model can predict and interpolate the missing data based on the input actual monitoring data to output the target monitoring data.

[0174] The target monitoring data may include estimates for missing values ​​in the actual monitoring data.

[0175] The time series data interpolation device based on variable self-attention provided in the present application is described below. The time series data interpolation device based on variable self-attention described below and the time series data interpolation method based on variable self-attention described above can be referenced to each other.

[0176] The time series data interpolation method based on variable self-attention provided in the embodiment of the present application can be performed by a time series data interpolation device based on variable self-attention. In the embodiment of the present application, the time series data interpolation method based on variable self-attention is performed by a time series data interpolation device based on variable self-attention as an example to illustrate the time series data interpolation device based on variable self-attention provided in the embodiment of the present application.

[0177] An embodiment of the present application also provides a time series data interpolation device based on variable self-attention.

[0178] like Figure 5 As shown, the time series data interpolation device based on variable self-attention includes: a first processing module 510, a second processing module 520, a third processing module 530, a fourth processing module 540 and a fifth processing module 550.

[0179] A first processing module 510 is used to process multiple sets of historical monitoring data acquired from the monitoring device within a historical period to acquire at least one set of data to be interpolated corresponding to the monitoring device;

[0180] The second processing module 520 is used to input at least one set of data to be interpolated into the embedding layer of the data interpolation model to obtain the embedding vector corresponding to each target object included in the data to be interpolated output by the embedding layer;

[0181] The third processing module 530 is used to sequentially input each embedding vector into the sub-neural network unit of the data interpolation model to obtain a target vector corresponding to each target object output by the sub-neural network unit;

[0182] The fourth processing module 540 is used to input the target vector into the linear layer of the data interpolation model to obtain the initial monitoring data output by the linear layer corresponding to the data to be interpolated; the data interpolation model is used to perform interpolation processing on the historical monitoring data;

[0183] The fifth processing module 550 is used to train a data interpolation model based on the initial monitoring data.

[0184] According to the time series data interpolation device based on variable self-attention provided by the embodiment of the present application, by constructing a data interpolation model to interpolate the data to be interpolated, independent time series are used as variable labels, the time series characteristics of the data can be fully utilized, and the data to be interpolated is interpolated by obtaining the correlation between each target object in the data to be interpolated, thereby reducing the deviation between the interpolated value and the original data, improving the precision and accuracy of the interpolation, and being able to meet high-precision analysis needs.

[0185] In some embodiments, the sub-neural network unit includes a plurality of sub-neural network modules connected in sequence; the third processing module 530 may also be used to:

[0186] Input each embedding vector into a first sub-neural network module among the multiple sub-neural network modules in sequence, and obtain multiple first vectors output by the first sub-neural network module;

[0187] Input multiple first vectors to a next sub-neural network module connected to the first sub-neural network module among multiple sub-neural network modules, and obtain multiple target vectors output by the next sub-neural network module.

[0188] In some embodiments, the sub-neural network includes a self-attention layer and a feedforward neural network layer connected in sequence; the third processing module 530 can also be used to:

[0189] Input multiple embedding vectors into the self-attention layer in sequence, and obtain a second vector corresponding to the multiple embedding vectors output by the self-attention layer;

[0190] The plurality of second vectors are input into the feedforward neural network layer, and the plurality of first vectors output by the feedforward neural network layer are obtained.

[0191] In some embodiments, the third processing module 530 may also be used to:

[0192] Performing linear transformation on the embedding matrices corresponding to the multiple embedding vectors to obtain the query vector, key vector, and value vector corresponding to the embedding matrix;

[0193] Perform a dot product operation on the query vector and the key vector to obtain the correlation weight between the query vector and the key vector;

[0194] The relevance weight and the value vector are weighted summed to obtain a second vector.

[0195] In some embodiments, the fifth processing module 550 may also be used to:

[0196] Construct a loss function based on the initial monitoring data and the data to be interpolated;

[0197] Train a data interpolation model based on a loss function.

[0198] In some embodiments, the time series data interpolation device based on variable self-attention may further include a sixth processing module, which is used to obtain multiple groups of actual monitoring data of the monitoring device within the target period after training the data interpolation model based on the initial monitoring data;

[0199] Multiple groups of actual monitoring data are input into the trained data interpolation model to obtain the target monitoring data corresponding to the monitoring equipment output by the trained data interpolation model.

[0200] The time series data interpolation device based on variable self-attention in the embodiment of the present application can be an electronic device, or a component in an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a car electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0201] The time series data interpolation device based on variable self-attention in the embodiment of the present application can be a device with an operating system. The operating system can be an Android operating system, an IOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0202] The time series data interpolation device based on variable self-attention provided in the embodiment of the present application can achieve Figures 1 to 4 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0203] In some embodiments, Figure 6 As shown, an embodiment of the present application also provides an electronic device 600, including a processor 601, a memory 602, and a computer program stored in the memory 602 and executable on the processor 601. When the program is executed by the processor 601, each process of the embodiment of the time series data interpolation method based on variable self-attention is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0204] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0205] On the other hand, the present application also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the various processes of the above-mentioned embodiment of the time series data interpolation method based on variable self-attention, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0206] On the other hand, the present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it is implemented to execute the various processes of the above-mentioned embodiment of the time series data interpolation method based on variable self-attention, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0207] On the other hand, an embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned embodiment of the time series data interpolation method based on variable self-attention, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0208] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0209] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0210] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution can be essentially or in other words, the part that contributes to the relevant technology can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0211] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A time series data interpolation method based on variable self-attention, characterized in that: include: Processing multiple sets of historical monitoring data of the monitoring device acquired within a historical period to obtain at least one set of data to be interpolated corresponding to the monitoring device; Inputting the at least one set of data to be interpolated into an embedding layer of a data interpolation model to obtain an embedding vector corresponding to each target object included in the data to be interpolated output by the embedding layer; Input each of the embedding vectors into the sub-neural network unit of the data interpolation model in sequence to obtain a target vector corresponding to each of the target objects output by the sub-neural network unit; Inputting the target vector into the linear layer of the data interpolation model to obtain initial monitoring data output by the linear layer corresponding to the data to be interpolated; The data interpolation model is used to perform interpolation processing on the historical monitoring data; The data interpolation model is trained based on the initial monitoring data.

2. The time series data interpolation method based on variable self-attention according to claim 1 is characterized in that: The sub-neural network unit includes a plurality of sub-neural network modules connected in sequence; the sub-neural network unit sequentially inputs each of the embedding vectors into the sub-neural network unit of the data interpolation model to obtain a target vector corresponding to each of the target objects output by the sub-neural network unit, including: Input each of the embedding vectors into a first sub-neural network module among the plurality of sub-neural network modules in sequence, and obtain a plurality of first vectors output by the first sub-neural network module; The multiple first vectors are input to a next sub-neural network module connected to the first sub-neural network module among the multiple sub-neural network modules, and the multiple target vectors output by the next sub-neural network module are obtained.

3. The time series data interpolation method based on variable self-attention according to claim 2 is characterized in that: The sub-neural network includes a self-attention layer and a feedforward neural network layer connected in sequence; the embedding vectors are sequentially input into a first sub-neural network module among the plurality of sub-neural network modules, and a plurality of first vectors output by the first sub-neural network module are obtained, including: Inputting the plurality of embedding vectors into the self-attention layer in sequence, and obtaining a second vector corresponding to the plurality of embedding vectors output by the self-attention layer; Input multiple second vectors into the feedforward neural network layer, and obtain multiple first vectors output by the feedforward neural network layer.

4. The method for time series data interpolation based on variable self-attention according to claim 3 is characterized in that: The step of sequentially inputting the plurality of embedding vectors into the self-attention layer, and obtaining a second vector corresponding to the plurality of embedding vectors output by the self-attention layer, comprises: Performing a linear transformation on the embedding matrices corresponding to the plurality of embedding vectors to obtain a query vector, a key vector, and a value vector corresponding to the embedding matrix; Performing a dot product operation on the query vector and the key vector to obtain a correlation weight between the query vector and the key vector; The relevance weight and the value vector are weightedly summed to obtain the second vector.

5. The time series data interpolation method based on variable self-attention according to any one of claims 1 to 4, characterized in that: The step of training the data interpolation model based on the initial monitoring data comprises: Constructing a loss function based on the initial monitoring data and the data to be interpolated; The data interpolation model is trained based on the loss function.

6. The time series data interpolation method based on variable self-attention according to any one of claims 1 to 4, characterized in that: After training the data interpolation model based on the initial monitoring data, the method further includes: Acquire multiple groups of actual monitoring data of the monitoring device within a target period; The multiple groups of actual monitoring data are input into the trained data interpolation model to obtain the target monitoring data corresponding to the monitoring device output by the trained data interpolation model.

7. A time series data interpolation device based on variable self-attention, characterized in that: include: A first processing module is used to process multiple sets of historical monitoring data acquired from the monitoring device within a historical period to obtain at least one set of data to be interpolated corresponding to the monitoring device; A second processing module is used to input the at least one set of data to be interpolated into an embedding layer of a data interpolation model to obtain an embedding vector corresponding to each target object included in the data to be interpolated output by the embedding layer; A third processing module is used to input each of the embedding vectors into the sub-neural network unit of the data interpolation model in sequence to obtain a target vector corresponding to each of the target objects output by the sub-neural network unit; A fourth processing module, used for inputting the target vector into the linear layer of the data interpolation model to obtain initial monitoring data output by the linear layer corresponding to the data to be interpolated; The data interpolation model is used to perform interpolation processing on the historical monitoring data; A fifth processing module is used to train the data interpolation model based on the initial monitoring data.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, it implements the time series data interpolation method based on variable self-attention as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the time series data interpolation method based on variable self-attention as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, it implements the time series data interpolation method based on variable self-attention as described in any one of claims 1 to 6.

Images