Space-time prediction model training method and device

Abstract

The invention provides a spatio-temporal prediction model training method and device, and the method comprises the steps: determining first filling data corresponding to original missing data and second filling data corresponding to each piece of mask missing data; wherein each piece of mask missing data is obtained by adding data masks with different covering proportions to the original missing data; generating first prediction data corresponding to the original missing data based on the first filling data, and generating second prediction data corresponding to the mask missing data based on the second filling data; wherein the first prediction data and the second prediction data are obtained based on a pre-constructed space-time prediction model; and updating model parameters in the space-time prediction model by using an error between the first prediction data and each second prediction data and an error between the first prediction data and real label data corresponding to the original missing data so as to train the space-time prediction model. According to the embodiment, the applicability and robustness of the space-time prediction model can be improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method and device for training a spatiotemporal prediction model. Background Art

[0002] Spatiotemporal data is data that contains time and space information, which can describe the changes of things at different times and places. Spatiotemporal prediction is based on spatiotemporal data to predict what may happen at a certain time and place in the future, such as weather forecast, traffic flow, etc.

[0003] However, in the actual process of spatiotemporal data collection, missing values ​​may exist in the spatiotemporal data due to factors such as sensor failure, network interruption and human error. When existing technologies use missing spatiotemporal data for spatiotemporal prediction, they are often unable to adapt to various spatiotemporal data missing situations, which reduces the robustness of the spatiotemporal prediction model and leads to inaccurate prediction data in the final prediction. Summary of the invention

[0004] In view of this, the embodiments of the present disclosure at least provide a spatiotemporal prediction model training method and device, which can improve the applicability and robustness of the spatiotemporal prediction model, and thus improve the accuracy of the final prediction data.

[0005] In a first aspect, an embodiment of the present disclosure provides a spatiotemporal prediction model training method, comprising:

[0006] Determine first filling data corresponding to the original missing data and second filling data corresponding to each masked missing data; wherein each masked missing data is obtained by adding data masks with different covering ratios to the original missing data;

[0007] Generate first prediction data corresponding to the original missing data based on the first filling data, and generate second prediction data corresponding to each mask missing data based on each second filling data; wherein the first prediction data and the second prediction data are obtained based on a pre-constructed spatiotemporal prediction model;

[0008] The model parameters in the spatiotemporal prediction model are updated by using the error between the first prediction data and each second prediction data, and the error between the first prediction data and the true label data corresponding to the original missing data, so as to train the spatiotemporal prediction model.

[0009] Optionally, determining first filling data corresponding to the original missing data includes:

[0010] Using random variables to fill missing values ​​in the original missing data; wherein the random variables obey normal distribution; the filled original missing data includes the original values ​​and the filling values ​​obtained by filling the missing values;

[0011] Construct a missing flag tensor to identify the original values ​​and filled values ​​in the original missing data after filling;

[0012] The filled original missing data and the missing flag tensor are used as the first filling data.

[0013] Optionally, the method further comprises:

[0014] In response to the missing identification tensor indicating that a proportion of fill values ​​in the original missing data exceeds a preset threshold, performing a linear transformation on the missing identification tensor;

[0015] The missing identification tensor after linear transformation and the original missing data after filling are used as the first filling data.

[0016] Optionally, using the error between the first predicted data and each second predicted data, and the error between the first predicted data and the true label data corresponding to the original missing data, updating the model parameters in the spatiotemporal prediction model to train the spatiotemporal prediction model includes:

[0017] The first loss function is to minimize the dissimilarity between the first predicted data and the true label data corresponding to the original missing data, and the second loss function is to minimize the dissimilarity between the first predicted data and each second predicted data, and the model parameters of the spatiotemporal prediction model are updated to train the spatiotemporal prediction model.

[0018] Optionally, the spatiotemporal prediction model includes a historical encoding network and a future decoding network; the historical encoding network includes multiple spatiotemporal attention modules for determining the historical spatiotemporal representations corresponding to the original missing data and each masked missing data based on the first filling data and each second filling data; the future decoding network includes multiple prediction attention modules for obtaining the first prediction data corresponding to the original missing data and each second prediction data corresponding to each masked missing data based on the historical spatiotemporal representation.

[0019] Optionally, the spatiotemporal attention module is constructed based on the temporal attention mechanism and the spatial attention mechanism; the predictive attention module is constructed based on the predictive attention mechanism and the spatial attention mechanism.

[0020] In a second aspect, the present disclosure provides a spatiotemporal prediction model training device, including:

[0021] A determination module, used to determine first filling data corresponding to the original missing data and second filling data corresponding to each masked missing data; wherein each masked missing data is obtained by adding data masks with different covering ratios to the original missing data;

[0022] A generating module, configured to generate first prediction data corresponding to the original missing data based on the first filling data, and to generate respective second prediction data corresponding to respective mask missing data based on respective second filling data; wherein the first prediction data and the second prediction data are obtained based on a pre-constructed spatiotemporal prediction model;

[0023] The training module is used to update the model parameters in the spatiotemporal prediction model by using the error between the first prediction data and each second prediction data, and the error between the first prediction data and the true label data corresponding to the original missing data, so as to train the spatiotemporal prediction model.

[0024] In a third aspect, an embodiment of the present disclosure further provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the above-mentioned first aspect, or the steps in any optional implementation of the first aspect are performed.

[0025] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned first aspect, or any optional implementation of the first aspect are executed.

[0026] In a fifth aspect, an embodiment of the present disclosure further provides a computer program product, including a computer program, which implements the method of any of the above embodiments when executed by a processor.

[0027] Any of the above aspects or any implementation of any of the aspects, by filling the original missing data to determine the first filling data, and using the pre-built spatiotemporal prediction model, based on the first filling data to generate the first prediction data corresponding to the original missing data, thereby enabling the spatiotemporal prediction model to use the filling data to capture the dependencies in space and time, and learn the complex dynamic associations between different time points and different locations. In addition, in order to enhance the robustness of the model, the disclosed embodiment also introduces masked missing data, that is, by adding different masking ratios to the original missing data, multiple versions of the second filling data are generated, and the spatiotemporal prediction model is used to generate multiple second prediction data corresponding to the multiple second filling data. Then, the distance between the first prediction data and the true label data, as well as the distance between the first prediction data and the multiple second prediction data, can be used to adjust the parameters of the spatiotemporal prediction model. Specifically, by using the error between the first predicted data and the actual labeled data to train the spatiotemporal prediction model, the distance between the predicted value and the actual value can be shortened, thereby improving the accuracy of the model prediction results; by using the error between the first predicted data and each second predicted data to train the spatiotemporal prediction model, it is possible to simulate different degrees of data missing situations and shorten the distance between the predicted value and the actual predicted value under different data missing situations. This not only enables the model to learn data completion methods under different missing modes, but also enhances the model's adaptability to missing data scenarios, thereby improving the applicability and robustness of the spatiotemporal prediction model.

[0028] The effects of the above-mentioned spatiotemporal prediction model training device, electronic device and storage medium can be found in the description of the above-mentioned spatiotemporal prediction model training method, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can also be obtained based on these drawings without creative work.

[0030] Figure 1 A flow chart of a spatiotemporal prediction model training method provided by an embodiment of the present disclosure is shown;

[0031] Figure 2 A schematic diagram of a process flow of a spatiotemporal prediction model training method provided by an embodiment of the present disclosure is shown;

[0032] Figure 3A schematic diagram of the structure of a spatiotemporal prediction model in a spatiotemporal prediction model training method provided by an embodiment of the present disclosure is shown;

[0033] Figure 4 A schematic diagram showing the structure of a spatiotemporal attention module and a prediction attention module in a spatiotemporal prediction model training method provided by an embodiment of the present disclosure is shown;

[0034] Figure 5 A schematic diagram of a spatiotemporal prediction model training device provided by an embodiment of the present disclosure is shown;

[0035] Figure 6 An exemplary system architecture is shown in which embodiments of the present disclosure may be applied;

[0036] Figure 7 A schematic diagram of the structure of a computer system of a terminal device or a server used to implement an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical scheme and advantages of the embodiments of the present disclosure clearer, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.

[0038] It should be noted that in the technical solution of the present invention, the collection, use, storage, sharing and transfer of user personal information involved are in compliance with the provisions of relevant laws and regulations, and it is necessary to inform the user and obtain the user's consent or authorization. When applicable, the user's personal information is de-identified and / or anonymized and / or encrypted.

[0039] The raising of the above problems and the solutions are the results obtained by the inventor after practice and careful research. The discovery process of the above problems and the solutions proposed for the above problems should be the contributions made by the inventor to the present disclosure during the disclosure process.

[0040] The technical solutions in the present disclosure will be clearly and completely described below in conjunction with the drawings in the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure claimed for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.

[0041] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0042] To facilitate understanding of this embodiment, a spatiotemporal prediction model training method disclosed in the embodiment of the present disclosure is first introduced in detail. The execution subject of the spatiotemporal prediction model training method provided in the embodiment of the present disclosure is generally a computer device with certain computing capabilities, and the computer device includes, for example: a terminal device or a server or other processing device, and the terminal device can be a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant, PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementations, the spatiotemporal prediction model training method can be implemented by a processor calling a computer-readable instruction stored in a memory.

[0043] See also Figure 1 As shown, it is a flow chart of the spatiotemporal prediction model training method provided by the embodiment of the present disclosure, and the method includes S101 to S103, wherein:

[0044] S101: Determine first filling data corresponding to original missing data and respective second filling data corresponding to respective masked missing data.

[0045] In the embodiment of the present disclosure, before determining the first filling data corresponding to the original missing data and the second filling data corresponding to the mask missing data, the mask missing data may be determined first. Figure 2As shown in , enhanced samples can be constructed by adding additional missing values ​​to the original missing data. The enhanced samples here are the masked missing data. In other words, the masked missing data is the data obtained by adding data masks with different missing patterns and missing rates on the basis of the original missing data. By introducing each masked missing data into the subsequent training steps, it not only helps the spatiotemporal prediction model learn data completion methods under different missing patterns and missing rates, but also enhances the adaptability of the spatiotemporal prediction model to missing data scenarios. It should be noted that the missing pattern here refers to the distribution of missing values ​​in the data set, among which common missing patterns include random missing, row missing, column missing, and block missing.

[0046] Specifically, Figure 2 The χ in the formula represents a given continuous N in Historical readings χ=X1,X2,…,X in , that is, the original missing data; Represents the missing flag tensor, which is used to identify which data in the original missing data is missing. By preprocessing the original missing data with the missing flag tensor, we can get the corresponding and Corresponding Here and represents the preprocessed χ and Then, by and Input the spatiotemporal attention prediction network to obtain the first predicted data corresponding to the original missing data For the first prediction data The corresponding high-dimensional space vector representation, where the superscript n indicates that the original missing data has passed through the n-layer spatiotemporal attention prediction network, and the subscript fur indicates that the current variable is the result of predicting the future. At the same time, the masked missing data can also be obtained based on the original missing data in the manner described above. Here, the masked missing data is Figure 2 k augmented samples in , Figure 2 χ (1) ,x (2) ,…,χ (k) Respectively represent the 1st mask missing data to the kth mask missing data; Then they represent the missing identification tensors corresponding to the 1st mask missing data to the kth mask missing data. By performing similar processing as described above on these k enhanced samples, k second prediction data corresponding to the k enhanced samples can be obtained. and The corresponding vector representation of k high-dimensional spaces Then, the spatiotemporal prediction model can be trained based on the first prediction data and k second prediction data using self-supervision loss and regression loss. The spatiotemporal prediction model here is Figure 2 Spatiotemporal attention prediction network in .

[0047] In the specific implementation, in the process of collecting spatiotemporal data, data missing may occur due to sensor failure, network interruption, human error and other reasons. In this case, in order to ensure that the spatiotemporal prediction model can receive complete data input, the original missing data and each masked missing data can be filled before training the model.

[0048] In one possible implementation, statistical interpolation methods (such as mean, median or time series interpolation) can be used to fill the missing values ​​in the original missing data and each masked missing data. Specifically, for time series data, linear interpolation can be performed using the data of the adjacent time points before and after; and for spatial dimension data, the mean of the adjacent area data can be used to fill the missing values. In addition, the missing values ​​can be directly filled with 0 or other specific values ​​to obtain the filled data.

[0049] In another possible implementation, considering that directly filling the missing values ​​with 0 may cause a change in data distribution, thereby affecting the generalization ability of the spatiotemporal prediction model, random variables that obey a normal distribution can be used in the disclosed embodiment to fill the missing values ​​in the original missing data and each masked missing data. Specifically, determining the first filling data corresponding to the original missing data includes: filling the missing values ​​in the original missing data with random variables; wherein the random variables obey a normal distribution; the filled original missing data includes the original values ​​and the filling values ​​obtained by filling the missing values; constructing a missing identification tensor to identify the original values ​​and the filling values ​​in the filled original missing data; and using the filled original missing data and the missing identification tensor as the first filling data.

[0050] In the specific implementation, the original missing data can be filled with reference to the following formula:

[0051]

[0052] Among them, i, j, and k represent the indexes of the input data χ of the spatiotemporal prediction model in the three dimensions of node, time, and feature, respectively. ijk Indicates whether the original missing data is missing at the index position ijk. ijk = 0, it means that there is no missing data at this position, and when m ijk =1, it means that there is missing data at this position. ijkIt means that when there is no missing data in the original missing data at the index position, the data at this position is the original value in the original missing data; and when there is missing data in the original missing data at the index position, the data at this position can be filled, and ε is the filled random variable, which obeys the normal distribution with a mean of 0 and a standard deviation of σ. It should be noted that the input data χ of the spatiotemporal prediction model here can be obtained by standard normalization of the original missing data. In addition, by identifying the original value and the filled value in the original missing data after filling by the missing identification tensor, the spatiotemporal prediction model can distinguish different data sources when processing data, and will not treat the filled data and the real data equally, that is, the original value in the original missing data can be considered reliable, and the filled value is only used as an estimate or guess, thereby optimizing the processing ability of the original missing data, reducing the negative impact of the filled value, and thus improving the prediction accuracy and robustness of the model. Based on the filled original missing data and the missing identification tensor added to the filled original missing data, the first filled data corresponding to the original missing data can be obtained. Similarly, the same method can also be adopted to obtain each second filled data corresponding to each mask missing data.

[0053] Optionally, considering that there are a large number of missing data in the original missing data, the 0 and 1 values ​​of the missing identification tensor may cause changes in data distribution, so a prompt tensor can be constructed based on the missing identification tensor. Specifically, the method also includes: in response to the missing identification tensor indicating that the proportion of filling values ​​in the original missing data exceeds a preset threshold, linearly transforming the missing identification tensor; and using the missing identification tensor after linear transformation and the original missing data after filling as the first filling data.

[0054] In a specific implementation, when there is a large amount of missing data in the original missing data, the introduction of a missing identification tensor may affect the data distribution. This is because the missing identification tensor introduces a differentiated processing mechanism between the original value and the filled value in the training process of the spatiotemporal prediction model, which makes the model have different strategies for these two types of data when processing. As the training deepens, the differentiated processing strategy may cause the data distribution to shift. In order to solve this problem, the original missing identification tensor can be linearly or nonlinearly changed, and the missing identification tensor after the linear change and the original missing data after filling are used as the first filling data corresponding to the original missing data. Alternatively, the above purpose can also be achieved by constructing a prompt tensor based on the missing identification tensor. For example, the prompt tensor It can be expressed as:

[0055]

[0056] in, Represents the hint tensor under the i, j, k indexes; the meaning of the above formula is that when the data under the index position ijk in the original missing data is missing, the hint tensor corresponding to this position is 1; otherwise it is -1.

[0057] It should be noted that the above method of filling the original missing data and each masked missing data to obtain the first filling data and each second filling data is only exemplified as a feasible implementation method of the embodiment of the present disclosure, and does not constitute an improper limitation on the present invention. In practical applications, the corresponding filling method can be set according to actual needs and circumstances. The embodiment of the present disclosure does not make specific limitations on this, and the ability to achieve its functions shall prevail.

[0058] S102: Generate first prediction data corresponding to the original missing data based on the first filling data, and generate respective second prediction data corresponding to respective masked missing data based on respective second filling data.

[0059] In the embodiment of the present disclosure, the spatiotemporal prediction model includes a historical encoding network and a future decoding network; the historical encoding network includes a plurality of spatiotemporal attention modules for determining the historical spatiotemporal representations corresponding to the original missing data and the masked missing data based on the first filling data and the second filling data; the future decoding network includes a plurality of prediction attention modules for obtaining the first prediction data corresponding to the original missing data and the second prediction data corresponding to the masked missing data based on the historical spatiotemporal representations.

[0060] Specifically, Figure 3 As shown in the figure, the spatiotemporal prediction model includes two parts: the historical encoding network and the future decoding network. Among them, the time encoding module is a module shared by the historical encoding network and the future decoding network, which is used to encode the timestamp information into a time information vector to assist the timely representation learning of multi-head self-attention. In the specific implementation, time encoding methods such as sin / cos position encoding or learnable position encoding can be used to embed time features into the spatiotemporal prediction model, so that the spatiotemporal prediction model can perceive the sequential information of different time steps. Figure 3 As shown, the left side is the historical encoding network, and the right side is the future decoding network, T his Represents the time series of history, which passes through the time encoding module and then reaches the spatiotemporal attention module; and It represents the original missing data with missing marker tensor after preprocessing. After adding mask to the mask input encoder, k masked missing data are formed and then enter the spatiotemporal attention module. is a learnable node representation matrix that is directly input into the spatiotemporal attention module, where Represents an N l ×D modelspace, indicating the size of V, N l is the amount of data, D model is the dimension size of the representation; each row of the matrix represents a node and can be randomly initialized and input into the spatiotemporal attention and prediction attention modules to facilitate the self-attention mechanism to capture spatial correlations from the data; the mask input encoder can be a 2-layer 1*1 CNN (Convolutional Neural Network) or a multilayer perceptron (MLP). Figure 3 As shown in Figure 1, the spatiotemporal prediction model includes n spatiotemporal attention modules and n prediction attention modules, where n is a network hyperparameter. The spatiotemporal attention module and the prediction attention module can be composed of multi-head spatial attention, multi-head temporal attention and multi-head prediction attention. The input data mentioned above can get the corresponding After n spatiotemporal attention modules in the history encoding network, the corresponding Then, after n spatiotemporal attention modules, we get We can reach the prediction attention module, where V ＊ It also directly inputs the prediction attention module, T fur The time series representing the future moment also enters the prediction attention module after passing through the time encoding module. After passing through each layer of the prediction attention module, the corresponding After passing through n layers of prediction attention modules, the final And the first forecast data like Figure 4 As shown, the spatiotemporal attention module can be constructed based on the temporal attention mechanism and the spatial attention mechanism; the prediction attention module can be constructed based on the prediction attention mechanism and the spatial attention mechanism.

[0061] like Figure 4 As shown in the figure, the spatiotemporal attention module can adopt a parallel multi-head temporal attention mechanism and a multi-head spatial attention mechanism, that is, the spatiotemporal representation of the output of the previous module Respectively with time code T his and spatial encoding V ＊ After adding, input the multi-head attention mechanism; in spatiotemporal representation In the figure, the subscript his indicates that the spatiotemporal representation is obtained in the history encoding network based on the historical time series. The superscript i-1 indicates that the output data of the previous module of the current module is used as the input data of the current module. The superscript number increases by one after each layer of spatiotemporal attention module until it passes through n layers of spatiotemporal attention modules.

[0062] In a possible implementation, Figure 4 As shown, in any spatiotemporal attention module, the output data of the previous spatiotemporal attention module can be first With the historical time series T his Input into the multi-head temporal attention unit of the spatiotemporal attention module, and then the output data of the previous spatiotemporal attention module can be With spatial encoding V ＊ Input to the multi-head spatial attention unit of the spatiotemporal attention module. Finally, based on the output of the multi-head temporal attention unit and the output of the multi-head spatial attention unit, combined with the output data of the previous spatiotemporal attention module Get the output data of the current spatiotemporal attention module

[0063] For the prediction attention module, considering the initial representation of its input It is only the sum of time encoding and space encoding, and does not contain any information about the current sensor readings. The superscript 0 indicates that it is the input data of the first layer prediction attention module. After passing through the first layer prediction attention module, the superscript number increases by 1; the subscript fur indicates that it is obtained in the future decoding network based on the time series at the future moment. Therefore, you can choose to first convert the output data of the previous prediction attention module Historical time series T his , the output data of the nth spatiotemporal attention module The time series T with the future moment fur Input to the multi-head prediction attention unit, and then the output data of the multi-head prediction attention unit is combined with the spatial encoding V ＊ Input to the multi-head spatial attention unit in the prediction attention module, and finally use the output of the multi-head spatial attention unit and the output data of the previous prediction attention module Get the output data of the current prediction attention module It should be noted that the construction method of the above-mentioned spatiotemporal attention module and the predictive attention module is only a possible implementation method in the embodiment of the present disclosure. In practical applications, different needs can be met by increasing or adjusting the number or order of applications of multi-head spatial attention, multi-head temporal attention and multi-head predictive attention. This embodiment does not make specific limitations on this.

[0064] In the disclosed embodiment, a multi-head spatial attention unit output representation matrix in the previous layer can be given Divide it into T different representation matrices in the time dimension, namely in, The subscript of represents the spatial dimension, i is the multi-head spatial attention unit of the current layer module, and its superscript i-1 represents the multi-head spatial attention unit of the previous layer module. For any representation matrix The following steps can be used to calculate. The calculation process of the multi-head spatial attention mechanism is explained below. First, the learnable parameters and Multiply to generate the query matrix ( The subscript s indicates that the current calculation dimension is the spatial dimension, and the superscript i indicates that the calculation of the multi-head spatial attention unit in the i-th layer module is currently being executed; Represents the learnable parameters mapped to the query vector Q, and its subscript Q(s) indicates that the parameter is used to generate the learnable mapping weight matrix of the query vector Q in the spatial dimension. The query matrix represents the features that the spatiotemporal prediction model currently focuses on, that is, it determines what information to extract from the first input filling data and each second filling data. The query matrix is ​​equivalent to a question, and the model uses the query matrix to find relevant information). Similarly, the learnable parameters mapped by the key vector K The learnable parameters mapped to the value vector V Respectively Multiply to generate the key matrix With value matrix The key matrix represents the feature description of each position (or node) in the input first filling data and each second filling data, and is used to calculate the similarity between the query and the key together with the query matrix to measure which data is most relevant to the current query; the value matrix contains the features actually to be extracted from the input first filling data and each second filling data, and is the part that is ultimately weighted and summed according to the attention weight. The meanings of the superscripts and subscripts of the above-mentioned variables related to the key matrix and the value matrix refer to the meanings of the superscripts and subscripts of the variables related to the query matrix described above, and will not be repeated here. After determining the above-mentioned query matrix, key matrix and value matrix, the transposed matrix of the query matrix and the key matrix can be multiplied together, and then normalized. Specifically, the calculation process can be expressed by the following formula:

[0065]

[0066] in, The representation matrix The size parameter (i.e. the number of rows); and The meanings of the upper and lower subscripts are as described above and will not be repeated here; represents the transpose. Then, we can use the softmax function to Converted into a weight distribution, representing the relevance of each element, the formula can be expressed as:

[0067]

[0068] Among them, a ef It represents the attention score, which is the similarity score obtained by the dot product of the query matrix and the key matrix. It can measure the similarity between the e-th position in the query matrix and the f-th position in the key matrix. The larger its value, the stronger the correlation between the two. It should be noted that the base e and α in the above formula are ef with a ef The meaning of "e" in the formula is different. The base e is the base of the exponential function, and a ef With α ef Then it is the element in the formula, indicating the e-th position. ef Represents the normalized attention weight, which is obtained by ef The softmax function is used to represent the relative importance of the e-th position in the query matrix and the f-th position in the key matrix. Its value is between 0 and 1, and the sum of the weights of all positions is 1. Thus, the attention weight matrix is ​​obtained. Then, the weighted matrix is ​​passed through a multi-layer fully connected neural network to obtain The MLP() operator is an operation that maps and nonlinearly transforms input data through several fully connected layers, and represents a multi-layer fully connected neural network in the disclosed embodiment.

[0069] The calculation process of the multi-head temporal attention mechanism is similar to that of the multi-head spatial attention mechanism. The difference is that the temporal attention mechanism splits the spatiotemporal representation into multiple representation matrices along the spatial dimension. The remaining steps are similar to the calculation process for the multi-head spatial attention mechanism mentioned above, and will not be repeated here. The calculation process can be expressed by the following formula:

[0070]

[0071] in, The subscript t represents the time dimension, j represents the jth representation matrix obtained after division, and the superscript i represents the multi-head time attention unit of the current layer module; The superscript i-1 indicates the multi-head temporal attention unit of the previous module; are the learnable parameters of Q, K, and V mappings, respectively. Their definitions are similar to those in the previous description of the multi-head spatial attention mechanism and will not be repeated here. The Softmax operator is a normalization operation used to convert the input vector into a probability distribution.

[0072] In the embodiment of the present disclosure, the multi-head prediction attention mechanism is a special spatiotemporal attention mechanism. Since it is impossible to encode future sensor readings, the multi-head prediction attention mechanism is directly used for the temporal attention mechanism. The Q, K, V mapping cannot get accurate results. Therefore, we can first use the final output of the historical coding network Calculate K and V, and then use the future decoding network Calculate Q, and then use the calculated Q, K, and V for subsequent attention operations. The specific formula can be expressed as:

[0073]

[0074] in, For a complete representation of space and time The local spatiotemporal representation at node j.

[0075] S103: Using the error between the first prediction data and each second prediction data, and the error between the first prediction data and the true label data corresponding to the original missing data, the model parameters in the spatiotemporal prediction model are updated to train the spatiotemporal prediction model.

[0076] In the embodiment of the present disclosure, the first filling data corresponding to the original missing data can be input into the pre-built spatiotemporal prediction model, and after the steps described in S102, the first predicted data corresponding to the original missing data can be obtained, that is, As described above, data masks simulating different missing patterns and missing rates can be added to the original missing data to form k masked missing data, and the k masked missing data are also input into the pre-built spatiotemporal prediction model to obtain the corresponding In this way, the model parameters in the spatiotemporal prediction model are updated to achieve the purpose of training the spatiotemporal prediction model. Specifically, the model parameters in the spatiotemporal prediction model are updated using the error between the first prediction data and each second prediction data, and the error between the first prediction data and the real label data corresponding to the original missing data, so as to train the spatiotemporal prediction model, including: minimizing the dissimilarity between the first prediction data and the real label data corresponding to the original missing data as a first loss function, minimizing the dissimilarity between the first prediction data and each second prediction data as a second loss function, updating the model parameters of the spatiotemporal prediction model, so as to train the spatiotemporal prediction model.

[0077] In the specific implementation, Figure 2 As shown, the second loss function By shortening the distance between the first prediction data and each second prediction data, the spatiotemporal prediction model can be applied to different missing patterns and missing rates, thus improving the robustness and applicability of the model. It can be expressed by the following formula:

[0078]

[0079] Among them, ∥·∥ represents the regression loss function, and [·] represents constantization; represents the second prediction data corresponding to the i-th enhanced sample among the k enhanced samples; represents the first predicted data corresponding to the original missing data. In addition, the first loss function can also be used To narrow the distance between the first predicted data and the true label data, the accuracy of the spatiotemporal prediction model can be improved. The first loss function can be expressed as represents the first prediction data, For The corresponding vector representation of the high-dimensional space, y represents the true label data corresponding to the original missing data.

[0080] It should be noted that the above loss function is only used as an example of a possible implementation of the embodiment of the present disclosure. In practical applications, the corresponding loss function can be set according to actual tasks and requirements. The embodiment of the present disclosure does not make specific limitations on this, and the function that can be realized shall prevail.

[0081] According to a second aspect of the embodiment of the present disclosure, Figure 5 As shown, a spatiotemporal prediction model training device 500 is provided, comprising:

[0082] The determination module 501 is used to determine the first filling data corresponding to the original missing data and each second filling data corresponding to each masked missing data; wherein each masked missing data is obtained by adding a data mask with a different covering ratio to the original missing data;

[0083] A generating module 502, configured to generate first prediction data corresponding to the original missing data based on the first filling data, and to generate respective second prediction data corresponding to respective mask missing data based on respective second filling data; wherein the first prediction data and the second prediction data are obtained based on a pre-constructed spatiotemporal prediction model;

[0084] The training module 503 is used to update the model parameters in the spatiotemporal prediction model by using the error between the first prediction data and each second prediction data, and the error between the first prediction data and the true label data corresponding to the original missing data, so as to train the spatiotemporal prediction model.

[0085] Optionally, the determination module 501 is specifically configured to:

[0086] Using random variables to fill missing values ​​in the original missing data; wherein the random variables obey normal distribution; the filled original missing data includes the original values ​​and the filling values ​​obtained by filling the missing values;

[0087] Construct a missing flag tensor to identify the original values ​​and filled values ​​in the original missing data after filling;

[0088] The filled original missing data and the missing flag tensor are used as the first filling data.

[0089] Optionally, the determining module 501 is further configured to:

[0090] In response to the missing identification tensor indicating that a proportion of fill values ​​in the original missing data exceeds a preset threshold, performing a linear transformation on the missing identification tensor;

[0091] The missing identification tensor after linear transformation and the original missing data after filling are used as the first filling data.

[0092] Optionally, the training module 503 is specifically used for:

[0093] The first loss function is to minimize the dissimilarity between the first predicted data and the true label data corresponding to the original missing data, and the second loss function is to minimize the dissimilarity between the first predicted data and each second predicted data, and the model parameters of the spatiotemporal prediction model are updated to train the spatiotemporal prediction model.

[0094] Optionally, the spatiotemporal prediction model includes a historical encoding network and a future decoding network; the historical encoding network includes multiple spatiotemporal attention modules for determining the historical spatiotemporal representations corresponding to the original missing data and each masked missing data based on the first filling data and each second filling data; the future decoding network includes multiple prediction attention modules for obtaining the first prediction data corresponding to the original missing data and each second prediction data corresponding to each masked missing data based on the historical spatiotemporal representation.

[0095] Optionally, the spatiotemporal attention module is constructed based on the temporal attention mechanism and the spatial attention mechanism; the predictive attention module is constructed based on the predictive attention mechanism and the spatial attention mechanism.

[0096] According to the third aspect of an embodiment of the present disclosure, an electronic device for training a spatiotemporal prediction model is provided, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors implement the method provided by the first aspect of the embodiment of the present invention.

[0097] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable medium is provided, on which a computer program is stored, and when the program is executed by a processor, the method provided by the first aspect of the embodiment of the present disclosure is implemented.

[0098] According to a fifth aspect of an embodiment of the present invention, a computer program product is provided, including a computer program, and the computer program implements the method of any of the above embodiments when executed by a processor.

[0099] Figure 6 An exemplary system architecture 600 is shown to which the spatiotemporal prediction model training method or spatiotemporal prediction model training apparatus implemented by the present disclosure can be applied.

[0100] like Figure 6 As shown, system architecture 600 may include terminal devices 601, 602, 603, network 604 and server 605. Network 604 is used to provide a medium for communication links between terminal devices 601, 602, 603 and server 605. Network 604 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0101] Users can use terminal devices 601, 602, and 603 to interact with server 605 through network 604 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 601, 602, and 603, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only examples).

[0102] The terminal devices 601 , 602 , and 603 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers, etc.

[0103] The server 605 may be a server that provides various services, such as a background management server (only an example) that provides support for shopping websites browsed by users using the terminal devices 601, 602, and 603. The background management server may process the received spatiotemporal prediction model training request and feed back the processing result (only an example) to the terminal device.

[0104] It should be noted that the spatiotemporal prediction model training method provided in the embodiment of the present invention is generally performed by the server 605, and accordingly, the spatiotemporal prediction model training device is generally set in the server 605. The spatiotemporal prediction model training method provided in the embodiment of the present invention can also be performed by the terminal devices 601, 602, and 603, and accordingly, the spatiotemporal prediction model training device can be set in the terminal devices 601, 602, and 603.

[0105] It should be understood that Figure 6 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0106] Reference below Figure 7 , which shows a schematic diagram of the structure of a computer system 700 of a terminal device suitable for implementing an embodiment of the present invention. Figure 7 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0107] like Figure 7 As shown, the computer system 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage part 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the system 700 are also stored. The CPU 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0108] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, a mouse, etc.; an output section 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, a modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed, so that a computer program read therefrom is installed into the storage section 708 as needed.

[0109] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the central processing unit (CPU) 701, the above-mentioned functions defined in the system of the present invention are executed.

[0110] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0111] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0112] The modules involved in the embodiments of the present invention may be implemented by software or hardware. The modules described may also be set in a processor. For example, a processor includes a determination module, a generation module, and a training module, wherein the names of these modules do not constitute a limitation on the modules themselves in certain cases. For example, the determination module may also be described as "a module for determining first filling data corresponding to original missing data and second filling data corresponding to each masked missing data".

[0113] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device implements the following method: determining first filling data corresponding to the original missing data and second filling data corresponding to each mask missing data; wherein each mask missing data is obtained by adding a data mask with a different masking ratio to the original missing data; generating first prediction data corresponding to the original missing data based on the first filling data, and generating second prediction data corresponding to each mask missing data based on each second filling data; wherein the first prediction data and the second prediction data are obtained based on a pre-constructed spatiotemporal prediction model; using the error between the first prediction data and each second prediction data, and the error between the first prediction data and the true label data corresponding to the original missing data, the model parameters in the spatiotemporal prediction model are updated to train the spatiotemporal prediction model.

[0114] Finally, it should be noted that the above embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure is described in detail with reference to the above embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above embodiments within the technical scope disclosed in the present disclosure, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A spatiotemporal prediction model training method, characterized in that: include: Determine first filling data corresponding to the original missing data and second filling data corresponding to each masked missing data; wherein each masked missing data is obtained by adding data masks with different covering ratios to the original missing data; Generate first prediction data corresponding to the original missing data based on the first filling data, and generate second prediction data corresponding to each mask missing data based on each second filling data; wherein the first prediction data and the second prediction data are obtained based on a pre-constructed spatiotemporal prediction model; The model parameters in the spatiotemporal prediction model are updated by using the error between the first prediction data and each second prediction data, and the error between the first prediction data and the true label data corresponding to the original missing data, so as to train the spatiotemporal prediction model.

2. The method according to claim 1, characterized in that: Determining first filling data corresponding to the original missing data includes: Filling missing values ​​in the original missing data with random variables; wherein the random variables obey a normal distribution; and the filled original missing data include original values ​​and filled values ​​obtained by filling the missing values; Construct a missing flag tensor to identify the original values ​​and filled values ​​in the original missing data after filling; The filled original missing data and the missing identification tensor are used as the first filling data.

3. The method according to claim 2, characterized in that The method further comprises: In response to the missing identification tensor indicating that a proportion of fill values ​​in the original missing data exceeds a preset threshold, performing a linear transformation on the missing identification tensor; The missing identification tensor after linear transformation and the original missing data after filling are used as the first filling data.

4. The method according to claim 1, characterized in that Using the error between the first predicted data and each second predicted data, and the error between the first predicted data and the true label data corresponding to the original missing data, updating the model parameters in the spatiotemporal prediction model to train the spatiotemporal prediction model, including: The first loss function is to minimize the dissimilarity between the first predicted data and the true label data corresponding to the original missing data, and the second loss function is to minimize the dissimilarity between the first predicted data and each second predicted data, and the model parameters of the spatiotemporal prediction model are updated to train the spatiotemporal prediction model.

5. The method according to claim 1, characterized in that: The spatiotemporal prediction model includes a historical encoding network and a future decoding network; the historical encoding network includes a plurality of spatiotemporal attention modules for determining the historical spatiotemporal representations corresponding to the original missing data and the masked missing data based on the first filling data and the second filling data; the future decoding network includes a plurality of prediction attention modules for obtaining the first prediction data corresponding to the original missing data and the second prediction data corresponding to the masked missing data based on the historical spatiotemporal representations.

6. The method according to claim 5, characterized in that The spatiotemporal attention module is constructed based on the temporal attention mechanism and the spatial attention mechanism; the predictive attention module is constructed based on the predictive attention mechanism and the spatial attention mechanism.

7. A spatiotemporal prediction model training device, characterized in that: include: A determination module, used to determine first filling data corresponding to the original missing data and second filling data corresponding to each masked missing data; wherein each masked missing data is obtained by adding data masks with different covering ratios to the original missing data; A generating module, configured to generate first predicted data corresponding to the original missing data based on the first filling data, and to generate respective second predicted data corresponding to respective masked missing data based on respective second filling data; wherein the first predicted data and the second predicted data are obtained based on a pre-constructed spatiotemporal prediction model; A training module is used to update the model parameters in the spatiotemporal prediction model by using the error between the first prediction data and each second prediction data, and the error between the first prediction data and the true label data corresponding to the original missing data, so as to train the spatiotemporal prediction model.

8. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images