Learning method for nonholonomic multivariate time series prediction

Abstract

The embodiment of the present application provides a method for training a student model, comprising: obtaining a first training set, which comprises a plurality of original samples without missing treatment and corresponding labels; obtaining a teacher model trained by using the original samples and the labels, which comprises a feature extractor for extracting a teacher representation from the original samples and a regression layer for obtaining a teacher prediction result according to the teacher representation; processing each original sample by using a plurality of missing rates to obtain a corresponding missing sample, and all the missing samples and the corresponding labels form a second training set; obtaining a student model, which comprises a feature extractor for extracting a student representation from the missing sample and a regression layer for obtaining a student prediction result according to the student representation; training the student model by using the second training set, the teacher representation and the teacher prediction result, and updating parameters of the student model based on a total loss weighted by a representation loss, a prediction result loss, a contrast loss and an absolute error loss during the training.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of machine learning, in particular to the field of multivariate time series application in machine learning, and more particularly to a learning method for non-complete multivariate time series prediction. BACKGROUND

[0002] Multivariate time series is a dataset composed of two or more related time series, which can be different indicators under the same topic, such as multiple related data indicators in the fields of economy, meteorology, medicine, etc. Multivariate time series prediction is an important direction in the fields of predictive analysis and data science, which involves using multiple time series variables to predict future values, which can help enterprises and government departments make more accurate decisions with limited information. Compared with univariate time series prediction, multivariate time series prediction utilizes the mutual relationship between variables to provide more accurate and comprehensive prediction results. However, due to the nonlinearity of time series itself and the complex spatial dependence relationship between different time series, the accuracy of prediction has been a difficult problem for academia and industry.

[0003] The mainstream multivariate time series prediction techniques can be generally divided into statistical methods, machine learning methods and deep learning methods. Among them, the statistical method relies on modeling the time series process into a mathematical formula in the form of prior knowledge, which is relatively effective in dealing with regular and stationary data; however, for non-stationary data, this method often fails to fully utilize expert experience to analyze the dynamic changes of data. Machine learning methods, such as support vector machines and random forests, enhance the analysis of historical data and the prediction ability of future trends through nonlinear modeling, but machine learning methods are mainly designed for univariate time series prediction and fail to effectively analyze the interaction between multiple time series.

[0004] Compared with statistical methods and machine learning methods, deep learning methods have received widespread attention due to their excellent nonlinear modeling ability and in-depth analysis of spatial relationships. Classic deep learning models, such as recurrent neural networks and convolutional neural networks, effectively mine the temporal and spatial dependencies between multivariate time series through sequence modeling and convolution layer technology. In addition, with the continuous expansion of computing resources, spatio-temporal graph neural networks and attention-based Transformer models further enhance the utilization of context information and the modeling of spatial relationships, making deep learning models widely used.

[0005] Although deep learning models have achieved satisfactory results, the evaluation data sets they use are usually complete and cleaned. However, in real life, the multivariate time series prediction task usually faces an important challenge, that is, the problem of data missing, which is usually caused by the failure of the data collector. When the data collector is affected by bad weather or natural disasters, according to the time and location of the natural disasters, the data collector will usually fail at different times, which causes the data to be randomly missing in the time and space dimensions. In summary, the abnormality of the data collector will cause the multivariate time series to have a non-complete problem, and the sensor that fails will not output valid values for a period of time or multiple periods of time, which will destroy the original time series. Moreover, when some sensors fail, only the data collected by the normal data collectors in the obtained multivariate time series has data, and the data corresponding to the failed sensors is 0.

[0006] When the existing model is applied to a data set with a large data missing rate, the model is usually difficult to produce satisfactory prediction results, in addition, the data missing rate usually changes over time, and the existing work usually needs to train multiple models for different missing rates, which limits the robustness of the model.

[0007] It should be noted that the background art is only used to introduce the relevant information of the present application, so as to help understand the technical solutions of the present application, but does not mean that the relevant information must be the prior art. In the absence of evidence that the relevant information has been disclosed before the filing date of the present application, the relevant information should not be regarded as prior art. SUMMARY

[0008] Therefore, the purpose of the present application is to overcome the defects of the above-mentioned prior art, and to provide a learning method for non-complete multivariate time series prediction.

[0009] The purpose of the present application is achieved by the following technical solutions:

[0010] According to a first aspect of the present application, a method for training a student model is provided, the method comprising: obtaining a first training set comprising a plurality of original samples without missing treatment and a label corresponding to each original sample, the original sample being a historical observation value, and the label being a true value observed after the corresponding historical observation value; obtaining a teacher model trained using the original samples and the labels, the teacher model comprising a feature extractor for extracting a teacher representation from the original samples and a regression layer for obtaining a teacher prediction result according to the teacher representation; performing missing treatment on each original sample in the first training set using a plurality of missing rates to obtain a corresponding missing sample, all missing samples and the corresponding labels forming a second training set; obtaining a student model comprising a feature extractor for extracting a student representation from the missing samples and a regression layer for obtaining a student prediction result according to the student representation; and training the student model using the second training set, the teacher representation and the teacher prediction result, wherein during the training, parameters of the student model are updated based on a preset total loss function to obtain a trained student model, the total loss function comprising a weighted sum of the following loss functions: a representation loss function for determining a loss between the teacher representation and the student representation; a prediction result loss function for determining a loss between the teacher prediction result and the student prediction result; a contrast loss function for determining a contrast loss between the student representations of different missing samples corresponding to the same original sample and between the student representations of missing samples corresponding to different original samples; and an absolute error loss function for determining a loss between the student prediction result and the corresponding label.

[0011] In some embodiments of the present application, the total loss function is:

[0012]

[0013] wherein, represents the representation loss function, represents the prediction result loss function, represents the contrast loss function, represents the absolute error loss function, and α, β, γ and λ respectively represent the weight values corresponding to the representation loss function, the prediction result loss function, the contrast loss function and the absolute error loss function.

[0014] In some embodiments of the present application, the representation loss function is:

[0015]

[0016] wherein, N represents the number of original samples, m represents the number of missing rates, represents the teacher representation of the nth original sample, represents the student representation of the ith missing sample corresponding to the nth original sample, and Mean(·) represents the mean value of a tensor.

[0017] In some embodiments of the present application, the predicted result loss function is:

[0018]

[0019] wherein N represents the number of original samples, m represents the number of missing rates, represents the teacher predicted result of the nth original sample, represents the student predicted result of the ith missing sample corresponding to the nth original sample, and Mean(·) represents the mean of a tensor.

[0020] In some embodiments of the present application, the contrast loss is determined in the following manner: student representations corresponding to all missing samples are obtained, and the student representations of all missing samples corresponding to each missing rate are taken as a representation sample set; each two representation sample sets form a sample group, and an anchor sample, a positive sample and a negative sample are determined from each sample group, wherein the positive sample and the anchor sample are student representations of missing samples derived from the same original sample but different missing rates, and the negative sample and the anchor sample are student representations of missing samples derived from different original samples; the sub-loss of each anchor sample is calculated according to the positive sample and the negative sample corresponding to the anchor sample; the sub-contrast loss of each sample group is determined according to the sub-loss of all samples in each sample group; and the contrast loss is determined based on a preset contrast loss function according to the sub-contrast loss of all sample groups.

[0021] In some embodiments of the present application, the contrast loss function is represented as:

[0022]

[0023] wherein m represents the number of missing rates, PQ represents a sample group formed by the Pth representation sample set and the Qth representation sample set, and L PQ represents the sub-contrast loss of the sample group PQ.

[0024] In some embodiments of the present application, when the student model is a weather prediction model, the historical observation values are multivariate time series composed of weather data at different locations, and the labels are real weather values observed after the historical observation values; or when the student model is a traffic flow prediction model, the historical observation values are multivariate time series composed of traffic flow data at multiple intersections, and the labels are real traffic flow data values observed after the historical observation values.

[0025] According to the second method of the present application, a method for implementing prediction by using the trained student model obtained by the method of the first aspect of the present application is provided, which comprises: obtaining historical observation values; inputting the historical observation values into the trained student model for prediction to obtain a prediction result.

[0026] Compared with the prior art, the present application has the advantages of:

[0027] By setting multiple missing rates to process the original samples to be missing, the missing data collected in the actual application scenario is simulated, so that the representation loss function determined according to the representation of the original sample and the representation of the missing sample, the prediction result loss function determined by the prediction result of the original sample and the student prediction result of the missing sample, the contrast loss function determined by the representation of the missing sample, and the absolute error loss function determined by the prediction result of the missing sample and the corresponding label are weighted and summed to obtain the total loss function. The difference between the representation of the missing sample and the representation of the original sample, and the difference between the prediction result of the missing sample and the prediction result of the original sample can be reduced, and the distance between the missing samples of different missing rates belonging to the same original sample can be reduced, and the distance between the missing samples not belonging to the same original sample can be increased. The student model updated according to the total loss function has better generalization ability, and the prediction result is more accurate whether it is for complete data prediction or missing data prediction, especially for missing data prediction, the prediction accuracy of the model for data under different missing rates is good, and the prediction performance of the model is more stable. The problem that the existing model cannot predict multiple missing rate data and has low prediction accuracy is solved, and the model obtained by the method has more practical application value. BRIEF DESCRIPTION OF DRAWINGS

[0028] The embodiments of the present application will be further described below with reference to the accompanying drawings, in which:

[0029] Figure 1 The flowchart of a method for training a student model according to an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below through specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0031] As mentioned in the background section, although the existing method achieves satisfactory results, the existing method is aimed at relatively complete and cleaned data. In real life, due to the influence of environmental factors on the data collector, random missing of data occurs in time and space dimensions. When the existing model is applied to a data set with a high missing rate, the model usually has difficulty in producing satisfactory prediction results. In addition, the missing rate of data usually changes over time, and the existing work usually needs to train multiple models for different missing rates, that is, set multiple missing rates, collect samples for each missing rate and train a model separately, and then detect the missing rate of the input data and input it into the model corresponding to the missing rate for prediction, which limits the robustness of the model.

[0032] To solve the above problems, the present application proposes a method for training a student model, which obtains a teacher model trained by using original data without missing processing and labels, extracts the original data without missing processing by using the teacher model to obtain teacher representation and teacher prediction results; extract the missing samples by using the student model to obtain student representation and student prediction results, wherein the missing samples are obtained by missing processing the original samples by using multiple missing rates to simulate the missing data collected in actual application scenarios; the teacher representation and the teacher prediction results extracted by the teacher model are transmitted to the student model as knowledge, and the teacher representation, the teacher prediction results, the student representation and the student prediction results are used to train the student model. In the training, the representation loss function determined by the teacher representation and the student representation, the prediction result loss function determined by the teacher prediction result and the student prediction result, the contrast loss function determined by the student representation, and the absolute error loss function determined by the student prediction result and the corresponding label are weighted and summed to obtain a total loss function to update the parameters of the student model. In this way, the trained student model not only has high prediction accuracy and robustness for non-missing data, but also has high prediction accuracy and robustness for missing data, solving the problem that the existing model cannot predict multiple missing rate data and has low prediction accuracy, making the model obtained by the method more practical.

[0033] In order to better understand the present application, the method proposed by the present application is applied to the field of traffic flow prediction. The following describes the technical solution of the present application from four aspects of training sample, model structure, training process and application scenario.

[0034] I. Training sample

[0035] Before training the student model, samples need to be organized. According to one embodiment of the present invention, a first training set is obtained, which includes multiple unprocessed original samples and labels corresponding to each original sample. The original samples are historical observations, and the labels are the actual values ​​observed after the corresponding historical observations. The historical observations are multivariate time series. Illustratively, the historical observations are multivariate time series composed of traffic flow data from multiple intersections, and the labels are the actual values ​​of the traffic flow data observed after the corresponding historical observations. The traffic flow data includes: traffic volume, speed data, occupancy rate, congestion data, and / or vehicle type. For example, the original samples in the first training set are X = {x1, x2, x3, x4}, and the corresponding labels are Y = {y1, y2, y3, y4}, representing unprocessed original samples from one intersection. The corresponding label is the actual value observed after the original value, for example: etc., where a i This can be represented as the traffic flow at time i, b i This can be represented as the velocity data at the i-th time point, c i This can be represented as the occupancy rate at the i-th time. The above is just an example, and the present invention is not limited thereto.

[0036] According to one embodiment of the present invention, each original sample in a first training set is subjected to missing data processing using multiple missing data rates to obtain corresponding missing samples. All missing samples and their corresponding labels constitute a second training set. Setting multiple missing data rates to process each original sample in the first training set is used to simulate missing data collected in real-world application scenarios. Illustratively, for an unprocessed original sample... The original sample was processed using three missing values ​​(25%, 50%, and 75%) to obtain the missing samples corresponding to the original sample: and Therefore, the missing samples in the second training set are the original samples with three different missing rates. The above is merely an example, and the invention is not limited thereto.

[0037] According to one embodiment of the present invention, the original samples can be obtained from an existing traffic flow dataset (e.g., the PEMS04 dataset). A sliding window (of size N+M) is used to determine the original samples and corresponding labels from the existing dataset, and historical observations at N times are used to predict observations at one future time (M=1) or multiple times (M≥2). Illustratively, a multivariate time series in the existing dataset is... Set the sliding window size to 8+1, and use this sliding window to truncate X1 to obtain the original sample: The corresponding tags are Or, the original sample is: The corresponding label is That is, the observation value of the future 1 time is predicted by using the historical observation values of the 8 times; the size of the sliding window is set to 4+4, the sliding window is used to intercept X1, and the original sample is obtained as: The corresponding label is Or, the original sample is: The corresponding label is That is, the observation values of the future 4 times are predicted by using the historical observation values of the 4 times. The above is only an example, and it should be understood that those skilled in the art can modify the size of the sliding window to obtain more embodiments.

[0038] II. Model structure

[0039] The present application provides a teacher model and a student model, which are traffic flow prediction models for predicting traffic flow, the teacher model comprising a feature extractor for extracting traffic flow teacher representation from original samples without missing treatment and a regression layer for obtaining traffic flow teacher prediction results according to traffic flow teacher representation; the student model comprising a feature extractor for extracting traffic flow student representation from missing samples and a regression layer for obtaining traffic flow student prediction results according to traffic flow student representation.

[0040] According to one embodiment of the present application, the feature extraction layer for extracting traffic flow representation can be composed of an embedding layer and an L-layer MLP with residual connection, the embedding layer being used to convert traffic flow data into high-dimensional hidden representation, and the L-layer MLP being used to mine high-dimensional hidden representation to obtain traffic flow representation.

[0041] According to one example of the present application, for a traffic flow data x, the embedding layer is used to convert it to obtain high-dimensional hidden representation H=PC(x), wherein FC(·) represents a full connection layer; the L-layer MLP mines H to obtain the representation H of the sample. l+1 =FC(Relu(FC(H l )))+H l , wherein H l represents the traffic flow representation extracted by the lth layer MLP, Relu(·) represents an activation function, and finally a regression layer is used to predict according to the representation H L to obtain the traffic flow prediction result Y=FC(H L ).

[0042] III. Training process

[0043] Based on the above-organized training samples, the parameters of the student model are updated by using the total loss function to obtain a traffic flow prediction model for actual application scenarios.

[0044] According to one embodiment of the present application, the student model is trained by using the second training set, the traffic flow teacher representation and the traffic flow teacher prediction result, wherein during the training, the parameters of the student model are updated based on a preset total loss function to obtain a trained student model, the total loss function is determined according to a weighted sum of the following losses: a representation loss function, a prediction result loss function, a comparison loss function and an absolute error loss function; wherein: the representation loss function is used to determine the loss between the traffic flow teacher representation and the traffic flow student representation; the prediction result loss function is used to determine the loss between the traffic flow teacher prediction result and the traffic flow student prediction result; the comparison loss function is used to determine the comparison loss between the traffic flow student representations of different missing samples corresponding to the same original sample and the traffic flow student representations of missing samples corresponding to different original samples; and the absolute error loss function is used to determine the loss between the traffic flow student prediction result and the corresponding label.

[0045] According to one embodiment of the present application, the total loss function can be expressed as:

[0046]

[0047] wherein, represents the representation loss function, represents the prediction result loss function, represents the comparison loss function, represents the absolute error loss function, and α, β, γ and λ respectively represent the weight values corresponding to the representation loss function, the prediction result loss function, the comparison loss function and the absolute error loss function.

[0048] In order to enable the student model to have good prediction accuracy for missing data and non-missing data, the present application proposes a representation loss function determined according to sample representation and a prediction result loss function determined according to sample prediction result. According to one embodiment of the present application, the representation loss function is determined according to the sub-representation loss functions of all original samples, and the sub-representation loss function of each original sample is determined according to the traffic flow teacher representation of each original sample and the traffic flow student representation of the missing sample corresponding to the original sample, and the sub-representation loss function of each original sample can be expressed as:

[0049]

[0050] wherein m represents the number of missing rates, represents the traffic flow teacher representation of the nth original sample, represents the traffic flow student representation of the ith missing sample corresponding to the nth original sample, and Mean(·) represents the mean value of a tensor. Illustratively, the representation loss function can be expressed as: wherein N denotes the number of original samples. More specifically, the representation loss function can be represented as:

[0051]

[0052] wherein N denotes the number of original samples, m denotes the number of missing rates, denotes the traffic flow teacher representation of the nth original sample, denotes the traffic flow student representation of the ith missing sample corresponding to the nth original sample, and Mean(·) denotes the mean of a tensor. The technical scheme of this embodiment can at least achieve the following beneficial technical effects: the student model is trained by using the representation of the data without missing processing and the representation of the data with missing processing, the difference between the representations generated by the student model and the teacher model is reduced, the representation of the missing sample obtained by the student model is closer to the representation of the complete data, and thus the prediction ability of the student model is improved.

[0053] According to an embodiment of the present application, the prediction result loss is determined based on the sub-prediction result losses of all original samples, the sub-prediction result loss of each original sample is determined based on the traffic flow teacher prediction result of each original sample and the traffic flow student prediction result of the missing sample corresponding to the original sample, and the sub-prediction result loss function of each original sample can be represented as:

[0054]

[0055] wherein m denotes the number of missing rates, denotes the traffic flow teacher prediction result of the nth original sample, denotes the traffic flow student prediction result of the ith missing sample corresponding to the nth original sample, and Mean(·) denotes the mean of a tensor. Illustratively, the prediction result loss function can be represented as: More specifically, the prediction result loss function can be represented as:

[0056]

[0057] wherein N denotes the number of original samples, m denotes the number of missing rates, denotes the traffic flow teacher prediction result of the nth original sample, The traffic flow student prediction result of the i th missing sample corresponding to the n th original sample is represented. The technical scheme of this embodiment can at least achieve the following beneficial technical effects: the prediction results of the data without missing processing and the prediction results of the data after missing processing are used for student model training, the difference between the prediction results of the student model and the teacher model is reduced, the prediction result of the student model for the missing sample is closer to the prediction result of the complete data, and the prediction ability of the student model is improved.

[0058] Considering that the missing rate of historical observation values in a real scene is not fixed, in order to enhance the prediction accuracy of the student model for data under different missing rates, the application proposes a contrast loss function. According to an embodiment of the application, the contrast loss function is determined according to the following manner: the traffic flow student representation corresponding to all missing samples is obtained, and the traffic flow student representation of all missing samples corresponding to each missing rate is taken as a representation sample set; each two representation sample sets form a sample group, and an anchor sample, a positive sample and a negative sample are determined from each sample group, wherein the positive sample and the anchor sample are traffic flow student representations corresponding to missing samples derived from the same original sample but with different missing rates, and the negative sample and the anchor sample are traffic flow student representations corresponding to missing samples derived from different original samples; the sub-loss of each anchor sample is calculated according to the positive sample and the negative sample corresponding to the anchor sample; the sub-contrast loss of each sample group is determined according to the sub-loss of all samples in each sample group; and the contrast loss is determined according to the sub-contrast loss of all sample groups. The technical scheme of this embodiment can at least achieve the following beneficial technical effects: the contrast loss can narrow the distance between each anchor sample and the student representation of the same original sample to which the anchor sample belongs, and widen the distance between each anchor sample and the student representation of the different original sample to which the anchor sample does not belong, so that the student model trained according to the contrast loss can have good prediction accuracy for data under different missing rates.

[0059] According to an embodiment of the application, the sub-loss of each anchor sample can be calculated in the following manner:

[0060]

[0061] wherein z p represents the p th anchor sample, z q represents the q th positive sample of the anchor sample, z k represents the k th negative sample of the anchor sample, t represents a temperature parameter, and sim(·) represents a cosine similarity function.

[0062] Illustratively, the sub-contrast loss of each sample group can be determined according to the sub-loss of all samples in each sample group as follows:

[0063]

[0064] wherein PQ represents a sample group composed of the Pth representation sample set and the Qth representation sample set, and N represents the number of samples in the representation sample set.

[0065] Illustratively, the contrast loss function determined according to the sub-contrast loss of all sample groups can be represented as:

[0066]

[0067] wherein m represents the number of missing rates, PQ represents a sample group composed of the Pth representation sample set and the Qth representation sample set, and L PQ represents the sub-contrast loss of the sample group PQ.

[0068] According to another embodiment of the present application, in order to reduce the complexity of calculation, when calculating the contrast loss, a fully connected layer can be used to reduce the dimension of the traffic flow student representation extracted by the feature extraction layer of the student model to obtain traffic flow reduced representation corresponding to the traffic flow student representation, and all traffic flow reduced representations corresponding to each missing rate are taken as a representation sample set; each two representation sample sets are taken as a sample group, the anchor sample is determined from each sample group, and the traffic flow reduced representation belonging to the same original sample as the anchor sample in the sample group is taken as a positive sample, and the traffic flow reduced representation not belonging to the same original sample as the anchor sample is taken as a negative sample; the sub-loss of each anchor sample is calculated according to the positive sample and the negative sample of the anchor sample; the sub-contrast loss of each sample group is determined according to the sub-loss of all samples in each sample group; and the contrast loss is determined according to the sub-contrast loss of all sample groups. Illustratively, in the sub-loss of each anchor sample given above, Z p , Z q , z k represents the traffic flow reduced representation of the original sample.

[0069] In addition to the loss function described above, the present application also combines the absolute error loss function determined by the student prediction result and the corresponding label to update the parameters of the model. According to one embodiment of the present application, the absolute error loss function can be represented as:

[0070]

[0071] wherein, represents the label of the nth original sample, represents the traffic flow student prediction result of the ith missing sample corresponding to the nth original sample.

[0072] According to one example of the present application, as Figure 1As shown, a workflow schematic diagram of a method for training a student model is shown, the existing unprocessed original samples x1, x2, x3, the corresponding labels are y1, y2, y3, the original samples are processed by using three missing rates (25%, 50% and 75%) to obtain the missing samples x 11 、x 12 、x 13 、x 21 、x 22 、x 23 、x 31 、x 32 、x 33 , the traffic flow teacher representation and the traffic flow teacher prediction result of the original sample are extracted by using the teacher model, the traffic flow teacher representation is: The traffic flow teacher prediction result is: The traffic flow student representation and the traffic flow student prediction result of the missing sample are extracted by using the student model, the traffic flow student representation is: The traffic flow student prediction result is: According to the traffic flow student representation, the representation sample set is: The representation loss is calculated according to the traffic flow teacher representation and the traffic flow student representation, the prediction result loss is calculated according to the traffic flow teacher prediction result and the traffic flow student prediction result, the comparison loss under different missing rates is calculated according to the traffic flow student representation, the absolute error loss is calculated according to the traffic flow student prediction result and the corresponding label, the total loss is obtained by weighted summation of the representation loss, the prediction result loss, the comparison loss and the absolute error loss, and the parameters of the student model are updated by using the total loss. The representation loss is: The prediction result loss is: The comparison loss is: The absolute error loss is:

[0073] According to an embodiment of the present application, the teacher model is trained according to the original sample without missing processing and the label. More specifically, the teacher model can be trained by using the original data without missing processing and the label, or can be trained by using the first training set. The training process is as follows: the original sample without missing processing is input into the teacher model to obtain the corresponding prediction result; the absolute error loss is calculated according to the prediction result and the corresponding label, and the parameters of the teacher model are updated according to the absolute error loss. The absolute error loss can be expressed as:

[0074]

[0075] wherein, represents a label of the nth original sample, represents a traffic flow teacher prediction result of the nth original sample.

[0076] IV. Application scenarios

[0077] In an actual application scenario, historical observation values are obtained, the historical observation values are input into the trained student model obtained by the above method, and corresponding prediction results are obtained.

[0078] According to an embodiment of the present application, historical observation values are obtained, the historical observation values are multivariate time series composed of traffic flow data at different positions; the historical observation values are input into the trained student model obtained in the foregoing embodiments, and traffic flow prediction results are obtained.

[0079] It should be understood that the method proposed by the present application can be applied not only to the traffic flow prediction field provided in the foregoing embodiments, but also to the meteorological prediction field. The data in the traffic flow prediction field given in the foregoing embodiments is replaced by data in the meteorological prediction field to obtain embodiments related to meteorological prediction. Wherein:

[0080] In the training sample part, the multivariate time series composed of traffic flow data of multiple intersections is replaced by a multivariate time series composed of meteorological data at different positions, and correspondingly, the label is replaced by the true value of the observed meteorological data after the corresponding historical observation values, wherein the meteorological data includes temperature, air pressure, humidity, wind speed, wind direction, precipitation, cloud cover and / or visibility. Replace the existing traffic flow data set (such as the PEMS04 data set) with the existing meteorological data set (such as the JenaClimate data set), replace one intersection with one position, replace a i may be replaced by temperature, b i may be replaced by air pressure, c i may be replaced by humidity.

[0081] In the model structure part, the traffic flow teacher representation extracted by the teacher model is replaced by the meteorological teacher representation, and the traffic flow teacher prediction result is replaced by the meteorological teacher prediction result; the traffic flow student representation extracted by the student model is replaced by the meteorological student representation, and the traffic flow student prediction result is replaced by the meteorological student prediction result.

[0082] In the training process part, the traffic flow prediction model is replaced by a weather prediction model, the traffic flow teacher representation is replaced by a weather teacher representation, the traffic flow teacher prediction result is replaced by a weather teacher prediction result, the traffic flow student representation is replaced by a weather student representation, the traffic flow student prediction result is replaced by a weather student prediction result, and the traffic flow reduced dimension representation is replaced by a weather reduced dimension representation.

[0083] In the application scenario part, the multivariate time series composed of traffic flow data of multiple intersections is replaced by a multivariate time series composed of weather data at different positions, and the traffic flow prediction result is replaced by a weather prediction result.

[0084] Based on the method for training a student model given in the above embodiment, in order to make the method better applied, the present application gives a corresponding training device, the functional interface and details of the device are shown in Table 1:

[0085] Table 1 Functional interface of training device

[0086]

[0087] In order to show the prediction ability of the student model obtained by the method for training a student model of the present application, comparative experiments are carried out on the present application and the prior art. In the experiment, the prediction error of the model on different missing rate existing data sets is tested, and the following experimental results are used to show the technical effect of the present application.

[0088] The schemes used in the comparative experiment include:

[0089] (1) Using the prediction result and the label to train the MLP to obtain MLP1;

[0090] (2) Using the prediction result and the label to train the MLP including the interpolation model to obtain MLP2;

[0091] (3) Using the training method proposed in the present application to train the MLP to obtain MLP3.

[0092] The data set used in the comparative experiment is a traffic flow data set (PEMS04). The data set is processed with missing rates of 25%, 50%, 75% and 90% to obtain missing samples. The models obtained by the above three schemes are used to predict the missing samples with different missing rates, and the prediction error results are shown in Table 2:

[0093] Table 2 Prediction error results of models obtained by prior art on samples with different missing rates

[0094]

[0095] According to Table 2, no matter which missing rate data, the prediction error of the model (MLP3) obtained by the method proposed in the application is lower than the prediction error of the model in other schemes, and as the missing rate is greater, the prediction error in each scheme increases, but the prediction error of MLP3 increases less, the prediction error of MLP1 and MLP2 increases more, and at a missing rate of 90%, the prediction error of MLP1 is nearly 6.7% higher than that of MLP3, and the prediction error of MLP2 is nearly 4.5% higher than that of MLP3. Therefore, according to the experimental results, it can be seen that the model obtained by the method proposed in the application has lower prediction error for data under different missing rates, although the prediction error increases as the missing rate increases, but the increase is small, which can reflect the stability of the model.

[0096] In summary, the method for training a student model proposed in the application has the following advantages: by setting multiple missing rates to perform missing processing on the original sample to simulate the missing data collected in the actual application scenario, the total loss function obtained by weighting and summing the representation loss function determined according to the representation of the original sample and the representation of the missing sample, the prediction result loss function determined by the prediction result of the original sample and the student prediction result of the missing sample, the contrast loss function determined by the representation of the missing sample, and the absolute error loss function determined by the prediction result of the missing sample and the corresponding label, not only can reduce the difference between the representation of the missing sample and the representation of the original sample, and the difference between the prediction result of the missing sample and the prediction result of the original sample, but also can reduce the distance between the missing samples belonging to the same original sample, and increase the distance between the missing samples not belonging to the same original sample; the student model updated according to the total loss function has better generalization ability, whether it is for complete data prediction or missing data prediction, the prediction result is more accurate, especially for missing data prediction, it has good prediction accuracy for data under different missing rates, and the prediction performance of the model is more stable, solving the problem that the existing model cannot predict multiple missing rate data and the prediction accuracy is not high, making the model obtained by the method more practical and valuable.

[0097] It should be noted that although the above describes the steps in a specific order, it does not mean that the steps must be performed in the above specific order, in fact, some of the steps can be executed concurrently, or even in a different order, as long as the desired function can be achieved.

[0098] The application can be a system, a method and / or a computer program product. The computer program product can include a computer readable storage medium having computer readable program instructions loaded thereon for causing a processor to implement various aspects of the application.

[0099] A computer readable storage medium can be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, semiconductor, or any other suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves.

[0100] Having described above several embodiments, now will be described a number of modifications and alternatives. Such description is included to provide for a more complete understanding of the various embodiments and is not intended in any way to limit the scope of possibilities. Many modifications in addition to those described above can be made by one skilled in the relevant art without departing from the scope and spirit of the described embodiments. The scope of the various embodiments disclosed herein cover all technical solutions falling within the principles and their practical applications or improvements to the art, as well as enabling other ordinary skilled in the art to understand the various embodiments disclosed herein.

Claims

1. A method of training a student model, the method comprising: The method comprises: obtaining a first training set comprising a plurality of original samples without missing treatment and labels corresponding to each original sample, the original sample being a historical observation value, and the label being a true value observed after the corresponding historical observation value; obtaining a teacher model trained using the original samples and the labels, the teacher model comprising a feature extractor for extracting a teacher representation from the original sample and a regression layer for obtaining a teacher prediction result according to the teacher representation; obtaining a plurality of missing rates, and performing missing treatment on each original sample in the first training set to obtain a corresponding missing sample, all missing samples and corresponding labels forming a second training set; obtaining a student model comprising a feature extractor for extracting a student representation from the missing sample and a regression layer for obtaining a student prediction result according to the student representation; training the student model using the second training set, the teacher representation and the teacher prediction result, wherein during the training, parameters of the student model are updated based on a preset total loss function to obtain a trained student model, the total loss function comprising a weighted sum of the following loss functions: a representation loss function for determining a loss between the teacher representation and the student representation, the representation loss function being: ； wherein, denotes the number of original samples, denotes the number of missing rates, denotes the teacher representation of the th original sample, denotes the student representation of the th original sample corresponding to the th missing sample, denotes the mean of the tensor; a prediction result loss function for determining a loss between the teacher prediction result and the student prediction result, the prediction result loss function being: ； wherein, denotes the number of original samples, denotes the number of missing rates, denotes the teacher prediction result of the th original sample, denotes the th missing sample corresponding to the th original sample, denotes the mean of the tensor; a contrast loss function for determining a contrast loss between student representations of different missing samples corresponding to the same original sample and between student representations of missing samples corresponding to different original samples, the contrast loss function being: ； wherein, a quantity representing a loss rate, a quantity representing a loss rate of a sample group composed of the i-th feature sample set and the j-th feature sample set, a quantity representing a loss rate of a sample group composed of the i-th feature sample set and the j-th feature sample set, a quantity representing a loss rate of a sample group composed of the i-th feature sample set and the j-th feature sample set. an absolute error loss function for determining a loss between the student prediction result and the corresponding label.

2. The method of claim 1, wherein, The total loss function is: ； wherein, represents a characterization loss function, represents a prediction result loss function, represents a contrast loss function, represents an absolute error loss function, , , and respectively represent weights corresponding to the characterization loss function, the prediction result loss function, the contrast loss function, and the absolute error loss function.

3. The method of claim 2, wherein, The contrast loss is determined in the following manner: obtaining student representations corresponding to all missing samples, and regarding the student representations of all missing samples corresponding to each missing rate as a representation sample set; grouping each two representation sample sets into a sample group, and determining an anchor sample, a positive sample and a negative sample from each sample group, wherein the positive sample and the anchor sample are student representations of missing samples derived from the same original sample but with different missing rates, and the negative sample and the anchor sample are student representations of missing samples derived from different original samples; calculating a sub-loss of each anchor sample according to the positive sample and the negative sample corresponding to the anchor sample; determining a sub-contrast loss of each sample group according to the sub-losses of all samples in the sample group; determining the contrast loss based on the preset contrast loss function and the sub-contrast losses of all sample groups.

4. The method according to any one of claims 1-3, wherein: when the student model is a weather prediction model, the historical observation value is a multivariate time series composed of weather data at different locations, and the label is a weather true value observed after the corresponding historical observation value; or when the student model is a traffic flow prediction model, the historical observation value is a multivariate time series composed of traffic flow data at a plurality of intersections, and the label is a traffic flow data true value observed after the corresponding historical observation value.

5. A method for implementing prediction using the trained student model obtained by the method according to any one of claims 1-4, comprising: obtaining historical observation values; inputting the historical observation values into the trained student model for prediction to obtain a prediction result.

6. A computer readable storage medium characterized by, A computer program is stored thereon, and the computer program is executable by a processor to implement steps of the method of any one of claims 1 to 5.

7. An electronic device, comprising: comprising: one or more processors; and a memory, wherein the memory is configured to store executable instructions; the one or more processors are configured to implement steps of the method of any one of claims 1 to 5 via execution of the executable instructions.

Images