Method for analyzing a multivariate time series and associated computer program output

Abstract

Method for analyzing a multivariate time series and associated computer program product. The method (100) for analyzing a multivariate time series, implemented by computer, comprises the steps of: - acquiring (110) a multivariate time series, the multivariate time series containing missing data, the multivariate time series being represented by a matrix (M); - selecting (120), in the matrix (M), a reference matrix (MRef) containing no missing data, by implementing a heuristic selection algorithm; - testing (130) a plurality of techniques for imputation of missing data on the reference matrix and selecting the optimal technique from among the plurality of techniques for imputation of missing data; - correcting (140) the multivariate time series by calculating imputed values ​​for the missing data by implementing the optimal technique;and, - analyze (150) the multivariate time series thus corrected. Figure for the abstract: Figure 1;

Description

Title of the invention: Method for analyzing a multivariate time series and associated computer program product

[0001] The present invention relates to methods for analyzing multivariate time series comprising missing data, in particular analysis methods comprising a step of correction by imputation of data from the analyzed multivariate time series.

[0002] The present invention is applicable to the management of data from many types of systems, such as processors, servers, satellites, sensor networks, etc.

[0003] For example, it applies to telemetry monitoring of energy consumption on distribution networks, internet traffic management, road traffic monitoring, seismic monitoring of a geographical area, etc.

[0004] A sensor network allows data to be acquired over time. At each sampling step, each sensor measures the instantaneous value of a variable. A multivariate time series brings together the instantaneous values ​​of the different variables measured by each of the sensors in the network, for each sampling step of a time interval. A multivariate time series can be represented by a matrix whose different columns correspond to the different variables measured by the sensors over time, and whose different rows correspond to the different sampling steps subdividing the acquisition interval.

[0005] A time series provides information on the state of a system (for example the state of road traffic during the acquisition interval) and makes it possible to estimate the future state of this system (for example predict the state of road traffic at a future time).

[0006] When acquiring data by a sensor network, data may be missing due to, for example, the failure of a sensor, the failure of the means for transmitting data between a sensor and the collection system (for example a centralized data server), or the failure of the data collection system (such as a breakdown of the data server).

[0007] Thus, a multivariate time series generally includes a significant percentage of missing data.

[0008] However, many data analysis methods, including certain machine learning methods, require complete data sets.

[0009] It is therefore necessary to impute the missing data by an imputed value. This imputed value must be as probable as possible. Indeed, the imputed values to missing data directly influence the quality of the analysis process applied to the multivariate time series once corrected for its missing data.

[0010] Many methods of imputation of missing data exist, such as for example imputation by an average value, by linear interpolation, by automatic learning (KNN algorithm, recurrent neural network, compressed acquisition method or "compressive sensing" in English).

[0011] However, for a particular multivariate time series, it is difficult to establish a priori which is the best imputation method to use. Indeed, the best method depends on the nature of the data in the series considered, such as the correlations between its different variables, the temporal correlations between its data, the distribution of missing data, etc.

[0012] The aim of the present invention is to solve this problem.

[0013] For this purpose, the invention relates to a method for analyzing a multivariate time series, the method being implemented by computer, the method comprising the steps consisting of:

[0014] - acquire a multivariate time series, the multivariate time series with missing data, the multivariate time series being represented by a matrix;

[0015] - select, in the matrix, a reference matrix containing no missing data, by implementing a heuristic selection algorithm;

[0016] - testing a plurality of missing data imputation techniques on the reference matrix and select the optimal technique from among the plurality of missing data imputation techniques;

[0017] - correct the multivariate time series by calculating imputed values ​​for the missing data by implementing the optimal technique; and,

[0018] - analyze the multivariate time series thus corrected.

[0019] According to particular embodiments, the method comprises one or more of the following characteristics, taken in isolation or in all technically possible combinations:

[0020] - the heuristic selection algorithm is an iterative process consisting of each iteration to:

[0021] a) aggregating a column of the matrix (M) representing the multivariate time series, to an intermediate group of columns to obtain a candidate group of columns;

[0022] b) evaluating a score of the candidate column group, the score of a column group implementing a metric indicative of the size of at least one candidate reference matrix that can be extracted from the column group considered; and,

[0023] c) iterating steps a) and b) if the score of the candidate column group is greater than score of the intermediate column group, the candidate column group retained in an iteration being the intermediate column group of the following iteration;

[0024] - the execution of the heuristic selection algorithm allows, at each iteration, to retain a plurality of candidate column groups;

[0025] - testing a plurality of missing data imputation techniques consists of:

[0026] - determine a statistic of the missing data in the matrix (M) representing the multivariate time series;

[0027] - delete data from the reference matrix (MRef) according to the sta determined tistics, to develop a test matrix (MTest); and, for each imputation technique of the plurality of imputation techniques,

[0028] - apply the considered imputation technique on the test matrix to obtain a corrected reference matrix; and,

[0029] - compare the reference matrix and the corrected reference matrix in order to evaluate the imputation technique considered.

[0030] - the acquisition step consists of acquiring, over an acquisition time interval, the values ​​measured, at each sampling step, by a plurality of sensors of a sensor network, each variable of the multivariate time series being associated with a single sensor of the sensor network;

[0031] - the analysis method is implemented for the analysis of road traffic, in which each value measured by a sensor corresponds to a number of vehicles per unit of time detected at a location where the sensor is installed;

[0032] - the step of analyzing the corrected multivariate time series consists of estimating a future state of road traffic.

[0033] The invention also relates to a computer program comprising software instructions which, when executed by a computer, implement a method for analyzing a multivariate time series as defined above.

[0034] The invention and its advantages will be better understood on reading the detailed description which follows of a particular embodiment, given solely by way of non-limiting example, this description being made with reference to the appended drawings in which:

[0035] [Fig-1] [Fig.l] is a block representation of a mode of realization preferred implementation of the missing data imputation method according to the invention;

[0036] [Fig.2] [Fig.2] is an illustration of the results of the treatments carried out during the implementation of the method of [Fig.l];

[0037] [Fig.3] [Fig.3] is a representation of an algorithm for evaluating a plurality of imputation methods, which is executed during the implementation of the method of [Fig.l]; and,

[0038] [Fig.4] [Fig.4] is an illustration, using the example of [Fig.2], of the calculation of a score on a group of columns.

[0039] In the following description, the term “series” is used for “multivariate time series”.

[0040] Generally speaking, since the properties of a particular series are difficult, or even impossible, to determine a priori, the analysis method according to the present invention provides a correction step during which different imputation techniques are empirically tested.

[0041] For such a test, a complete reference set (i.e. without missing data) is first developed from the series considered.

[0042] Data is then artificially removed from this reference set to obtain a test set.

[0043] In order to be able to extrapolate the test results of an imputation technique from the test set to the entire series, it is necessary to ensure that this test set is representative of all the data in the series. Therefore, a test set that contains sufficient data is required. A test set is also required whose data correspond to a statistical distribution close to that of the data in the series under consideration. Finally, the pattern of missing data in the test set must also be representative of the pattern of missing data in the series under consideration.

[0044] The question of finding the largest possible reference set corresponds to finding, in the series considered, a set of variables sharing a time interval where none of these variables has missing data. Finding the largest possible reference set is a complex problem. To do this, the present invention uses a strategy inspired by beam search algorithms.

[0045] [Fig. 1] represents a preferred embodiment of a method for analyzing a time series according to the invention.

[0046] The analysis method 100 begins with a step 110 of acquiring data from the multivariate time series.

[0047] The acquired multivariate time series groups together the values ​​taken successively during an acquisition time interval, by each variable of a plurality of variables.

[0048] Each of these variables is for example associated with a physical quantity monitored by a sensor.

[0049] For example, in a road traffic management application, we consider a network of identical and synchronized sensors.

[0050] Each sensor is capable of detecting the passage of a vehicle. These are, for example, sensors of the laser beam type between a transmitting cell and a receiving cell. A short interruption in the reception of the laser beam is indicative of the passage of a vehicle.

[0051] The instantaneous value delivered by a sensor is, for example, the number of vehicles detected per minute.

[0052] The acquisition interval is for example the day.

[0053] The acquired multivariate time series is represented by an initial matrix M.

[0054] Each column of the matrix M is associated with a variable and each row of the matrix M is associated with a sampling step of the sensors.

[0055] The matrix M has C columns, where C is the number of variables considered, i.e. the number of sensors in the sensor network.

[0056] The matrix M has T lines, where T is the ratio of the acquisition interval To to the sampling step.

[0057] The matrix M comprises present data (each present data item being equal to the value actually measured of the variable corresponding to the sampling step considered) and missing data (each missing data item being equal to a predefined value, for example -1 for the case of road traffic management).

[0058] Part A of [Fig.2] illustrates an initial matrix M.

[0059] Step 120 consists of constructing a reference matrix MRef from the initial matrix M. The reference matrix MRef represents a series without missing data.

[0060] The reference matrix MRef corresponds to the gathering of data from several of the columns of the matrix M over a reference time interval TRef included in the acquisition interval To.

[0061] This selection step implements a heuristic algorithm which will be described in more detail below in relation to [Fig. 3]. It makes it possible to construct an optimal reference matrix by simultaneously determining a matrix M' resulting from the permutation of a part of the columns of the initial matrix M and a succession of a part of the rows of the initial matrix M, so as to construct a sub-matrix without missing data, which is none other than the reference matrix MRef sought.

[0062] This is illustrated in part B of [Fig.2].

[0063] It should be noted that, in this operation, the order of the rows of the matrix M (i.e., the chronological order) must be preserved. On the other hand, the order of the columns of the matrix M can be altered. The reference matrix MRef is a submatrix of the matrix M' derived from the matrix M by a permutation of the columns of the matrix M.

[0064] Once the reference matrix MRef has been constructed, a test matrix MTest must be developed to test a plurality of possible imputation techniques.

[0065] For example, step 125 first consists of determining a statistic of the missing data of the initial matrix M.

[0066] Step 125 continues with the construction of the test matrix MTest by removing certain data from the reference matrix MRef while respecting the pre-statistics previously determined.

[0067] The test matrix can thus be obtained from the reference matrix by randomly deleting the data so that the percentage of missing data in the test matrix corresponds to the percentage of missing data in the initial matrix. A more detailed analysis of the distribution of missing data in the initial matrix can be carried out in order to reproduce this distribution in the test matrix. It is also possible to extract patterns of missing data from the initial matrix and apply these patterns to the reference matrix.

[0068] Then, in step 130, each imputation technique is applied to the test matrix MTest to obtain a corrected reference matrix MRefCoiT.

[0069] Finally, the comparison of the corrected reference matrix MRefCoiT and the reference matrix MRef makes it possible to evaluate the effectiveness of the imputation technique tested. For example, a difference is calculated by summing, for each missing data item in the test matrix MTest, the difference between the imputed value and the actual value of this data item.

[0070] This is illustrated by parts C, D and E of [Fig.2].

[0071] Alternatively or in addition, the test and corrected reference matrices are used to calibrate the imputation method tested, i.e. for example to learn the coefficients of a linear combination, adjust the parameters of a neural network, etc.

[0072] Comparing the scores calculated for each of the imputation techniques considered makes it possible to identify the best imputation technique for the multivariate time series considered, acquired in step 110.

[0073] In step 140, this best imputation technique is used on the initial matrix M to obtain a corrected matrix MCoir. The latter represents a corrected multivariate time series, i.e. without missing data.

[0074] Finally, once the missing data have been imputed, in a step 150, an analysis of the corrected multivariate time series is carried out. This analysis consists, for example, in making a prediction of the future evolution of the different variables, that is to say, in the application to road traffic management, in calculating an estimate of the state of the traffic at one or more future instant(s) from the data of the corrected matrix M CoiT. Other types of multivariate data analysis can be carried out, such as for example the detection of anomalies.

[0075] As an example, [Fig.2] illustrates an initial matrix M representing a multivariate time series where each column (numbered from 1 to 17) represents a variable and each row (unnumbered) represents a time step.

[0076] Present data ranges and missing data ranges are represented differently.

[0077] By permuting the columns of the matrix M, we see that the columns of the group rRef={17, 9, 10, 3, 6, 7, 1, 13, 14, 15} share a time interval TRef on which none of these columns has missing data. The set of data in these columns and over this time interval constitutes the reference matrix MRef, that is to say a set of data from the initial series without missing data.

[0078] The size nRef of the reference matrix MRef is the number of elements in this matrix — the interval ^select

[0079] where linterval is the number of rows in the time interval without missing data (i.e., the time interval TRef divided by the sampling step) and Cselect is the number of selected columns (in this example 10).

[0080] Finding the largest reference matrix MRef is a complex problem. Indeed, for an initial matrix M with C columns, if we want to test all possible groups of columns to find the matrix without missing data having the largest size nref, we must test 2AC combinations of columns.

[0081] The algorithm implemented in step 130 is based on a heuristic approach to provide an approximate solution to this problem. The result of implementing this algorithm does not necessarily lead to the best possible reference set, but to one of the best reference sets.

[0082] This algorithm will now be presented in relation to Figures 3 and 4.

[0083] Each column ck (k integer between 1 and C) is represented as the union of the data intervals without missing data. Thus, for example, in part A of Figure 4, which continues the example of Figure 2, column c8 is represented as the union of its three intervals without missing data: [infS sup$] U [inf|, sup^J u [ inf* sup^]-

[0084] A group of columns is represented as the intersection of its columns. Thus, on the left of Figure 4, for example, the group of columns { C8 , C4, C16} is represented. This group therefore corresponds to:

[0085] 'C$' =C' n C',nCi6= ' [^[. siip*] u [111¾ suPi] u ['"('> sups]) n ( [inTi, supi] U su^]) n ( supri] U [inl;6, sup>6] U [ir.lVAup]6] ) sup®]

[0086] The different intervals resulting from this intersection are displayed in the column located on the right of [Fig.4].

[0087] Each of these four intervals of the column group [c8, C4, C|6} corresponds to a candidate reference matrix.

[0088] The size of each candidate reference matrix is ​​obtained by multiplying the size of the time interval characterizing it and the number of columns in the column group. For example, for the column group { cg , C4,} we have four matrices candidate references of respective sizes:

[0089] ^SUp4 _ infx 3

[0090]

[0091]

[0092]

[0093]

[0094]

[0095]

[0096]

[0097]

[0098]

[0099]

[0100]

[0101]

[0102]

[0103] (sup8 - inf 2) x 3 (sup^6-inf|)x 3 ($up| - inf 36) x 3 Graphically, this is the length of the data intervals in the right column of [Fig.4], multiplied by the number of columns in the group. [Fig.3] illustrates a preferred embodiment of the heuristic algorithm for selecting the reference matrix MRef from the initial matrix M, i.e. step 120 of the analysis method of [Fig.l]. This is an iterative algorithm. The iteration rank of the algorithm is identified by an integer i greater than or equal to 1. The principle of this algorithm is, at each iteration, to seek to aggregate a new column to an intermediate group of columns which was retained at the previous iteration, in order to construct a candidate group of columns from which a reference matrix larger than that which can be extracted from the corresponding intermediate group of columns can be extracted. Advantageously, the algorithm is inspired by beam search algorithms. Instead of retaining only one intermediate column group at each iteration, several intermediate column groups are retained at each iteration. The intermediate column groups retained at each iteration are indexed by an integer between 1 and b, where b is the number of intermediate column groups at each iteration. An intermediate group of columns, p1., is therefore indexed by the integers i and j. The b intermediate column groups are initialized to the empty set: pj = 0. At the end of the implementation of the method, at iteration I, one of the b intermediate column groups pl is chosen as the reference group rRef from which an optimal reference matrix MRef in terms of size is extracted. At iteration i, for the b groups of columns p., we define the group of columns of search, F' h, which contains the columns of the matrix M from which we can draw new columns ck to aggregate to the groups of columns pi to search for increase the size of the candidate reference matrix. The search group r*earch is initialized with all columns of the matrix M. Step 210 is the initialization step. At iteration i, in a step 220, we create all the possible doublets {p£ j which associate a group of columns p., among the b groups of intermediate columns at iteration i, and a column ck not belonging to p. ci< is chosen in the group of r'ea!.ch search - Each doublet is a candidate column group.

[0104] In step 230, it is then ensured that there is no duplicate between the different doublets created in step 220. Indeed, certain doublets can be permutations of each other (for example |c8, C4; C16} and {c4, C16; C8))• In the case where several doublets are identical by permutation, only one of these doublets is kept.

[0105] In step 240, a score q is calculated for each doublet Ck|. The metric used to calculate this score must make it possible to evaluate whether the doublet considered is a promising candidate for constituting the reference matrix MRef.

[0106] Several choices can be adopted for this metric. For example, we can take as a score the size of the largest matrix without missing data that can be extracted from the group of columns considered. Thus, taking the example of Figure 4, the score r of the aggregation of column 16 to the group consisting of columns 8 and 4 is equal to r _ îpfx 3' because 'c First time interval pnf8 §up41 is the longest.

[0107] Alternatively, we can also use the sum of the sizes of the 9 largest matrices without missing data that can be extracted from the group of columns considered. Still using the example of Figure 4 and with q = 2, we have: r = sup4-inf। jx 3 + (sup8- inf x 3-

[0108] At step 250, if the scores of all the doublets ck| are lower than the scores r(r]) of the b pi groups, this means that there is no added value in accumulating a new column to construct the reference matrix MRef. This test applies to all the j and all the k.

[0109] In this case the method proceeds to step 290.

[0110] Otherwise, the method continues with step 260, in which the b doublets {r-, qj which have the highest score are selected.

[0111] These b doublets constitute the groups of columns retained for the following iteration •pd+l 1 j

[0112] In step 270, each column ck that participates in a retained group is then subtracted from the search group for the next iteration r1*^-

[0113] In step 280, if the search group is empty, then the method proceeds to step 290.

[0114] Otherwise the algorithm is executed again at the next iteration: i — = i + 1.

[0115] In step 290, the group of columns p! is kept as reference group FRef. retained leading to the largest reference matrix.

[0116] Strictly speaking, a separate research group should be defined for each group p^ (i.e. a search group for each j at iteration i), this search group being the set of columns minus those already present in the group p1. considered. In the previous embodiment, ck belongs to the set of columns minus those of pi. In Variation, we only choose % from among the columns checking that the score of a column multiplied by the total number of columns ncoionnes is less than the highest score of the intermediate groups pi. This

[0117] variant allows to speed up the construction of the reference column group, by reducing the search group. Alternatively, multiple reference matrices can be identified in a series by applying the previous algorithm multiple times. Between two executions of the algorithm, the data from the first reference matrix identified by the algorithm are deleted (they are declared as missing data) in order to select a second reference matrix in the series under consideration.

[0118] Once the reference matrix MRef is determined, its data can be used to test different methods of imputation of missing data.

[0119] The present invention was evaluated in the context of road traffic management in a city. Road traffic is monitored by a sensor network comprising 240 sensors located at different locations in the city's road network.

[0120] Each sensor delivers, every 3 minutes, a measurement of the flow of vehicles over a 3 minute interval.

[0121] The acquisition interval corresponds to one year of road traffic measurements.

[0122] The matrix M corresponding to this multivariate time series therefore comprises 240 columns of 175,199 rows.

[0123] Due to sensor or data collection system failures, 10.35% of the data is missing. Time intervals with missing data for a sensor show a large variability, from a few lines to several thousand consecutive missing lines.

[0124] The heuristic algorithm presented above was used to extract a reference matrix.

[0125] The heuristic algorithm was evaluated with values ​​between 1 and 5 of the number b. In the particular application of traffic management, the number b=3 was chosen for faster convergence while allowing broad exploration of the possibilities offered by the column combinatorics.

[0126] For the calculation of the score, the metric corresponds to the sum of the sizes of the q=3 largest matrices without missing data extractable from a group of columns.

[0127] This led to the selection of a reference matrix with 42 columns over a time range of 22,317 time steps (this represents approximately 46 days of consecutive measurements) without any missing data.

[0128] Missing data patterns were extracted from the real data. First, 42 columns were randomly selected from the matrix M, then for each of these columns, the missing data pattern was selected over an interval of 22,317 consecutive rows, also randomly selected. This process was performed several times to select several missing data patterns for the test. For each of the selected patterns, it was ensured that the missing data rate was at least 10%. The selected patterns were used to construct the test matrix.

[0129] Among various possible imputation techniques (k-neighbor, neural network, compressive sensing, etc.), a technique based on a linear reconstruction using the pseudo-inverse of Moore-Penrose gave the best test results. The overall error on the reconstruction of the data was evaluated at 6% error in standard deviation of the residuals.

[0130] Once the missing data imputation technique was validated on the reference matrix, it was used to correct the matrix representing the initial series.

[0131] The corrected series was used to predict traffic over a variable time horizon (from 3 minutes to 30 minutes). This prediction was carried out using an encoder-decoder neural network based on a long-short-term memory (LSTM) architecture.

[0132] The method according to the invention makes it possible to quantify the effectiveness of a method for imputing missing data on a data distribution close to the real data (since it is a large subset of the real data). It is therefore possible to test different strategies for imputing missing data and choose the best one or at least evaluate whether a strategy for imputing missing data is satisfactory. Once the best method for imputing missing data has been identified, the missing data can be imputed across the entire series, prior to an analysis of the series thus corrected.

[0133] The present invention is advantageous because it allows the data imputation techniques to be tested and validated on a data distribution close to the real data distribution.

[0134] It also allows the performance of data imputation to be evaluated. We therefore have greater confidence in the chosen imputation technique.

[0135] The data for testing imputation techniques are derived from real data. Therefore, the properties of real data such as spatial and / or temporal dependencies are preserved. Therefore, the results on the test data can be better extrapolated to the real data.

[0136] The heuristic algorithm allows to find complete series of large size in incomplete series. Due to its large size, a complete series (i.e. the reference matrix) is more representative of the incomplete series from which it is extracted.

[0137] It should be noted that the implementation of this heuristic algorithm is parallelizable, in particular to accelerate the identification of the best candidate column groups in the beam search approach.

Claims

Claims

1. A method (100) for analyzing a multivariate time series, the method being implemented by computer, characterized in that the method comprises the steps of: - acquiring (110) a multivariate time series, the multivariate time series comprising missing data, the multivariate time series being represented by a matrix (M); - selecting (120), from the matrix (M), a reference matrix (MRef ) comprising no missing data, by implementing a heuristic selection algorithm; - testing (130) a plurality of missing data imputation techniques on the reference matrix and selecting the optimal technique from the plurality of missing data imputation techniques; - correcting (140) the multivariate time series by calculating imputed values ​​for the missing data by implementing the optimal technique; and, - analyzing (150) the multivariate time series thus corrected.

2. Method (100) according to claim 1, in which the heuristic selection algorithm is an iterative process consisting at each iteration of: a) aggregating a column of the matrix (M) representing the multivariate time series, to an intermediate group of columns to obtain a candidate group of columns; b) evaluating a score of the candidate group of columns, the score of a group of columns implementing a metric indicative of the size of at least one candidate reference matrix that can be extracted from the group of columns considered; and, c) iterating steps a) and b) if the score of the candidate group of columns is greater than the score of the intermediate group of columns, the candidate group of columns retained at an iteration being the intermediate group of columns of the following iteration.

3. Method according to claim 2, in which the execution of the heuristic selection algorithm allows, at each iteration, to retain a plurality of candidate column groups.

4. A method according to any one of claims 1 to 3, wherein testing a plurality of missing data imputation techniques consists of: - determining a statistic of the missing data in the matrix (M) representing the multivariate time series; - deleting data from the reference matrix (MRef) in accordance with the determined statistic, to develop a test matrix (MTest); and, for each imputation technique of the plurality of imputation techniques, - applying the imputation technique considered on the test matrix to obtain a corrected reference matrix; and, - comparing the reference matrix and the corrected reference matrix in order to evaluate the imputation technique considered.

5. Method according to any one of the preceding claims, in which the acquisition step consists of acquiring, over an acquisition time interval, the values ​​measured, at each sampling step, by a plurality of sensors of a sensor network, each variable of the multivariate time series being associated with a single sensor of the sensor network.

6. Method according to claim 5, implemented for the analysis of road traffic, in which each value measured by a sensor corresponds to a number of vehicles per unit of time detected at a location of installation of the sensor.

7. The method of claim 6, wherein the step of analyzing the corrected multivariate time series comprises estimating a future state of road traffic.

8. A computer program comprising software instructions which, when executed by a computer, implement a method of analyzing a multivariate time series according to any one of the preceding claims.

Images