DEVICE AND METHOD FOR DETERMINING THE LENGTH OF A RELEVANT STORY

Description

FIELD OF THE INVENTION

[0001] The invention relates to devices for determining a length of a relevant history of instances of a first type of data.

[0002] The invention further relates to methods of determining a length of a relevant history of instances of a first type of data.

[0003] The invention also relates to methods of reducing a dataset, methods of predicting an instance of a second type of data based on a history of instances of a first type of data, and computer program products for determining a length of a relevant history of instances of a first type of data.BACKGROUND OF THE INVENTION

[0004] In machine learning technologies historical data is often used to train the machine, to enable the machine to predict an event based on a recent set of data. In many cases, a very long data history is used to train the machine as well as possible. However, it may require quite a lot of data storage and processing power to use such a long history. Also when new data arrives on which the training must be based, all historical data, including the new data, must be processed once again. Also some other predictive technologies directly use a data history of determined events and measured values. Then, often, the whole stored data history available is used and when a prediction has to be made on the basis of a recent set of data, the whole data history is processed to find similarities between the recent data with the data history - this requires quite a lot of processing power. There is a need to reduce the amount of historical data to be stored.

[0005] Often experts are consulted to get their opinion about the relation between the data to be predicted and the measured (sensor) data. It may be that the data that is being used to predict an event is data obtained from a plurality of sensors. In the traditional approach one determines a single history length for the data of all the different sensors. However, it may be that the length of the history of specific sensors may be reduced more because only relative short histories of these specific sensors comprise most information with respect to the event that is going to be predicted. Consequently, the traditional approach of reducing the length of the history to be used in the predictive technologies does either result in storing too much data or results in inaccurate predictions because too much data is thrown away.

[0006] Above it has been mentioned that the predictive technology is suitable for predicting "an event". It has to be noted that the term "event" must be read broadly. "Event" represents a characteristic of a physical entity, for example, "a component of a machine is going to break down or not", or "the energy consumption of the factory is going to be too high or not". These examples relate to a binary prediction: "something is true or not". However, the above discussed predictive technologies are not limited to binary characteristics and may predict the characteristics also for characteristics that have a value in a higher base numeral system, for example, "the energy consumption of this city is going to be low, medium, or high". The above discussed predictive technologies may also apply to regression use cases in which a scalar value is obtained based on historical data. This interpretation also applies to the remaining of this document.

[0007] The article by R. BEGLEITER ET AL: "On Prediction Using Variable Order Markov Models", JOURNAL OF ARTIFICIAL INTELLIGENCE RESEARCH, vol 22, 1 December 2004, pages 385-421, XP055280865 discusses algorithms for prediction of discrete sequences over a finite alphabet, using variable order Markov models. Six algorithms are compared, including Context Tree Weighting (CTW). The article focusses on completing a sequence of the same type of data, see page 386, lines 30-34: "We focus on a setting in which a learner has a set of training samples of some sequence (e.g., a musical piece) and the goal is to generate a model capable of predicting the rest of the [same] sequence. We also consider a scenario where the learner has a set or training samples from some domain and the goal is to predict, as accurately as possible, new sequences from the same domain".

[0008] The article by CAGNAZZO MET AL: "Mutual information-based context quantization", SIGNAL PROCESSING: IMAGE COMMUNICATION, ELSEVIER SCIENCE PUBLISHERS, AMSTERDAM, NL, vol. 25, NO. 1, 19 September 2009, pages 64-74, XP026824946, ISSN:0923-5965 discusses context quantization and provides an analytical formation thereof. A proposed approach allows a steeper path toward the cost function minimum.SUMMARY OF THE INVENTION

[0009] It is an object of the invention to provide a system and a method capable of determining a length of a relevant history of instances of a first type of data that must be kept or used when an instance of second type of data has to be predicted on the basis of (more recent) instances of the first type of data. The term relevant means in this context: a reliable prediction of the instance of the second type of data is possible on the basis of (more recent) instances of the first type of data and on the basis of historical data that comprises sequences of a limited length with historical information about instances of data of the first type.

[0010] For this purpose, according to an aspect of the invention, a data reduction system for reducing a data set comprising instances of a first type of data is provided as defined in claim 1.

[0011] For this purpose, according to another aspect of the invention, a method for reducing a data set comprising instances of a first type of data is provided as defined in claim 2.

[0012] For this purpose, according to a further aspect of the invention, a computer program product is provided.

[0013] An effect of the features and the measures of the system and the method is that one obtains a number for the length of the sequences of instances of the first type of data that must be kept to maintain enough historic information in the first set of data to reliably predict instances of the second type of data.

[0014] The conditional entropies are a measure that defines the remaining uncertainty of (current and future) instances of the second data type given the knowledge of a set of data that comprises sequences of (historic) instances of the first type of data. If the conditional entropy is relatively low, the set of data that comprises sequences of (historic) instances of the first type of data comprises a lot of information about (current and future) instances of the second type of data and is, therefore, useful for predicting instances of the second type of data when a (more recent) sequence of instances of the first type of data is known.

[0015] In the field of information theory it is known and proven that a conditional entropy for, for example, a variable y given a sequence of instances x 1 ..x n increases or remains equal if one of the instances of the sequence x 1 ..x n is removed. In the context of the above described device and method, the sequence x 1 ..x n can be seen as a history of instances of the first type of data. Thus, if one desires to reduce the amount of historical data to be kept, one has to accept that the conditional entropy may increase and that the uncertainty about variable y given less historical data may increase. With the above described device or by applying the above described method one finds until which length the sequences with instances of the first type of data may be reduced such that the conditional entropy does not increase too much. Thus, the requirement of data reduction is balanced against the increase in uncertainty. How much the conditional entropy may increase has been defined by the maximum entropy increase factor and, thus, the maximum entropy increase factor determines the balance between data reduction and the increase in uncertainty. If one does not want to accept a large increase in uncertainty, a low maximum entropy increase threshold factor must be used. If one wants more data reduction, a higher maximum entropy increase threshold factor must be used. Given the defined maximum entropy increase threshold, the device or the method finds to which number the lengths of the instances the sequences with (historic) instances of the first data type may be reduced.

[0016] Instances of data of the first type are being based on measured sensor data. This means that the data of the first type is measurements of a specific sensor, but is also a specific combination of measured values by two specific sensors that are combined, "fused", into a single value. For example, several temperature measurements sensed by sensors that sense a temperature of a specific physical entity may be averaged into a single temperature value representing the temperature of the specific physical entity.

[0017] It is to be noted that the determined length of the relevant history relates to one specific combination of a first type of data and of a second type of data. For example, the first type of data is data obtained from one specific type of sensor that has a specific function in an apparatus. The first type of data does not relate to other sensors of the apparatus or the same type of sensor but with a different function in the apparatus. It may also be that the first type of data is from sensors of the specific type having the specific function in different apparatuses of the same type. For example, the first type of data is the temperature of a specific motor in the apparatus. In the example, the apparatus may have more temperature sensors that measure the temperature of other components, but the measured values of these other sensors are of a different type of data. This also applies to situation when values of different sensors are combined into one specific value: then instances of the first type of data always relate to the same combination of the same type of sensors that have the same functions. The same applies to the second type of data: it is a specific characteristic of a specific type of a physical entity. For example, whether a component of an apparatus fails or not. In the example, the failure status of other components is another type of data. One instance of the second set corresponds to a sequence in the first set - this means that at the moment in time that an instance of the second set was determined at a specific machine, the specific sensor of the specific machine measured during an interval of time preceding the moment in time the instances of the corresponding sequence. It might also be that the sequence comprises an instance of the sensor obtained at the moment in time that the specific (corresponding) instance of the second set was determined.

[0018] Because the determined length of the relevant history relates to one specific combination of the first type of data and of the second type of data, one may use the device or method to determine for different combinations to obtain different lengths of a relevant history. This is in particular advantageous if data of, for example, different sensors is available for predicting a characteristic of a physical entity. By using the device or the method for several combinations one may find for every sensor the optimal history length and thereby one may create an even more reduced set of sensor data for use in predictive or machine learning technologies without losing too much prediction accuracy.

[0019] Optionally, the determining of the length of the relevant history comprises i) obtaining a temporary reduced first set of data from the first set data of data, in the temporary reduced first set of data for each sequence an oldest instance of data has been removed compared to the sequences of the first set of data, ii) obtaining the second conditional entropy for the second set of data given the temporary reduced set, iii) comparing the second conditional entropy with the first conditional entropy for determining whether the second conditional entropy is within the maximum entropy increase threshold from the first conditional entropy, and a) if the second conditional entropy is within the maximum entropy increase threshold from the first conditional entropy, then remove from the sequences of the temporary reduced first set of data the oldest instances and the obtaining of the second conditional entropy and the comparing of the second conditional entropy with the first conditional entropy are performed once again, or b) if the second conditional entropy is not within the maximum entropy increase threshold from the first conditional entropy, then the determined length of the relevant history is the length of the sequences of the temporary reduced first set plus one.

[0020] This optional embodiment provides an effective embodiment for determining the length of the relevant history. According to the embodiment, a search is started for the length of the relevant history by starting the search with a relatively large reduced first set of data and subsequently reducing the size of the reduced first set in small steps and determining whether this set is still within the boundaries defined by the first conditional entropy and the maximum entropy increase threshold.

[0021] It is to be noted that embodiments of the determining of the length of the relevant history are not limited to the above discussed search algorithm. The determining of the length of the relevant history can also be based on a binary search algorithm: first a second conditional entropy is determined for a temporary reduced first set having sequences that have only the most recent half of the sequences of the first set and determining in which direction the length of the relevant history can be found. If it is known whether the length of the relevant history is in the most recent half or the older half of the history, a temporary reduced first set is obtained for a point in the middle of the most recent half or the older half of the history, respectively, and it is determined whether the length of the relevant history can be found before or after this points, etc. The determining of the length of the relevant history can also be based on other search algorithms that are appropriate for sorted lists: the values of the second conditional entropy increase or remain equal when the lengths of the sequences in the temporary reduced first set are reduced.

[0022] In an embodiment, the obtaining of the first conditional entropy comprises estimating the first conditional entropy by i) applying a Context Tree Weighting method to the second set and first set to obtain a first context tree, and ii) using a weighted probability of the root of the first context tree to calculate the estimation of first conditional entropy. Every unique sequence of the first set is represented by a path in the first context tree and counts stored in the nodes of the first context tree are based on the corresponding elements of the second set. The Context Tree Weighting method is a well-known method in the field of text compression. If, in the context of the invention, it is assumed that the data of the first set is generated by a tree source, the Context Tree Weighting method is an efficient and effective method to generate an estimated tree for this tree source - the estimated tree is the first context tree in the above discussed context. The Context Tree Weighting method provides efficient and effective means to calculate weighted probabilities for the nodes of the context tree, and, given the above assumption(s), an estimation of the first conditional entropy can be obtained from the weighted probability of root of the first context tree. Thus, the Context Tree Weighting method provides effective and efficient means to estimate the first conditional entropy.

[0023] In an embodiment, the obtaining of the second conditional entropy comprises estimating the second conditional entropy by i) applying the Context Tree Weighting method to the second set and the reduced first set or the temporary reduced first set to obtain a second context tree, and ii) using a weighted probability of the root of the second context tree to calculate the estimated second conditional entropy. Every unique sequence of the reduced first set or the temporary reduced first set is represented by a path in the second context tree and counts stored in the nodes of the second context tree are based on the corresponding elements of the second data set. In line with the discussion above, the Context Tree Weighting method is an efficient and effective method to obtain an estimation of the second conditional entropy.

[0024] In an embodiment, instead of completely applying the Context Tree Weighting method to the second set and the reduced first set or the temporary reduced first set, the second context tree is obtained by removing leafs from the first context tree that have, seen from the root, a depth that is larger than the length of the sequences of the reduced first set or the temporary reduced first set. As discussed previously, one may estimate the second conditional entropy by building a new second context tree. The inventors observed that the second context tree is a sub tree of the first context tree that was built for estimating the first conditional entropy. The inventors have found that, if from each sequences of the first set the oldest instance is removed, the depth of the second context tree is reduced by one, namely, the second context tree does not contain the leafs of the tree end edges ending in the leaf nodes. By deduction one may conclude that when the sequences of the reduced first set are of a given length, that then all leafs and nodes (plus corresponding edges) of the first context tree with a depth larger than this given length have to be removed for obtaining the second context tree. This embodiment enables an even more efficient estimation of the second conditional entropy.

[0025] In an embodiment, the instances of the first type of data are discrete values. Using such continuous data results in much more complex ways of calculating or estimating the first and / or second conditional entropy. The Context Tree Weighting method is most efficient with respect to memory use and computational power if the number of discrete values of the instances of the first type are limited. In an embodiment, the method of determining the length of the relevant history also comprises quantizing instances of continuous data to obtain the discrete values of the instances of the first type of data, wherein the quantizing is performed such that a loss of mutual information between the second type of data and the first type of data is within a maximum information loss threshold. Quantizing results in a loss of mutual information, but the increase in efficiency of the method outweighs the loss of mutual information if this loss is kept within an acceptable maximum information loss threshold. An embodiment of such a quantizing is creating a number of equally sized bins in between the minimum and maximum value of the continuous data of the first type of data and assigning a number of the bin to the continuous data that falls within the boundaries of that bin. By increasing the number of bins, the loss of mutual information may be reduced. More advanced method to reduce the loss of mutual information may be used as well.

[0026] According to another aspect, a method of reducing a dataset is provided. The method of reducing the dataset comprises the method of determining the length of the relevant history according to anyone of the above discussed embodiments of this method, and comprises reducing the first data set by deleting in every sequence of the first set of data oldest instances of the first type of data until the sequence has the determined length of the relevant history. Thereby one obtains a significantly reduced first data set that is still very useful in predicting instances of the second type of data if one has a (more recent) sequence of instances of the first type of data.

[0027] According to a further aspect, a method of predicting an instance of a second type of data based on a history of instances of the first type of data is provided. The method comprises the above discussed method of reducing a data set and comprises: i) receiving a recent history of instances of the first type of data, and ii) using the reduced first set of data and the second set of data to predict an instance of the second type of data of train a prediction model. With this method instances of the second type of data are predicted relatively accurate while the amount of historical data stored is significantly reduced. Thereby a more efficient way of predicting instances of the second type of data is obtained. The exact way of predicting instances of the second type is not relevant in the context of this document - in this embodiment it is only relevant that the method of predicting uses historical data in the form of instances of the first type of data and of instances of the second type of data.

[0028] According to an aspect of the invention, a computer program product for determining a length of a relevant history of instances of a first type of data is provided. The program is operative to cause a processor to perform any one of the above discussed methods.

[0029] Further preferred embodiments of the device and method according to the invention are given in the appended claims.BRIEF DESCRIPTION OF THE DRAWINGS

[0030] These and other aspects of the invention will be apparent from and elucidated further with reference to the embodiments described by way of example in the following description and with reference to the accompanying drawings, in which Figure 1 schematically shows a device for determining a length of a relevant history of instances of a first type of data. Figure 2 schematically shows a method of determining a length of a relevant history of instances of a first type of data, Figure 3 schematically shows two machines of the same type that may generate instances of a first type of data and instances of a second type of data, Figure 4 schematically shows an embodiment of a computer program product.

[0031] The figures are purely diagrammatic and not drawn to scale. In the Figures, elements which correspond to elements already described may have the same reference numerals.DETAILED DESCRIPTION OF EMBODIMENTS

[0032] Figure 3 schematically shows two machines 300, 350 of the same type that may generate instances of a first type of data and instances of a second type of data. Fig. 3 is an example of an area where the device and method for determining a length of a relevant history of instances of a first type of data can be applied. This figure is being used in the discussion of Fig. 1 . It is to be noted that both machines 300, 350 are of the same type, which means that they are of the same design, perform the same function, etc. Both machines 300, 350 comprise sensors 302, 352 that generate instances of a first type of data. This means that sensors 302, 352 are similar to each other and have the same function in both machines 300, 350. For example, the sensor 302, 352 measure a temperature of a motor of the machine. Both machines 300, 350 both have a physical entity 304, 354 of which a characteristic is determined and / or has to be predicted. The characteristic is an instance of a second type of data. For example, the physical entity 304, 354 are a motor of the machine and the characteristic is "the motor 304 or 354 is operating or malfunctions".

[0033] Figure 1 schematically shows a device 100 for determining a length of a relevant history of instances of a first type of data. The instances of the first type of data comprise information that is useful and can be used to predict instance of the second type of data. The device comprising a first data set unit 102, a second data set unit 104, a first conditional entropy unit 106, a threshold unit 108, a length determining unit 114, and an output unit 116.

[0034] The second data set unit 104 is arranged for obtaining a second set of data that comprises instances of the second type of data. Instances of the second type of data comprise a characteristic of a physical entity. The second set is, for example, indicated by Y. The set Y comprises several instances yi that are a characteristic of the physical entity at different moments in time. For example, the instance yi are the characteristics "the motor 304 or 354 operates well or malfunctions". For example, a first element of the set is: y 1(machine 1, t=15) =1 if the motor of machine 1 operates well at the moment in time 15, a second element of the set is: y 2(machine 2, t=25) =0 if the motor of machine 2 malfunctions at the moment in time 25.

[0035] The first data set unit 102 is arranged for obtaining a first set of data that comprises sequences of instances of the first type of data. Each sequence provides a history of instances of the first type of data for a corresponding element of the second set. Each sequence comprises instances preceding, and optionally of, the moment in time at which the corresponding element of the second set is determined. The instances of the first type of data are ordered according to time in the sequences. Instances of the first type of data comprise measured sensor data. For example, the first set is indicated with X and comprises several sequences x 1 , x 2 , ... Sequence x 1 relates to instance y 1 of the second set Y and, thus, comprises instances of data of sensor 302 of an interval of time that precedes (and optionally comprises) the moment in time at which the characteristic y 1 =l of the motor 304 is obtained. For example, Sequence x 1 = (xi(motor sensor 302, t=15) X i (motor sensor 302, t=14) X i (motor sensor 302, t=13) X i (motor sensor 302, t=12) ...), wherein x i refers to an instance of the first type of data. Sequence x 2 relates to instance y 2 of the second set Y and comprises instance of sensor 352 of an interval of time that precedes (and optionally comprises) the moment in time that the characteristic of motor 354 is obtained. For example, sequence x 2 = (x i(motor sensor 352, t=25) x i(motor sensor 352, t=24) x i(motor sensor 352, t=23) x i(motor sensor 352, t=22) ... ). The sequences x 1 and x 2 comprise the same number of instances of the first type of data - thus, in other words, the intervals of time for which the sequences x 1 and x 2 comprise instances of the first type have the same length.

[0036] The first conditional entropy unit 106 is arranged for obtaining a first conditional entropy for the second set of data given the first set of data. In the example of the data of machines 300 and 350, the first conditional entropy is indicated by H 1 (Y|X). Conditional entropy is well defined in the field of information theory. Later in this document embodiments of calculating or estimating the first conditional entropy are provided. The first conditional entropy unit 106 may be coupled to the first data set unit 102 and the second data set unit 104 may be arranged for receiving the first set of data and the second set of data.

[0037] The threshold unit 108 is arranged for obtaining a maximum entropy increase threshold indicating a factor by which the first conditional entropy may increase when the length of the sequences in the first set of data is reduced. For example, in practical embodiments, the maximum entropy increase threshold is indicated by a and the value of a is a number in between zero and one. 1 means that the first conditional entropy may increase by 100%, 0.5 means that the first conditional entropy may increase by 50%, etc. Assume that the first set is reduced to X r , than the maximum entropy increase threshold defines that H(Y|X r ) <(1+α)H(Y|X) (in which the lengths of sequences of reduced first set X r are shorter than the sequences of X).

[0038] The length determining unit 114 is arranged for determining the length of the relevant history by determining a smallest length for the sequences of a reduced first set of data for which a second conditional entropy for the second set of data given the reduced first set of data is within the maximum entropy increase threshold from the first conditional entropy. The reduced first set of data comprises sequences of the smallest length and the sequences of the reduced set of data comprises the most recent instances of the first type of data of their corresponding sequences in the first set of data. The length determining unit 114 may be coupled to the first data set unit 102, to the second data set unit 104, to the threshold unit 108, and to the first conditional entropy unit 106 for receiving the first set of data, the second set of data, the maximum entropy increase threshold and the first conditional entropy, respectively.

Claims

1. A data reduction system (150) for reducing a data set comprising instances of a first type of data, the instances of the first type of data comprising information for predicting at least one instance of a second type of data, the system comprising a physical entity (304, 354) as a motor and sensors (302, 352) as temperature sensors for providing measured sensor data, the temperature sensors providing temperature data and indicating that the temperature data is the first type of data, a second data set unit (104) for obtaining a second set of data comprising instances of the second type of data, the second type of data being the indication of a malfunction or not, instances of the second type of data comprising data regarding a characteristic of the physical entity (304, 354), and the characteristic of the physical entity regarding an event as the malfunction of motor, a first data set unit (102) for obtaining a first set of data comprising sequences of instances of the first type of data, each sequence providing a history of instances of the first type of data for a corresponding element of the second set, each sequence comprising instances preceding the moment in time at which the corresponding element of the second set is determined, the instances of the first type of data being ordered according to time in the sequences, instances of the first type of data comprising data being based on the measured sensor data, data of different sensors being available for predicting the characteristic of the physical entity and each history relating to one specific combination of the first type of data based on a specific one of the different sensors and of the second type of data, a first conditional entropy unit (106) for obtaining a first conditional entropy for the second set of data given the first set of data for each specific one of said different combinations, a threshold unit (108) for obtaining a maximum entropy increase threshold indicating a factor by which the first conditional entropy may increase when the length of the sequences in the first set of data is reduced, a length determining unit (114) for determining the length of the relevant history for reducing the amount of history while maintaining enough historic information in the first set of data to reliably predict instances of the second type of data, by determining for each specific one of said different combinations a specific smallest length for the sequences of a reduced first set of data for which a second conditional entropy for the second set of data given the reduced first set of data is within the maximum entropy increase threshold from the first conditional entropy, the reduced first set of data comprising sequences of the smallest length and the sequences of the reduced first set of data comprising the most recent instances of the first type of data of their corresponding sequences in the first set of data, and an output unit (116) for providing for several combinations the smallest lengths for the sequences as the length of the relevant history for reducing the amount of history stored for the data of the first type, a data reduction unit (152) for obtaining a reduced first set of data being based on the first set of data and comprising sequences of instances of the first type of data by removing remove the oldest instances, wherein a length of the sequences of the reduced first set of data is based on said smallest lengths.

2. A computer-implemented method (200) for reducing a data set comprising instances of a first type of data in a system comprising a physical entity (304, 354) as a motor and sensors (302, 352) as temperature sensors for providing measured sensor data, the temperature sensors providing temperature data and indicating that the temperature data is the first type of data, the instances of the first type of data comprising information for predicting instances of a second type of data, the method comprising obtaining (202) a second set of data comprising instances of the second type of data, the second type of data being the indication of a malfunction or not, instances of the second type of data comprising data regarding a characteristic of the physical entity (304, 354), and the characteristic of the physical entity regarding an event as the malfunction of motor, obtaining (204) a first set of data comprising sequences of instances of the first type of data, each sequence providing a history of instances of the first type of data for a corresponding element of the second set, each sequence comprising instances preceding the moment in time at which the corresponding element of the second set is determined, the instances of the first type of data being ordered according to time in the sequences, instances of the first type of data comprising data being based on the measured sensor (302, 352) data, data of different sensors being available for predicting the characteristic of the physical entity and each history relating to one specific combination of the first type of data based on a specific one of the different sensors and of the second type of data, obtaining (206) a first conditional entropy for the second set of data given the first set of data for each specific one of said different combinations, obtaining (208) a maximum entropy increase threshold indicating a factor by which the first conditional entropy may increase when the length of the sequences in the first set of data is reduced, determining (210) the length of the relevant history by determining for each specific one of said different combinations a specific smallest length for the sequences of a reduced first set of data for which a second conditional entropy for the second set of data given the reduced first set of data is within the maximum entropy increase threshold from the first conditional entropy, the reduced first set of data comprising sequences of the smallest length and the sequences of the reduced first set of data comprising the most recent instances of the first type of data of their corresponding sequences in the first set of data, providing (212) for several combinations the smallest lengths for the sequences as the length of the relevant history for reducing the amount of history stored for the first type of data, and obtaining a reduced first set of data being based on the first set of data and comprising sequences of instances of the first type of data by removing remove the oldest instances, wherein a length of the sequences of the reduced first set of data is based on said smallest lengths.

3. A method (200) according to claim 2, wherein the determining of the length of the relevant history comprises obtaining a temporary reduced first set of data from the first set data of data, in the temporary reduced first set for each sequence an oldest instance has been removed compared to the sequences of the first set of data, obtaining the second conditional entropy for the second set of data given the temporary reduced first set, comparing the second conditional entropy with the first conditional entropy for determining whether the second conditional entropy is within the maximum entropy increase threshold from the first conditional entropy, and a) if the second conditional entropy is within the maximum entropy increase threshold from the first conditional entropy, then remove from the sequences of the temporary reduced first set of data the oldest instances and the obtaining of the second conditional entropy and the comparing of the second conditional entropy with the first conditional entropy are performed once again, or b) if the second conditional entropy is not within the maximum entropy increase threshold from the first conditional entropy, then the determined length of the relevant history is the length of the sequences of the temporary reduced first set of data plus one.

4. A method (200) according to any one of the claims 2 and 3, wherein the obtaining of the first conditional entropy comprises estimating the first conditional entropy by applying a Context Tree Weighting method to the second set and first set to obtain a first context tree, and using a weighted probability of the root of the first context tree to calculate an estimation of first conditional entropy, in the Context Tree Weighting method every unique sequence of the first set is represented by a path in the first context tree and counts stored in the nodes of the first context tree are based on the corresponding elements of the second set.

5. A method (200) according to any one of the claims 2 to 4, wherein the obtaining of the second conditional entropy comprises estimating the second conditional entropy by applying the Context Tree Weighting method to the second set and the reduced first set or the temporary reduced first set to obtain a second context tree, and using a weighted probability of the root of the second context tree to calculate the estimated second conditional entropy, in the Context Tree Weighting method every unique sequence of the reduced first set or the temporary reduced first set is represented by a path in the second context tree and counts stored in the nodes of the second context tree are based on the corresponding elements of the second data set.

6. A method (200) according to the combination of claims 4 and 5, wherein instead of completely applying the Context Tree Weighting method to the second set and the reduced first set or the temporary reduced first set, the second context tree is obtained by removing leafs and nodes from the first context tree that have, seen from the root, a depth that is larger than the length of the sequences of the reduced first set or the temporary reduced first set.

7. A method (200) according to one of the claims 2 to 6 wherein the instances of the first type of data are discrete values.

8. A method (200) according to claim 7, further comprising quantizing instances of continuous data to obtain the discrete values of the instances of the first type of data, wherein the quantizing is performed such that a loss of mutual information between the second type of data and the first type of data is within a maximum information loss threshold.

9. A method (200) according to any one of the claim 2 to 8, further comprising reducing the first data set by deleting in every sequence of the first set of data oldest instances of the first type of data until the sequence has the determined length of the relevant history.

10. A method (200) according to claim 9, further comprising receiving a recent history of instances of the first type of data, and using the reduced first set of data and the second set of data to predict an instance of the second type of data or to train a prediction model.

11. A computer program product (470) comprising computer executable instructions to cause the device of claim 1 to execute the steps of the method as claimed in any one of the claims 2 to 10.