Inference device and inference method

Abstract

An inference device (10) is configured to comprise: a patch cutout unit (11) that cuts out, as a patch, partial time series data for each variable from multivariate time series data in which a plurality of variables change in a time series; a contribution calculation unit (12) that calculates, for each patch which has been cut out by the patch cutout unit (11), contribution to an inference value at a prescribed time point or in a prescribed period; a contribution aggregation unit (13) that calculates an ultimate inference value at the prescribed time point or in the prescribed period, on the basis of the sum of contributions of respective patches which have been calculated by the contribution calculation unit (12); and an explanation display unit (14) that displays, for each patch, the contribution which is made to the ultimate inference value by the patch which has been cut out by the patch cutout unit (11).

Description

Inference device and inference method

[0001] This disclosure relates to an inference device and an inference method.

[0002] In recent years, machine learning models such as neural networks have been applied to a wide range of fields, including image processing, speech processing, and natural language processing. For example, sensor data acquired by sensors installed on target equipment such as factories, plants, buildings, and data centers is input into a neural network, and abnormalities in the target equipment are detected based on the output results. When using machine learning models to solve problems, identifying which inputs were important, or what the model looked at to make its judgment, is crucial for industrial applications.

[0003] For example, Patent Document 1 discloses a data processing device having a generation unit that generates weights for each feature of multivariate time series data to be processed, using a model that has been trained to generate weights representing the degree of contribution of each feature of multivariate time series data to the model's output using an attention mechanism, and an output unit that outputs each weight generated by the generation unit in association with each feature. In this case, the output unit outputs each weight generated by the generation unit multiplied by a feature map.

[0004] Patent No. 7267044

[0005] The data processing device disclosed in Patent Document 1 can infer which variables (features) in the multivariate time series data contributed to the model's output by multiplying the weights for each feature (each variable) generated using a model learned with an attention mechanism by a feature map and outputting the result. However, with this data processing device, it is difficult to infer which sub-time series data in the multivariate time series data contributed to the inferred value, which is the model's output.

[0006] This disclosure was made to solve the above-mentioned problems and aims to provide an inference device that can explain which partial time series data from multivariate time series data contributed to the inferred value.

[0007] The inference device relating to this disclosure includes: a patch extraction unit that extracts partial time series data as patches for each variable from multivariate time series data in which multiple variables change over time; a contribution calculation unit that calculates the contribution to the inference value at a predetermined time or period for each patch extracted by the patch extraction unit; a contribution aggregation unit that calculates the final inference value at a predetermined time or period based on the sum of the contributions for each patch calculated by the contribution calculation unit; and an explanatory display unit that displays the contribution that each patch extracted by the patch extraction unit has to the final inference value for each patch.

[0008] According to this disclosure, with the configuration described above, it is possible to provide an inference device that can explain which partial time series data from the multivariate time series data contributed to the inferred value.

[0009] This is a block diagram showing an example of the configuration of the inference device according to Embodiment 1. This is a flowchart showing an example of the operation of the inference device according to Embodiment 1. This is a diagram illustrating an example of processing by the patch extraction unit in Embodiment 1. This is a diagram illustrating an example of processing by the contribution calculation unit in Embodiment 1. This is a diagram illustrating an example in which the contribution calculation unit in Embodiment 1 calculates the contribution by considering the predicted inference value as a patch. This is a diagram illustrating an example of processing by the explanatory display unit in Embodiment 1. Figures 7A and 7B show an example of the hardware configuration of the inference device according to Embodiment 1. This is a block diagram showing an example of the configuration of the inference device according to Embodiment 2.

[0010] The embodiments will now be described in detail with reference to the drawings. Embodiment 1. Figure 1 is a block diagram showing an example configuration of the inference device 10 according to Embodiment 1. The inference device 10 is configured to include, for example, a patch extraction unit 11, a contribution calculation unit 12, a contribution aggregation unit 13, and an explanatory display unit 14, as shown in Figure 1. The inference device 10 is also connected to a display device 20, such as a display.

[0011] The patch extraction unit 11 acquires multivariate time series data in which multiple variables change over time, for example, from a device other than the inference device 10. The patch extraction unit 11 extracts partial time series data as patches for each variable from the acquired multivariate time series data. The patch extraction unit 11 outputs the extracted patches to the contribution calculation unit 12.

[0012] The contribution calculation unit 12 obtains patches extracted for each variable from the patch extraction unit 11. The contribution calculation unit 12 calculates the contribution to the inferred value at a predetermined time or period for each patch obtained from the patch extraction unit 11. The predetermined time or period is, for example, a time or period in the future from the present, or a time or period in the past from the present. The contribution calculation unit 12 outputs the calculated contribution for each patch to the contribution aggregation unit 13 and the explanation display unit 14.

[0013] Here, the contribution calculation unit 12 may include, for example, a patch coding unit 121 that calculates a feature representation for each patch extracted by the patch extraction unit 11, a patch mapping unit 122 that maps the feature representations for each patch calculated by the patch coding unit 121 to each other, and a patch decoding unit 123 that calculates the contribution of each patch to the inference value at a predetermined time or period from the feature representations mapped by the patch mapping unit 122. Details of the patch coding unit 121, the patch mapping unit 122, and the patch decoding unit 123 will be described later.

[0014] The contribution aggregation unit 13 obtains the contribution of each patch to the inferred value at a predetermined time or period from the contribution calculation unit 12. Based on the sum of the contributions of each patch obtained from the contribution calculation unit 12, the contribution aggregation unit 13 calculates the final inferred value at a predetermined time or period. The contribution aggregation unit 13 outputs the calculated final inferred value to the explanation display unit 14 and also outputs the calculated final inferred value to a device other than the inference device 10, for example.

[0015] The explanatory display unit 14 obtains the contribution of each patch to the inferred value at a predetermined time or period from the contribution calculation unit 12, and also obtains the final inferred value from the contribution aggregation unit 13. The explanatory display unit 14 displays the contribution that each patch has to the final inferred value on the display device 20 for each patch. At this time, the explanatory display unit 14 may also display the final inferred value on the display device 20 in addition to the contribution that each patch has to the final inferred value.

[0016] Next, an example of the operation of the inference device 10 shown in Figure 1 will be described. Figure 2 is a flowchart illustrating an example of the operation of the inference device 10.

[0017] First, the patch extraction unit 11 extracts partial time series data as patches for each variable from multivariate time series data in which multiple variables change over time (step ST1). Next, the contribution calculation unit 12 calculates the contribution to the inferred value at a predetermined time or period for each patch extracted by the patch extraction unit 11 (step ST2).

[0018] Next, the contribution aggregation unit 13 calculates the final inferred value at a predetermined time or period based on the sum of the contributions for each patch calculated by the contribution calculation unit 12 (step ST3). Next, the explanation display unit 14 displays the contribution that each patch extracted by the patch extraction unit 11 has to the final inferred value (step ST4).

[0019] The following explains the process for each step in detail.

[0020] <ST1: Processing by the patch cutting section 11> An example of processing by the patch cutting section 11 will be explained with reference to Figure 3.

[0021] The patch extraction unit 11 extracts sub-series data as patches from the multivariate time series data, for each variable. Figure 3 shows an example where the multivariate time series data consists of five variables arranged along the vertical axis, with the horizontal axis representing time. In Figure 3, each rectangular frame represents one patch.

[0022] At this time, the length of the patch cut out by the patch cutting unit 11 may be a fixed length or a variable length. Also, the length of the patch cut out by the patch cutting unit 11 may be the same for all variables, or may be changed for each variable. FIG. 3 shows a case where the length of the patch cut out by the patch cutting unit 11 is fixed to the same length for all variables (fixed length).

[0023] Also, among the patches cut out by the patch cutting unit 11, adjacent patches in the time direction may or may not have an overlapping portion in the time direction. Also, adjacent patches in the time direction may be continuous without a gap in the time direction, or may have a gap.

[0024] Furthermore, the patch cutting unit 11 may target not only the time series data actually observed so far but also future time series data that can be acquired in advance for cutting out. In FIG. 3, the former is displayed as "Historical variables" and the latter is displayed as "Future variables". For example, based on weather forecasts, calendar information, or a previously created production plan or promotion plan, it is possible to acquire future time series data. In such a case, the patch cutting unit 11 may target future time series data that can be acquired in advance for cutting out.

[0025] Also, when there are variations for each variable in the scale of the multi-variable time series data, the patch cutting unit 11 may normalize the multi-variable time series data in advance and then cut out the patches. For example, the patch cutting unit 11 may perform instance normalization in advance for each variable and then cut out the patches. When the final inference value is calculated by the contribution aggregation unit 13 after the patch cutting unit 11 normalizes the multi-variable time series data in advance, the contribution aggregation unit 13 may inverse-normalize the calculated final inference value using information such as the mean and variance of each variable obtained during the normalization by the patch cutting unit 11.

[0026] <ST2: Processing by the contribution calculation unit 12> Next, an example of the processing by the contribution calculation unit 12 will be described.

[0027] The contribution calculation unit 12 calculates the contribution to be given to the inference value at a predetermined time point or period for each patch cut out by the patch cutting unit 11. For example, the calculation of the contribution by the contribution calculation unit 12 is performed in the following three patterns. The following will be described in order. (1) Calculate the contribution independently for each patch (2) Calculate the contribution by associating between patches using an attention mechanism (3) Calculate the contribution by regarding the inference value of the prediction target as a patch

[0028] (1) Calculate the contribution independently for each patch For example, when the dimension of the vector representing the length of one patch (one partial sequence data) is P, and the dimension of the vector representing the contribution given by this patch to the inference value at a predetermined time point or period is C, the contribution calculation unit 12 uses a machine learning model pre-trained to take a P-dimensional vector representing the patch as input and output a C-dimensional vector representing the contribution, and calculates the contribution for each patch.

[0029] Here, for the sake of easy understanding, the inference value is assumed to be, for example, the inference value at a time point or period in the future from the currently set time. For example, when there is data indicating the sales of a certain product on each day of the week from Sunday to Saturday as a patch, the inference value is data indicating the sales of the product on the day after the Saturday (the next Sunday).

[0030] In this case, as the machine learning model, for example, any model such as a generalized linear model, a decision tree model, a decision tree ensemble model, a neural network, a support vector machine (SVM), and a Gaussian process regression can be used.

[0031] The contribution calculation unit 12 inputs a P-dimensional vector representing the patch to the machine learning model as described above, and obtains a C-dimensional vector representing the contribution to the inference value output from the machine learning model. At this time, the contribution calculation unit 12 may add static variable information that does not change over time to the input of the machine learning model. For example, the contribution calculation unit 12 may add, as static variable information that does not change over time, information indicating the product ID of the product to be predicted, the product category of the product to be predicted, or the store location of the store to be predicted, to the input of the machine learning model.

[0032] Furthermore, the contribution calculation unit 12 may input relative or absolute time information to the machine learning model in place of, or in addition to, static variable information that does not change over time. For example, the contribution calculation unit 12 may input information indicating the month, day of the week, time, or time relative to the current point in time as relative or absolute time information to the machine learning model. Also, the contribution calculation unit 12 only needs to calculate the contribution to the inferred value at a predetermined point in time or period for each patch, and it is not necessarily required to know what the specific value of the inferred value at the predetermined point in time or period is.

[0033] When using the machine learning model described above, the model may be prepared independently for each patch, i.e., for each patch, or it may be prepared in a way that it can be shared across all patches.

[0034] (2) Calculating contributions by associating patches using an attention mechanism Next, an example of calculating contributions by associating patches using an attention mechanism will be explained with reference to Figure 4.

[0035] First, the patch coding unit 121 calculates a feature representation for each patch extracted by the patch extraction unit 11. For example, the patch coding unit 121 uses a neural network to calculate a feature representation for each patch extracted by the patch extraction unit 11.

[0036] In this case, the neural network is, for example, a neural network that has been pre-trained to output an F-dimensional vector representing a feature representation in response to inputs of a P-dimensional vector representing a patch and an S-dimensional vector representing variable and time information.

[0037] Here, assuming that the number of patches extracted by the patch extraction unit 11 is B, the patch coding unit 121 calculates an F-dimensional vector representing the feature representation corresponding to each of the B patches. In other words, the patch coding unit 121 calculates B F-dimensional vectors representing feature representations.

[0038] Next, let's consider a case where, for example, N F-dimensional vectors representing the feature representations of N patches are inferred from B F-dimensional vectors representing feature representations. In the example in Figure 4, we see an example where 2 (N=2) F-dimensional vectors are inferred from 24 (B=24) F-dimensional vectors. In this case, the patch mapping unit 122 takes 24 F-dimensional vectors corresponding to 24 input patches and 2 F-dimensional vectors corresponding to 2 output patches as input, outputs 2 F-dimensional vectors, and uses an attention mechanism to map the 24 patches to the 2 patches.

[0039] A known attention mechanism can be used. For example, a typical attention mechanism used is multi-head attention, which employs a scaled dot-product as its calculation method. In multi-head attention, a single attention mechanism is executed multiple times. In Figure 4, the number of executions is N. ｈｅａｄ It is written as follows.

[0040] Furthermore, in Figure 4, the fourth F-dimensional vector from the right among the 24 F-dimensional vectors is Z ｓｒｃ This is written as follows, and the leftmost of the two F-dimensional vectors is Z ｔｇｔ This is how it is written. Furthermore, the F-dimensional vector Z ｔｇｔ It is assumed that this is already known.

[0041] The patch mapping section 122 is an F-dimensional vector, Z ｓｒｃ A linear transformation is performed on the value vector. The generated value vector is denoted as V in Figure 4. Specifically, the patch mapping unit 122 is an F-dimensional vector Z ｓｒｃBy multiplying with a predetermined weight for , a value vector V is generated. The patch association unit 122 performs this generation of the value vector V N ｈｅａｄ times. In each generation, the weight multiplied with Z ｓｒｃ , which is an F-dimensional vector, is different.

[0042] Similarly, the patch association unit 122 performs a linear transformation on Z ｓｒｃ , which is an F-dimensional vector, to generate a key vector. The generated key vector is denoted as K in FIG. 4. Specifically, the patch association unit 122 generates the key vector K by multiplying with a predetermined weight for Z ｓｒｃ , which is an F-dimensional vector. The patch association unit 122 performs this generation of the key vector K N ｈｅａｄ times. In each generation, the weight multiplied with Z ｓｒｃ , which is an F-dimensional vector, is different.

[0043] Similarly, the patch association unit 122 performs a linear transformation on Z ｔｇｔ , which is an F-dimensional vector, to generate a query vector. The generated query vector is denoted as Q in FIG. 4. Specifically, the patch association unit 122 generates the query vector Q by multiplying with a predetermined weight for Z ｔｇｔ , which is an F-dimensional vector. The patch association unit 122 performs this generation of the query vector Q N ｈｅａｄ times. In each generation, the weight multiplied with Z ｔｇｔ , which is an F-dimensional vector, is different.

[0044] The patch association unit 122 inputs the generated value vector V, key vector K, and query vector Q into an attention mechanism based on the scaled dot product. In the attention mechanism, first, the inner product of the key vector K and the query vector Q is calculated, and the calculation result is scaled. In FIG. 4, the calculation of the inner product is denoted as "MatMul", and the scaling of the calculation result is denoted as "Scale".

[0045] Next, the attention mechanism applies the softmax function to the values ​​obtained through scaling to obtain a probability distribution representing the weights. In Figure 4, the application of the softmax function is denoted as "Softmax". In the example in Figure 4, B = 24 and N = 2, so a total of 48 weights are obtained from the combinations of B and N. However, in the probability distribution obtained by applying the softmax function, the sum of the 48 weights is 1.

[0046] Finally, the attention mechanism calculates the dot product of the weights obtained as described above and the value vector V, and assigns weights to the value vector V. The multi-head attention mechanism ultimately assigns weighted N ｈｅａｄ Outputs a value vector (F-dimensional vector).

[0047] The patch attachment unit 122 receives the output of N by the attention mechanism. ｈｅａｄ The weighted value vectors are combined. In Figure 4, the combination of value vectors is denoted as "Concat". The patch decoding unit 123 then performs a linear transformation on the combined value vectors to obtain a P-dimensional vector with the same number of dimensions as the original patch.

[0048] In this case, the P-dimensional vector is represented as Z in Figure 4. ｐｒｅｄ This is how it is written. This Z ｐｒｅｄ However, vector Z ｔｇｔ Vector Z for ｓｒｃ This corresponds to the contribution of the P-dimensional vector Z. In other words, the P-dimensional vector Z ｐｒｅｄ This corresponds to the contribution of the fourth patch from the right out of 24 patches (subsequence data) to the left patch of the two patches that are the inferred value.

[0049] In this way, the contribution calculation unit 12 calculates the contribution of each of the B patches to each of the N inferred patches by using an attention mechanism and a linear transformation using a linear model. The patch coding unit 121 uses a P-dimensional vector Z ｐｒｅｄ By applying a linear transformation, we obtain patch Y, which corresponds to the inference value to be predicted later. ｐｒｅｄ Calculate.

[0050] (3) Calculate the contribution by treating the predicted inference values ​​as patches. The basic flow of (3) is similar to (2) above, but in (3), the predicted inference values ​​are also treated as original patches (original subsequence data), and these original patches are encoded with the static variable information and relative or absolute time information mentioned above. An example of calculating the contribution by treating the predicted inference values ​​as patches will be explained with reference to Figure 5.

[0051] In Figure 5, the symbol X ｐａｔｃｈ represents the original patch (original subsequence data), and the sign Y ｐｒｅｄ In Figure 5, the symbol X represents the patch corresponding to the predicted inference value. ｓｔａｔ This represents static variable information and relative or absolute time information.

[0052] X ｐａｔｃｈ and Y ｐｒｅｄ These are all P-dimensional vectors, and X ｓｔａｔ This is an S-dimensional vector. In the example in Figure 5, for simplicity of explanation, the patch Y to be predicted is used. ｐｒｅｄ The number of elements is assumed to be 2. Note that Y ｐｒｅｄ This is a patch that corresponds to the predicted inference value, and since it is an unknown value, Y ｐｒｅｄ The initial value of is a vector where all elements are 0.

[0053] The patch encoding unit 121 performs encoding using, for example, a patch encoder. Specifically, the patch encoding unit 121 sends the original patch X to the patch encoder. ｐａｔｃｈ By sequentially inputting the data, an F-dimensional vector representing the feature representation for each patch is obtained.

[0054] Furthermore, the patch coding unit 121 inputs an S-dimensional vector representing variable information and time information to the patch encoder, as well as a patch Y corresponding to the inference value to be predicted. ｐｒｅｄBy sequentially inputting these values, an F-dimensional vector representing the feature representation for each patch corresponding to the predicted inference value is obtained. At this time, the S-dimensional vector representing the variable information and time information includes information about when the inference value is to be obtained (for example, tomorrow or the day after tomorrow). Therefore, the patch encoder can output an F-dimensional vector representing the feature representation that includes information indicating when the inference value is to be obtained.

[0055] The patch matching unit 122 then calculates the contribution by associating the original patch with the patch corresponding to the predicted inference value, using an attention mechanism, similar to the example described in (2). The attention mechanism associates the patch Y corresponding to the predicted inference value. ｐｒｅｄ Outputs two F-dimensional vectors corresponding to [the given value].

[0056] The patch decoding unit 123 performs decoding using, for example, a patch decoder. Specifically, the patch decoding unit 123 inputs the two F-dimensional vectors to the patch decoder and decodes each F-dimensional vector into a P-dimensional vector. The P-dimensional vector obtained in this case corresponds to patch Y, which is the inference value to be predicted. ｐｒｅｄ The original patch X ｐａｔｃｈ This corresponds to the contribution of [the person / group].

[0057] The contribution calculation unit 12 calculates the patch Y corresponding to the predicted inference value as described above. ｐｒｅｄ All original Patch X ｐａｔｃｈ The contribution is calculated. Based on the calculated contribution, the contribution calculation unit 12 calculates the predicted inference value, and if possible, uses the calculated inference value to calculate patch Y. ｐｒｅｄ Updated, updated patch Y ｐｒｅｄ The above process may be repeated using this method. In this case, the S-dimensional vector representing the variable information and time information may include information indicating a point in the future beyond the point in time described above (for example, three or four days later).

[0058] <ST3: Processing by the Contribution Aggregation Unit 13> Next, an example of processing by the Contribution Aggregation Unit 13 will be described.

[0059] The contribution aggregation unit 13 calculates the final inferred value at a predetermined time or period based on the sum of the contributions for each patch calculated by the contribution calculation unit 12. Specifically, the contribution aggregation unit 13 calculates the sum of the contributions for each patch calculated by the contribution calculation unit 12, and uses the calculated result as the final inferred value at a predetermined time or period. At this time, the contribution aggregation unit 13 may add a predetermined constant when calculating the sum of the contributions for each patch.

[0060] The contribution aggregation unit 13 performs the following processing, for example, depending on what task the inference device 10 processes through inference.

[0061] (1) Time series forecasting Time series forecasting is the task of predicting time series data from the present to H time points into the future, based on multivariate time series data. In this case, the contribution of each of the multiple patches obtained by extracting the multivariate time series data to the inferred value is represented by an H-dimensional vector.

[0062] Here, if the number of patches cut out by the patch cutting unit 11 is, for example, B, the contribution aggregation unit 13 calculates the sum of the B H-dimensional vectors calculated by the contribution calculation unit 12, and uses the obtained H-dimensional vector as the inferred value at a point in time H times ahead from the present.

[0063] The above example can be applied when there is only one variable to predict, but there may be cases where you want to output the inferred value in H x M dimensions (where M is an integer of 2 or more), for example, when there are multiple variables to predict. In that case, the contribution calculation unit 12 calculates the contribution that the patch gives to the inferred value as an H x M dimension vector, and the contribution aggregation unit 13 calculates the sum of the B H x M dimension vectors calculated by the contribution calculation unit 12, and the resulting H x M dimension vector can be used as the inferred value.

[0064] Furthermore, in cases such as predicting quantiles that indicate the boundaries when the data is divided into several parts, in addition to the mean value of a certain variable, or outputting other predictive distribution parameters such as the standard deviation in addition to the mean value of a certain variable, the contribution calculation unit 12 calculates the contribution that the patch has to the inferred value as an H × M dimensional vector, and the contribution aggregation unit 13 calculates the sum of the B H × M dimensional vectors calculated by the contribution calculation unit 12, and uses the resulting H × M dimensional vector as the inferred value.

[0065] (2) Time series classification Time series classification is the task of classifying future time series data into C classes based on multivariate time series data. In this case, the contribution of each of the multiple patches obtained by the patch extraction unit 11 extracting the multivariate time series data to the inferred value is represented by a C-dimensional vector.

[0066] If, for example, there are B patches, the contribution aggregation unit 13 calculates the sum of the B C-dimensional vectors calculated by the contribution calculation unit 12, and applies the softmax function to the resulting C-dimensional vector. The contribution aggregation unit 13 then uses the class with the largest output value from the softmax function as the inference value.

[0067] In particular, in the case of binary classification where the task is C=2, the contribution calculation unit 12 calculates the contribution as a scalar rather than a vector, and the contribution aggregation unit 13 calculates the sum of the B scalars calculated by the contribution calculation unit 12. The contribution aggregation unit 13 may then apply a logistic function to the calculated sum and calculate the class, which is an inferred value, based on the output value.

[0068] (3) Time series transformation (including denoising and interpolation) Time series transformation is the task of transforming time series data of length L into time series data of the same length L, based on multivariate time series data. In this case, the contribution of each of the multiple patches obtained by the patch extraction unit 11 to the inferred value is represented by an L-dimensional vector.

[0069] Here, if the number of patches is, for example, B, the contribution aggregation unit 13 calculates the sum of the B L-dimensional vectors calculated by the contribution calculation unit 12, and uses the resulting L-dimensional vector as the inference value.

[0070] This time series transformation may involve inferring partial time series data at a point in time or period prior to the present, based on multivariate time series data. In this case, the partial time series data obtained through inference may have noise removed from the original partial time series data at that past point in time or period. Alternatively, this time series transformation may involve inferring missing partial time series data at a point in time or period prior to the present, based on multivariate time series data. In this case, the partial time series data obtained through inference may interpolate the missing partial time series data.

[0071] <ST4: Processing by the explanatory display unit 14> Next, an example of processing by the explanatory display unit 14 will be explained with reference to Figure 6.

[0072] The explanatory display unit 14 displays the contribution that each patch extracted by the patch extraction unit 11 makes to the final inference value.

[0073] For example, the explanatory display unit 14 displays on the display device 20 the contribution of each patch to the final inference value, that is, the contribution of each patch calculated by the contribution calculation unit 12. Here, the explanatory display unit 14 may also display on the display device 20 the final inference value at a predetermined time or period, calculated by the contribution aggregation unit 13, along with the contribution of each patch calculated by the contribution calculation unit 12. In this case, the user of the inference device 10 (hereinafter simply referred to as "user") can confirm not only the contribution of each patch to the final inference value, but also the final inference value itself.

[0074] Figure 6 shows an example of a display when the explanatory display unit 14 displays the contribution for each patch calculated by the contribution calculation unit 12 and the final inferred value calculated by the contribution aggregation unit 13 on the display device 20. This example shows the case where the electricity price (y) for the next 24 hours is predicted from the electricity price (y), system load (system_load) for the past 168 hours, and the power generation forecast (gen_forecast) for the past 168 hours to the next 24 hours. In Figure 6, the horizontal axis represents time, and the electricity price for the past 168 hours, system load, and power generation forecast for the past 168 hours to the next 24 hours are shown as solid line graphs.

[0075] In this example, as shown in Figure 6, the patch extraction unit 11 extracts sub-series data as patches every 24 hours from multivariate time series data consisting of three variables: electricity price, system load, and power generation forecast. In this example, patches are extracted in such a way that there is no overlap or gaps between adjacent patches. In this example, in addition to the power generation forecast for the past 168 hours, the power generation forecast for the next 24 hours is also used to forecast the electricity price for the next 24 hours.

[0076] In this example, the contribution calculation unit 12, when calculating the patch-by-patch contribution to electricity prices for the next 24 hours, uses information indicating the target electricity market as static variable information, and information indicating the day of the week and time as relative or absolute time information.

[0077] In Figure 6, the dotted line indicated by reference numeral 601 represents the final inferred value calculated by the contribution aggregation unit 13, i.e., the electricity price for the next 24 hours.

[0078] Furthermore, in Figure 6, the dotted line indicated by reference numeral 602 represents the sum of the contributions of each patch extracted for the variable y to the electricity price for the next 24 hours. In addition, the contribution of each patch extracted for the variable y to the electricity price for the next 24 hours is displayed for each patch as a stacked bar graph indicated by reference numeral 603. In other words, among the patches extracted for the variable y, the larger the area occupied by the stacked bar graph indicated by reference numeral 603, the greater the contribution to the final inferred value, the electricity price for the next 24 hours.

[0079] Furthermore, in Figure 6, the dotted line indicated by reference numeral 604 represents the sum of the contributions of each patch extracted for the variable system_load to the electricity price for the next 24 hours. In addition, the contribution of each patch extracted for the variable system_load to the electricity price for the next 24 hours is displayed for each patch as a stacked bar graph indicated by reference numeral 605. In other words, among the patches extracted for the variable system_load, the larger the area occupied by the stacked bar graph indicated by reference numeral 605, the greater the contribution to the final inferred value, the electricity price for the next 24 hours.

[0080] Furthermore, in Figure 6, the dotted line indicated by reference numeral 606 represents the sum of the contributions of each patch extracted for the variable gen_forecast to the electricity price for the next 24 hours. In addition, the contribution of each patch extracted for the variable gen_forecast to the electricity price for the next 24 hours is displayed for each patch as a stacked bar graph indicated by reference numeral 607. In other words, among the patches extracted for the variable gen_forecast, the larger the area occupied by the stacked bar graph indicated by reference numeral 607, the greater the contribution to the final inferred value, which is the electricity price for the next 24 hours.

[0081] In summary, in the example in Figure 6, of the patches relating to the variables y, system_load, and gen_forecast, the patch with the largest contribution to the final inferred value, which is the electricity price for the next 24 hours, is patch 4 of variable y, i.e., the electricity price from 71 hours ago to 48 hours ago.

[0082] In this way, the explanatory display unit 14 displays the contribution to the final inferred value on the display device 20 by breaking it down patch by patch. For example, as described above, in the image displayed on the display device 20, the explanatory display unit 14 displays the contribution of each variable to the final inferred value for each variable in each row of the column labeled "decomposition" in Figure 6, and along with this display, it also displays the contribution of each patch to the final inferred value as a stacked bar graph, for example, by using different colors for each patch. This allows the explanatory display unit 14 to clearly explain which variable and which patch contributed to the final inferred value and to what extent. Furthermore, the user can intuitively and easily grasp, through visual means, which variables contribute to the final inferred value and to what extent, or which patches contribute to the final inferred value and to what extent.

[0083] Steps ST1 to ST4 in the flowchart described above may be executed only once, or they may be executed multiple times in a sequential order. Here, when steps ST1 to ST4 are executed only once, it is called a "single inference," and when steps ST1 to ST4 are executed multiple times in a sequential order, it is called a "multiple inference."

[0084] In a single inference, the patch extraction unit 11 extracts the patches, the contribution calculation unit 12 calculates the contribution for each patch, and the contribution aggregation unit 13 calculates the final inference value based on the sum of the contributions for each patch, each of which is performed once. The explanatory display unit 14 also displays the contribution for each patch once. In this case, the user can intuitively and easily grasp the contribution of each patch to the final inference value in a single inference through visual means.

[0085] In multiple inferences, the multivariate time-series data input to the patch extraction unit 11 changes moment by moment, and consequently, the content of the patches extracted by the patch extraction unit 11 also changes moment by moment. As a result, the calculation of the contribution for each patch by the contribution calculation unit 12, and the calculation of the final inference value based on the sum of the contributions for each patch by the contribution aggregation unit 13 are performed continuously in a time series, and the content of the display of the contribution for each patch by the explanatory display unit 14 also changes in a time series. This allows the user to intuitively and easily grasp, through visual means, the changes in the contribution of each patch to the final inference value that change over time.

[0086] As described above, the inference device 10 includes a patch extraction unit 11, a contribution calculation unit 12, a contribution aggregation unit 13, and an explanatory display unit 14. This allows the inference device 10 to decompose and display the contribution of each patch to the final inference value, and to clearly explain which patches contributed to the final inference value and to what extent among the multivariate time series data.

[0087] Next, with reference to Figure 7, an example of the hardware configuration of the inference device 10 according to Embodiment 1 will be described. The functions of the patch extraction unit 11, contribution calculation unit 12, contribution aggregation unit 13, and explanation display unit 14 in the inference device 10 are realized by a processing circuit. The processing circuit may be dedicated hardware as shown in Figure 7A, or it may be a CPU (Central Processing Unit, central processing unit, processing unit, arithmetic unit, microprocessor, microcomputer, processor, or DSP (Digital Signal Processor)) 52 that executes a program stored in memory 53, as shown in Figure 7B.

[0088] If the processing circuit is dedicated hardware, the processing circuit 51 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof. The functions of each part, the patch extraction unit 11, the contribution calculation unit 12, the contribution aggregation unit 13, and the explanation display unit 14, may be implemented individually by the processing circuit 51, or the functions of each part may be implemented together by the processing circuit 51.

[0089] When the processing circuit is a CPU 52, the functions of the patch extraction unit 11, contribution calculation unit 12, contribution aggregation unit 13, and explanation display unit 14 are realized by software, firmware, or a combination of software and firmware. The software and firmware are written as programs and stored in memory 53. The processing circuit realizes the functions of each unit by reading and executing the programs stored in memory 53. In other words, the inference device 10 is equipped with memory for storing programs that, when executed by the processing circuit, result in the execution of each step shown in Figure 5, for example. These programs can also be said to cause the computer to execute the procedures and methods of the patch extraction unit 11, contribution calculation unit 12, contribution aggregation unit 13, and explanation display unit 14. Here, the memory 53 includes, for example, non-volatile or volatile semiconductor memory such as RAM (RAM Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable ROM), EEPROM (Electrically EPROM), magnetic disks, flexible disks, optical disks, compact disks, minidiscs, or DVDs (Digital Versatile Discs).

[0090] Furthermore, the functions of the patch extraction unit 11, contribution calculation unit 12, contribution aggregation unit 13, and explanation display unit 14 may be partially implemented by dedicated hardware and partially by software or firmware. For example, the patch extraction unit 11 can be implemented by a processing circuit as dedicated hardware, while the contribution calculation unit 12, contribution aggregation unit 13, and explanation display unit 14 can be implemented by a processing circuit reading and executing a program stored in memory 53.

[0091] Thus, the processing circuit can realize each of the above-mentioned functions through hardware, software, firmware, or a combination thereof.

[0092] As described above, according to this embodiment 1, the inference device 10 is configured to include: a patch extraction unit 11 that extracts partial time series data as patches for each variable from multivariate time series data in which multiple variables change over time; a contribution calculation unit 12 that calculates the contribution to the inferred value at a predetermined time or period for each patch extracted by the patch extraction unit 11; a contribution aggregation unit 13 that calculates the final inferred value at a predetermined time or period based on the sum of the contributions for each patch calculated by the contribution calculation unit 12; and an explanatory display unit 14 that displays the contribution that each patch extracted by the patch extraction unit 11 has to the final inferred value for each patch. As a result, the inference device 10 according to embodiment 1 can explain which partial time series data from the multivariate time series data contributed to the inferred value. Furthermore, users of the inference device 10 can intuitively and easily grasp, through visual means, which partial time series data from the multivariate time series data contributed to the inferred value.

[0093] Furthermore, the explanatory display unit 14 displays the final inference value calculated by the contribution aggregation unit 13. This allows the inference device 10 according to Embodiment 1 to present the final inference value calculated by the contribution aggregation unit 13 to the user. In addition, the user can intuitively and easily grasp the final inference value visually.

[0094] Furthermore, the contribution calculation unit 12 includes a patch coding unit 121 that calculates a feature representation for each patch extracted by the patch extraction unit 11, a patch mapping unit 122 that maps the feature representations for each patch calculated by the patch coding unit 121 to each other, and a patch decoding unit 123 that calculates the contribution of each patch to the inference value at a predetermined time or period from the feature representations mapped by the patch mapping unit 122. As a result, the inference device 10 according to Embodiment 1 can map the feature representation for each patch to the feature representation for the inference value at a predetermined time or period and calculate the contribution of each patch to the inference value.

[0095] Furthermore, the patch mapping unit 122 uses an attention mechanism to associate the feature representations of each patch calculated by the patch coding unit 121. As a result, the inference device 10 according to Embodiment 1 can accurately calculate the contribution of each patch to the inference value.

[0096] Furthermore, the contribution calculation unit 12 calculates the contribution for each patch by considering static variable information that does not change over time, in addition to the patches extracted by the patch extraction unit 11. As a result, the inference device 10 according to Embodiment 1 can accurately calculate the contribution of each patch to the inferred value.

[0097] Furthermore, the contribution calculation unit 12 calculates the contribution of each patch by considering relative or absolute time information with respect to the present, in addition to the patches extracted by the patch extraction unit 11. As a result, the inference device 10 according to Embodiment 1 can accurately calculate the contribution of each patch to the inferred value.

[0098] Furthermore, the explanatory display unit 14 displays the contribution of each patch to the final inference value calculated by the contribution aggregation unit 13 in a single inference. As a result, the inference device 10 according to Embodiment 1 can present the user with the contribution of each patch to the inference value in a single inference. Moreover, the user can intuitively and easily grasp the contribution of each patch to the inference value in a single inference through visual means.

[0099] Furthermore, the explanatory display unit 14 displays the contribution to the final inference value calculated by the contribution aggregation unit 13 in multiple inferences, patch by patch. This allows the inference device 10 according to Embodiment 1 to show the user how the contribution of each patch to the inference value changes over time. In addition, the user can intuitively and easily grasp how the contribution of each patch to the inference value changes over time through visual means.

[0100] Furthermore, the predetermined time or period mentioned above is a time or period in the future from the present. This allows the inference device 10 according to Embodiment 1 to explain which partial time series data from the multivariate time series data contributed to the inference value at a time or period in the future from the present.

[0101] Furthermore, the predetermined time or period mentioned above is a time or period in the past than the present. This allows the inference device 10 according to Embodiment 1 to explain which partial time series data from the multivariate time series data contributed to the inference value at a time or period in the past than the present.

[0102] Furthermore, the explanatory display unit 14 displays the contribution of each patch extracted by the patch extraction unit 11 to the final inferred value as a stacked bar graph for each patch. This allows the inference device 10 according to Embodiment 1 to present to the user the contribution of each patch to the final inferred value, broken down by patch. The user can also intuitively and easily grasp the contribution of each patch to the final inferred value through visual means.

[0103] Embodiment 2. Embodiment 1 described an inference device 10 capable of explaining which patches in multivariate time series data contributed to the inferred value. Embodiment 2 describes an inference device 10b that is expected to improve the inference accuracy compared to the inference device 10 of Embodiment 1.

[0104] Figure 8 shows an example of the configuration of the inference device 10b according to Embodiment 2. The inference device 10b according to Embodiment 2 has a learning unit 15 added to the inference device 10 according to Embodiment 1.

[0105] The learning unit 15 updates the parameters of the machine learning model, such as a neural network, used by the contribution calculation unit 12, so that the final inference value calculated by the contribution aggregation unit 13 at a predetermined time or period approaches the true value.

[0106] For example, the learning unit 15 obtains the final inference value from the contribution aggregation unit 13 and also obtains the true value from an external device other than the inference device 10b. Then, based on the obtained inference value and true value, the learning unit 15 updates the parameters of the machine learning model used by the contribution calculation unit 12 and outputs the updated parameters to the contribution calculation unit 12. The contribution calculation unit 12 uses the updated parameters to calculate the contribution to the inference value at a predetermined point in time or period for each patch.

[0107] The learning unit 15 typically uses stochastic gradient descent to update the parameters of the machine learning model, such as a neural network, to minimize the loss function used by the machine learning model for learning. For example, if the task being processed is time series forecasting, the learning unit 15 updates the parameters of the machine learning model to minimize the mean absolute error (MAE) or mean squared error (MSE) between the inferred value and the true value.

[0108] Furthermore, the learning unit 15 may minimize a quantile loss, which is a loss function that calculates a loss based on a specific quantile, in order to calculate quantiles. The learning unit 15 may also minimize a distribution loss, such as KL divergence, in order to calculate parameters that indicate the distribution of the predicted data. Additionally, the learning unit 15 may minimize a weighted sum of multiple loss functions.

[0109] Furthermore, if the task to be processed is time-series classification, for example, the learning unit 15 may minimize the cross-entropy loss, which is a loss function used to measure the difference between the actual class (true value) and the class predicted by the model (inferred value) in the classification problem.

[0110] Thus, by including the learning unit 15, the inference device 10b according to Embodiment 2 can reduce the error between the final inferred value and the true value at a predetermined time or period, and an improvement in inference accuracy can be expected compared to the inference device 10 according to Embodiment 1.

[0111] As described above, according to this second embodiment, the inference device 10b includes a learning unit 15 that updates the parameters of the machine learning model used by the contribution calculation unit 12 so that the final inference value calculated by the contribution aggregation unit 13 approaches the true value. As a result, in addition to the effects of the first embodiment, the inference device 10b can be expected to show improved inference accuracy compared to the inference device 10 according to the first embodiment.

[0112] Although preferred embodiments have been described in detail above, the invention is not limited to the embodiments described above, and various modifications and substitutions can be made to the embodiments described above without departing from the scope of the claims.

[0113] Furthermore, this disclosure allows for free combination of each embodiment, modification of any component of each embodiment, or omission of any component in each embodiment.

[0114] This disclosure makes it possible to explain which partial time series data within multivariate time series data contributed to the inferred value, and is suitable for use in inference devices and inference methods. For example, this disclosure can be used when inferring future electricity prices from historical electricity price data, or when inferring future electricity demand from historical electricity demand data. This disclosure can also be used when inferring future sales of a product from past sales data of the same product, or when inferring future inventory levels of a product from past inventory levels of the same product. In any case, this disclosure makes it possible to explain which partial time series data within multivariate time series data contributed to the inferred value.

[0115] 10, 10b Inference unit, 11 Patch extraction unit, 12 Contribution calculation unit, 13 Contribution aggregation unit, 14 Explanation display unit, 15 Learning unit, 121 Patch coding unit, 122 Patch mapping unit, 123 Patch decoding unit, 51 Processing circuit, 52 CPU, 53 Memory.

Claims

1. An inference device comprising: a patch extraction unit that extracts partial time series data as patches for each variable from multivariate time series data in which multiple variables change over time; a contribution calculation unit that calculates the contribution to the inferred value at a predetermined time or period for each patch extracted by the patch extraction unit; a contribution aggregation unit that calculates the final inferred value at the predetermined time or period based on the sum of the contributions for each patch calculated by the contribution calculation unit; and an explanatory display unit that displays the contribution that each patch extracted by the patch extraction unit has to the final inferred value for each patch.

2. The inference device according to claim 1, characterized in that the explanatory display unit displays the final inference value calculated by the contribution aggregation unit.

3. The inference apparatus according to claim 1 or 2, wherein the contribution calculation unit includes: a patch coding unit that calculates a feature representation for each patch extracted by the patch extraction unit; a patch mapping unit that maps the feature representations for each patch calculated by the patch coding unit to each other; and a patch decoding unit that calculates the contribution of each patch to the inference value at a predetermined time or period from the feature representations mapped by the patch mapping unit.

4. The inference device according to claim 3, characterized in that the patch mapping unit uses an attention mechanism to map the feature representations of each patch calculated by the patch coding unit to each other.

5. The inference apparatus according to any one of claims 1 to 4, characterized in that the contribution calculation unit calculates the contribution for each patch by considering static variable information that does not change over time, in addition to the patches extracted by the patch extraction unit.

6. The inference device according to any one of claims 1 to 5, characterized in that the contribution calculation unit calculates the contribution for each patch by considering relative or absolute time information with respect to the present, in addition to the patches cut out by the patch cutting unit.

7. The inference apparatus according to any one of claims 1 to 6, characterized in that the explanatory display unit displays the contribution to the final inference value calculated by the contribution aggregation unit in a single inference for each patch.

8. The inference apparatus according to any one of claims 1 to 6, characterized in that the explanatory display unit displays the contribution to the final inference value calculated by the contribution aggregation unit in multiple inferences, for each patch.

9. The inference device according to any one of claims 1 to 8, characterized in that the predetermined time or period is a time or period in the future from the present.

10. The inference device according to any one of claims 1 to 8, characterized in that the predetermined time or period is a time or period in the past than the present.

11. The inference apparatus according to any one of claims 1 to 10, characterized in that the explanatory display unit displays the contribution of each patch cut out by the patch cutting unit to the final inference value as a stacked bar graph for each patch.

12. The inference device according to any one of claims 1 to 11, further comprising a learning unit that updates the parameters of the machine learning model used by the contribution calculation unit so that the final inference value calculated by the contribution aggregation unit approaches the true value.

13. An inference method by an inference device, comprising: a step of a patch extraction unit extracting partial time series data as patches for each variable from multivariate time series data in which multiple variables change over time; a step of a contribution calculation unit calculating the contribution to the inferred value at a predetermined time or period for each patch extracted by the patch extraction unit; a step of a contribution aggregation unit calculating the final inferred value at the predetermined time or period based on the sum of the contributions for each patch calculated by the contribution calculation unit; and a step of an explanatory display unit displaying the contribution that each patch extracted by the patch extraction unit has to the final inferred value for each patch.

Images