Identification device, identification method, and program

Abstract

To handle even relatively complicated time series data when identifying time series data.SOLUTION: An identification device comprises a model calculation unit and a class selection unit. On the basis of a state transition model in which the state transition probability changes with time and the states thereof are associated with classes of an identification object and in which data based on a probability distribution associated with a state in a transition destination occurs, the model calculation unit calculates a likelihood of transition to each of the states under input time series data and calculates a posterior probability that data is classified into each of the classes of the identification object under the input time series data for each of the classes on the basis of the likelihood. The class selection unit selects one of classes on the basis of the calculated posterior probabilities.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an identification device, an identification method, and a program. [Background technology]

[0002] Time series data may be identified. For example, Patent Document 1 describes the identification of time-series signals using a neural network incorporating a hidden Markov model. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 3816762 Summary of the Invention [Problem to be solved by the invention]

[0004] When identifying time series data, it is preferable to be able to handle time series data that is as complex as possible.

[0005] An example of an object of the present invention is to provide a classification device, a classification method, and a program that can also handle relatively complex time series data when classifying the time series data. [Means for solving the problem]

[0006] According to a first aspect of the present invention, an identification device includes a model calculation unit that calculates, for each state based on input time series data, a likelihood of transitioning to that state based on the state transition model, in which a class to be identified is associated with a state of a state transition model in which the state transition probability changes over time, and in which data is generated based on a probability distribution associated with the destination state, and calculates, for each class to be identified based on the likelihood, a posterior probability that data will be classified into that class based on the input time series data; and a class selection unit that selects one of the classes based on the calculated posterior probability.

[0007] According to a second aspect of the present invention, a classification method includes the steps of: calculating, by a computer, a likelihood of transitioning to each state based on input time series data, based on a state transition model in which a class to be classified corresponds to a state of a state transition model in which state transition probabilities change over time, and in which data is generated based on a probability distribution associated with the destination state; calculating, based on the likelihood, a posterior probability that data will be classified into each class to be classified based on the input time series data; and selecting one of the classes based on the calculated posterior probability.

[0008] According to a third aspect of the present invention, a program causes a computer to execute the following steps: calculate, based on input time series data, a likelihood of transitioning to each state based on a state transition model in which a class to be identified corresponds to a state of a state transition model in which state transition probabilities change over time, and data is generated based on a probability distribution associated with a destination state; calculate, based on the likelihood, a posterior probability that data will be classified into each class to be identified based on the input time series data; and select one of the classes based on the calculated posterior probability. [Effects of the Invention]

[0009] According to the above-described identification device, identification method, and program, when identifying time series data, it is possible to handle relatively complicated time series data. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of an identification device according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of the configuration of a recursive calculation unit included in an HMM likelihood calculation unit according to the embodiment. [Figure 3] FIG. 10 is a time-series graph showing an example of input data used in Experiment 1 according to the embodiment. [Figure 4] FIG. 10 is a graph showing an example of input data used in Experiment 1 according to the embodiment. [Figure 5] FIG. 10 is a diagram showing the results of Experiment 1 according to the embodiment. [Figure 6] FIG. 10 is a diagram showing an example of input data used in Experiment 2 according to the embodiment. [Figure 7] FIG. 10 is a diagram showing the results of Experiment 2 according to the embodiment. [Figure 8] FIG. 12 is a diagram showing 12 actions in Experiment 3 according to the embodiment. [Figure 9] FIG. 10 is a diagram showing an example of the position of a myoelectric potential sensor in Experiment 3 according to the embodiment. [Figure 10] FIG. 10 is a diagram showing an example of a sensor measurement signal in Experiment 3 according to the embodiment. [Figure 11] FIG. 10 is a diagram showing the results of Experiment 3 according to the embodiment. [Figure 12] FIG. 10 is a diagram showing the results of Experiment 4 according to the embodiment. [Figure 13] FIG. 1 is a diagram illustrating an example of a configuration of a computer according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] The following describes embodiments of the present invention, but the following embodiments do not limit the scope of the invention as claimed. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0012] 1 is a diagram illustrating an example of the configuration of a classification device according to an embodiment. In the configuration illustrated in Fig. 1, the classification device 10 includes a nonlinear transformation unit 110, an input unit 120, a component likelihood calculation unit 130, a GMM likelihood calculation unit 140, an HMM likelihood calculation unit 150, a posterior probability calculation unit 160, and a class selection unit 170. The nonlinear transformation unit 110 , input unit 120 , component likelihood calculation unit 130 , GMM likelihood calculation unit 140 , HMM likelihood calculation unit 150 , and posterior probability calculation unit 160 are collectively referred to as model calculation unit 100 .

[0013] The identification device 10 receives input data and performs identification (classification) of the input data. In particular, the identification device 10 performs identification of the input data based on a hidden Markov model that is extended so that state transition probabilities change over time.

[0014] The time-varying state transition probability is also referred to as an adaptive transition probability. A hidden Markov model in which the state transition probability changes over time is also referred to as an adaptive hidden Markov model (AHMM).

[0015] The adaptive hidden Markov model is an example of a state transition model in which state transition probabilities change over time. In particular, the adaptive hidden Markov model in the classification device 10 is an example of a state transition model in which a class to be classified corresponds to a state, and data is generated based on a probability distribution associated with the state to be transitioned to.

[0016] In the following, an example will be described in which the classification device 10 is configured using a neural network. A neural network that performs calculations of an adaptive hidden Markov model will also be referred to as an adaptive hidden Markov network (AHMN).

[0017] The identification device 10 may be implemented as software using a computer, or may be implemented using dedicated hardware such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0018] The nonlinear transformation unit 110 performs nonlinear transformation on the input data to the identification device 10 . The nonlinear transformation performed by the nonlinear transformation unit 110 can be expressed as in equation (1).

[0019]

number

[0020] Here, the input data to the identification device 10 is represented by x, and x=[x1, x2, . . . , x D ] where D is an integer D≧1 indicating the number of elements of data x. For example, if data x is sensor data, D may represent the number of sensors. A superscript "T" on a matrix or vector indicates the transpose of the matrix or vector. X indicates the output data of the nonlinear transformation unit 110 . H is an integer H≧1 that indicates the number of elements of data X. H denotes the H-dimensional real vector space. The nonlinear transformation unit 110 nonlinearly transforms data x represented by a D-dimensional real vector into data X represented by an H-dimensional real vector. The nonlinear transformation performed by the nonlinear transformation unit 110 can be considered as preprocessing of input data to the neural network included in the classification device 10.

[0021] The input unit 120 functions as an input layer of the neural network included in the identification device 10, and performs identity transformation on the data input to the input unit 120. The input data to the input unit 120 is expressed as in equation (2).

number

[0022] X h (t) indicates the value at time t of the h-th element of data X, which is output data from nonlinear transformation section 110. Here, h is an integer in the range 1≦h≦H. Here, time is expressed in time steps, where t is an integer t≧1.

[0023] (1) I h (t) indicates the value of the h-th element of the input data to the input unit 120 at time t. Here, the input data to the i-th layer of the neural network is (i) Denoted by I, the output data of the ith layer of the neural network is (i) It shall be indicated by O. Furthermore, the input unit 120 is treated as the first layer of the neural network included in the classification device 10. The output data of the input unit 120 is expressed as in equation (3).

[0024]

number

[0025] (1) O h (t) indicates the value of the h-th element of the output data of the input unit 120 at time t.

[0026] The component likelihood calculation unit 130 calculates the likelihood (likelihood that input data occurs) of each component of a Gaussian mixture model (GMM). The component likelihood calculation unit 130 is treated as the second layer of the neural network included in the classification device 10. The input data to the component likelihood calculation unit 130 is expressed as in equation (4).

[0027]

number

[0028] c is an integer of 1≦c≦C that indicates (the identification number of) the class of the object to be identified by the identification device 10. C is an integer of C≧1 that indicates the class of the object to be identified by the identification device 10. k is the identification number of the state of the adaptive hidden Markov model, 1≦k≦K. c is an integer. c is the number of states corresponding to class c. c The adaptive hidden Markov model is an integer ≥ 1. c It is assumed that the object has individual states.

[0029] m indicates the component (identification number) of the Gaussian mixture distribution that composes class c, 1≦m≦M c,k is an integer. c,k is the number of components of the Gaussian mixture that make up state k of class c. c,k is an integer ≧ 1. The components of a Gaussian mixture here are the individual Gaussian distributions that make up the Gaussian mixture. The component likelihood calculation unit 130 is assumed to include a neuron for each class, for each state of the class, and for each component of the Gaussian mixture distribution that constitutes the state.

[0030] (2) I c,k,m (t) indicates the value of the input data to the neuron of the component likelihood calculation unit 130 corresponding to class c, state k, and component m at time t. w c,k,m,h indicates a weighting coefficient by which the h-th element of the output data of the input unit 120 is multiplied when it is input to a neuron of the component likelihood calculation unit 130 corresponding to class c, state k, and component m. c,k,m,h are the learning targets when the classification device 10 is trained.

[0031] Here, the output data of the input unit 120 (1) O h The calculation of equation (4) for (t) is regarded as preprocessing performed by the component likelihood calculation unit 130. Alternatively, the output data of the input unit 120 (1) O h (t) is taken as input data to the component likelihood calculation unit 130, (2) I c,k,m (t) can also be regarded as intermediate data in the component likelihood calculation unit 130.

[0032] The output data of the component likelihood calculation unit 130 is expressed as in equation (5).

[0033]

number

[0034] (2) O c,k,m (t) indicates the value of the output data of the neuron of the component likelihood calculation unit 130 corresponding to class c, state k, and component m at time t. (2) O c,k,m (t) indicates the likelihood for each component. exp indicates the power of Napier's constant e.

[0035] The GMM likelihood calculation unit 140 calculates the likelihood that the input data occurs at the calculation target time for each Gaussian mixture distribution, based on the likelihood for each component calculated by the component likelihood calculation unit 130. The GMM likelihood calculation unit 140 is an example of a Gaussian mixture distribution likelihood calculation unit. The GMM likelihood calculation unit 140 is treated as the third layer of the neural network included in the classification device 10. The input data to the GMM likelihood calculation unit 140 is expressed as in equation (6).

[0036]

number

[0037] (3) I c,k (t) indicates the value of the input data to the neuron of the GMM likelihood calculation unit 140 corresponding to class c and state k at time t. (3) I c,k (t) indicates the likelihood for each Gaussian mixture distribution.

[0038] Here, the output data of the component likelihood calculation unit 130 is (2) O c,k,m The calculation of equation (6) for (t) is regarded as preprocessing performed by the GMM likelihood calculation unit 140. Alternatively, the output data of the component likelihood calculation unit 130 (2) O c,k,m (t) is taken as input data to the GMM likelihood calculation unit 140, (3) I c,k (t) can also be regarded as intermediate data in the GMM likelihood calculation unit 140. The output data of the GMM likelihood calculation unit 140 is expressed as in equation (7).

[0039]

number

[0040] (3) O c,k (t) indicates the value of the output data of the neuron corresponding to class c and state k in the GMM likelihood calculation unit 140 at time t. Log indicates the natural logarithm. (3) O c,k (t) indicates the log-likelihood for each Gaussian mixture distribution (each Gaussian mixture model).

[0041] The HMM likelihood calculation unit 150 calculates the likelihood that the input time series data up to the calculation target time will occur for each state of the adaptive hidden Markov model, based on the likelihood calculated for each Gaussian mixture distribution by the GMM likelihood calculation unit 140. The likelihood calculated by the HMM likelihood calculation unit 150 can also be considered as the likelihood of transitioning to each state based on the input time series data. The HMM likelihood calculation unit 150 is an example of an adaptive hidden Markov model likelihood calculation unit.

[0042] While the GMM likelihood calculation unit 140 calculates likelihoods according to states when state transitions are not taken into consideration, the HMM likelihood calculation unit 150 calculates likelihoods when state transitions are taken into consideration. The HMM likelihood calculation unit 150 is treated as the fourth layer of the neural network included in the classification device 10. The input data to the HMM likelihood calculation unit 150 is expressed as in equation (8).

[0043]

number

[0044] (4) I c,k (t) indicates the value of the input data to the neuron of the HMM likelihood calculation unit 150 corresponding to class c and state k at time t. (4) I c,k (t) denotes the likelihood of being in state k at time t. (4) O c,k (t) indicates the value of the output data of the neuron of the HMM likelihood calculation unit 150 corresponding to class c and state k at time t. G c,k (t) is (4) O c,k’ (t-1) and (3) O c,k (t) is a time-varying function that adjusts the ratio of G c,k (t) is also called the gate function.

[0045] Here, the output data of the GMM likelihood calculation unit 140 is (3) O c,k (t), and the output data of the HMM likelihood calculation unit 150 (4) O c,k’ The calculation of equation (10) for (t) is regarded as preprocessing performed by the HMM likelihood calculation unit 150. Alternatively, the output data of the GMM likelihood calculation unit 140 (3) O c,k (t) is taken as input data to the HMM likelihood calculation unit 150, (4) I c,k (t) can also be regarded as intermediate data in the HMM likelihood calculation unit 150. Output data of the HMM likelihood calculation unit 150 (4) O c,k (t) is expressed as equation (9).

[0046]

number

[0047] (4) O c,k (t) indicates the value of the output data of the neuron of the HMM likelihood calculation unit 150 corresponding to class c and state k at time t. 4) O c,k (t) denotes the log-likelihood of being in state k at time t.

[0048] The posterior probability calculation unit 160 calculates the posterior probability for each class for the input data to the classification device 10 based on the likelihood for each state of the adaptive hidden Markov model. The posterior probability calculation unit 160 is assumed to be treated as the fifth layer of the neural network included in the classification device 10. The input data to the posterior probability calculation unit 160 is expressed as in equation (10).

[0049]

number

[0050] T is an integer T≧1 that indicates the length of the time series data input to the identification device 10. Specifically, data is input to the identification device 10 at each time from time 1 to time T. Therefore, (4) O c,k,m (T) indicates the value of the output data of the neuron of the HMM likelihood calculation unit 150 corresponding to class c and state k when the time-series data has been input to the classification device 10 up to the end. (5) I c (t) indicates the value of the input data to the neuron corresponding to class c in the posterior probability calculation unit 160 at time t.

[0051] Here, the output data of the HMM likelihood calculation unit 150 is (4) O c,k The calculation of equation (10) for (T) is regarded as preprocessing performed by the posterior probability calculation unit 160. Alternatively, the output data of the HMM likelihood calculation unit 150 (4) O c,k (T) is taken as input data to the posterior probability calculation unit 160, (5) I c (t) can also be regarded as intermediate data in the posterior probability calculation unit 160. The output data of the posterior probability calculation unit 160 is expressed as in equation (11).

[0052]

number

[0053] (5) O c (t) indicates the value of the output data of the neuron corresponding to class c in the posterior probability calculation unit 160 at time t. (5) O c (t) represents the posterior probability that input data will be classified into class c when certain time series data is obtained as input data to the classification device 10, that is, the probability that the time series data will be classified into class c. The processing performed by the posterior probability calculation unit 160 can be considered as processing for applying a softmax function to the logarithmic likelihood calculated by the HMM likelihood calculation unit in the output layer of the neural network.

[0054] The classification device 10 can be trained using the evaluation function J shown in equation (12).

[0055]

number

[0056] n is an integer 1≦n≦N that indicates (the identification number of) the training data included in the training data set. N is an integer N≧1 that indicates the number of training data included in the training data set. Here, one training data includes one time-series data and a correct answer (teacher data) for that time-series data. Therefore, n also corresponds to the identification number that identifies the time-series data included in the training data set.

[0057] A c (n) indicates the correct data for time series data n. If class c is the correct class, A c (n) = 1. On the other hand, if class c is the incorrect class, A c (n) =0. The value of the posterior probability calculated by the posterior probability calculation unit 160 (5) O c The closer (t) is to the correct answer, the smaller the value of the evaluation function J. In the learning of the classification device 10, a machine learning method such as backpropagation is used to search for values ​​of the learning parameters that will result in a smaller value of the evaluation function J. The value of the reinforcement function J can be considered as the Kullback-Leibler divergence, which indicates the degree of difference between the distribution of the posterior probability calculated by the posterior probability calculation unit 160 and the probability distribution of the correct answer.

[0058] The class selection unit 170 selects one of the classes based on the posterior probability of each class. For example, the class selection unit 170 may select one class with the highest (largest) posterior probability. The class selected by the class selection unit 170 corresponds to the classification result by the classification device 10.

[0059] The processing performed by the identification device 10 will be further described. First, consider representing the process by which an event to be identified occurs using a hidden Markov model. Specifically, assume that each event to be identified corresponds to a state group consisting of one or more states, and that when an event occurs, there is a probabilistic transition between states belonging to the state group associated with that event at each time. Also assume that data x(t) is observed at each time according to the probability distribution determined for each state. In time series data identification, based on the observed time series data x(1), x(2), ..., x(T), for each state, the posterior probability that the state at time T is that state is calculated, and for each event, the posterior probabilities of the states associated with that event are summed.

[0060] Here, while the state transition probability in a hidden Markov model is a fixed value, we consider extending the model so that the state transition probability changes over time. This extension makes it possible to use the model to more accurately represent events observed using complex time-series data, and is expected to improve classification accuracy. A hidden Markov model in which the state transition probability changes over time, as described above, is also called an adaptive hidden Markov model.

[0061] Furthermore, the time-varying state transition probability is expressed as in equation (13) using a time-varying gate function.

[0062]

number

[0063] r c k’,k(t) denotes the transition probability from state k' to state k of class c at time t. G c,k (t) denotes the value of the gate function of state k of class c at time t. The gate function G c,k (t) is expressed as in equation (14).

[0064]

number

[0065] w G1 (c,k)T , w G2 (c,k)T are all weighting coefficient matrices. x(t) is a matrix function with time t as an argument. b c,k is a constant indicating the bias. When the classifier 10 is trained, w G1 (c,k)T , w G2 (c,k)T , and b c,k may be used as a parameter to be learned.

[0066] In equation (14), a sigmoid function is used to differentiably associate the domain of (-∞,∞) with the range of (0,1). The partial functions in the sigmoid function are not limited to those shown in equation (14), and various partial functions can be used. The parameters included in the partial functions can be used as learning target parameters when training the classification device 10.

[0067] According to equation (13), the parameter γ of the state transition probability in the adaptive hidden Markov model c k’,k (t) with parameter w G1 (c,k)T , w G2 (c,k)T , and b c,k and the parameter γ c k’,k (t) can be hidden.

[0068] l c,k’ (t-1) indicates the likelihood of transitioning to state k' of class c at time t-1. Σ m=1 Mc,k r c,k,m g(x(t);μ c,k,m ,Σ c,k,m ) denotes the Gaussian mixture distribution in state k for class c.

[0069] As mentioned above, M c,k denotes the number of components in state k of class c, M c,k is an integer ≧1. r c,k,m indicates the degree of mixing. μ c,k,m denotes the mean vector. Σ c,k,m denotes the covariance matrix.

[0070] Equation (13) is the likelihood information l at a time earlier than the time of the calculation target. c,k’ (t-1) and the likelihood information Σ at the time of calculation m=1 Mc,k r c,k,m g(x(t);μ c,k,m ,Σ c,k,m ) and the function G c,k This corresponds to an example of an equation where each is weighted using (t).

[0071] By introducing a gate function as in equation (13), the degree of learning between past information and current information can be adjusted over time during learning by the classification device 10. This is expected to enable the classification device 10 to classify relatively complex time-series data with higher accuracy. The adjustment over time here may be adjustment according to the value of input data that changes over time.

[0072] On the other hand, equation (13) is based on the degree of mixing r c,k,m , mean vector μ (c,k,m) , and the covariance matrix Σ (c,k,m)These parameters have value constraints due to statistical properties such as the above. These constraints make it difficult to search for parameter values ​​when training the classification device 10, and it is thought that this will prevent the classification accuracy of the classification device 10 from improving.

[0073] Therefore, the subexpression Σ of the Gaussian mixture distribution in equation (13) m=1 Mc,k r c,k,m g(x(t);μ c,k,m ,Σ c,k,m ) is replaced by a formula using weighting factors that do not have statistical constraints. Specifically, r c,k,m g(x(t);μ c,k,m ,Σ c,k,m ) is transformed into equation (15) using the log-linearization technique.

[0074]

number

[0075] w X (c,k,m) is the weighting coefficient matrix. X (c,k,m) is the weighting coefficient w in Eq. (4) c,k,m,h is equivalent to X(t) is the time series data element of X in equation (1) with time t explicitly indicated. Equation (15) is a Gaussian mixture distribution r c,k,m g(x(t);μ c,k,m ,Σ c,k,m ) as the weighting factor w X (c,k,m)T and the transformed input data X(t) to the power exp{w X (c,k,m)T This corresponds to the example of the equation shown as {X(t)}.

[0076] Weighting coefficient matrix w X (c,k,m) can be treated as a learning parameter (parameter to be learned) that has no statistical constraints. This makes it easier for the classification device 10 to learn, and is expected to improve classification accuracy.

[0077] Equation (15) corresponds to equations (4) and (5) of component likelihood calculation section 130. Equation (13) corresponds to equation (8) of the HMM likelihood calculation unit 150.

[0078] As shown in equation (8), the HMM likelihood calculation unit 150 is configured recursively. 2 is a diagram showing an example of the configuration of a recursive calculation unit included in HMM likelihood calculation unit 150. In the configuration shown in FIG. 2, recursive calculation unit 210 includes gate function calculation unit 211, subtraction unit 212, multiplication units 213 and 214, addition unit 215, and power calculation unit 216.

[0079] The recursive calculation unit 210 calculates the likelihood l of the state of the adaptive hidden Markov model at time t. c,k Here, the likelihood of the state of the adaptive hidden Markov model is called the HMM likelihood. The HMM likelihood calculation unit 150 includes a recursive calculation unit 210 for each Claus class and for each permutation (k, k') of two states of that class.

[0080] Each recursive calculation unit 210 calculates exp{(1-G c,k (t)) (4) O c,k’ (t-1)+G c,k (t) (3) O c,k (t)} is calculated. The gate function calculation unit 211 receives input data x(t-1) at time t-1 and input data x(t) at time t, and calculates the gate function value G at time t based on equation (14). c,k The gate function calculation unit 211 calculates the calculated gate function value G c,k (t) is output to the subtraction unit 212 and the multiplication unit 214.

[0081] The subtractor 212 subtracts the value of the gate function G c,k Subtract (t) from 1 to get 1-G c,k The subtraction unit 212 calculates the calculated 1-Gc,k (t) is output to the multiplication unit 213. The multiplication unit 213 multiplies the 1-G calculated by the subtraction unit 212 by c,k (t) and the output data of the HMM likelihood calculation unit at time t-1 (4) O c,k’ Multiply (t-1) and (1-G c,k (t)) (4) O c,k’ Calculate the value of (t-1). (4) O c,k’ (t-1) is log{l c,k’ (t-1)}. The multiplication unit 213 calculates (1-G c,k (t)) (4) O c,k’ The value of (t−1) is output to the adder 215 .

[0082] The multiplication unit 214 multiplies the value of the gate function G c,k (t) and the output data of the GMM likelihood calculation unit 140 at time t (3) O c,k Multiply (t) and G c,k (t) (3) O c,k Calculate the value of (t). (3) O c,k (t) is log{Σ m= 1Mc,k y c,k,m (t)}, where y c,k,m (t) indicates the likelihood of component m in state k of class c at time t. In the identification device 10, the output data of the component likelihood calculation unit 130 is (2) O c,k,m (t) to y c,k,m Used as (t). The multiplication unit 214 multiplies the calculated G c,k (t) (3) O c,k The value of (t) is output to the adder 215.

[0083] The adder 215 multiplies the (1-G c,k (t)) (4) O c,k’ (t-1) and G calculated by the multiplication unit 214c,k (t) (3) O c,k Add (t) and (1-G c,k (t)) (4) O c,k’ (t-1)+G c,k (t) (3) O c,k Calculate the value of (t). The adder 215 calculates (1-G c,k (t)) (4) O c,k’ (t-1)+G c,k (t) (3) O c,k The value of (t) is output to the power calculation unit 216.

[0084] The power calculation unit 216 calculates the power (1-G c,k (t)) (4) O c,k’ (t-1)+G c,k (t) (3) O c,k Raise (t) to a power and get exp{(1-G c,k (t)) (4) O c,k’ (t-1)+G c,k (t) (3) O c,k Calculate the value of {(t)}. The recursive calculation unit 210 calculates exp{(1-G c,k (t)) (4) O c,k’ (t-1)+G c,k (t) (3) O c,k (t)}.

[0085] The neuron of the HMM likelihood calculation unit 150 corresponding to the state k of the class c calculates the state k of the class c and the state M from the output values ​​of the recursive calculation unit 210 corresponding to the state k of the class c and the state 1. c,k log[Σ m=1 Mc,k exp{(1-G c,k (t)) (4) O c,k’ (t-1)+Gc,k (t) (3) O c,k (t)}] to the output data (4) O c,k Output as (t).

[0086] Output Data (4) O c,k (t) is input to the multiplication unit 213 in the next time step. (4) O c,k (t-1). The input data at time t is used as x(t-1) input to the gate function calculation unit 211 in the next time step.

[0087] Next, we will explain an experiment on the identification device 10. An experiment was conducted to confirm the identification ability of the identification device 10.

[0088] (Experiment 1) In Experiment 1, we conducted a classification experiment using artificial data (artificially generated data). Using a hidden Markov model with 4 classes, 5 states per class, 4 components per state, and 2 dimensions, we prepared artificial data with a sequence length of 50 for four classes as input data for the experiment. Here, the four classes are referred to as class 0, class 1, class 2, and class 3.

[0089] In Experiment 1, in addition to the classification device 10 using an adaptive hidden Markov model, classification experiments were also conducted using RLLGMN (Recurrent Log-Linearized Gaussian Mixture Network), LSTM (Long Short-Term Memory), and GRU (Gated Recurrent Unit) as comparison objects.

[0090] RLLGMN is a model that uses a hidden Markov model in which state transition probabilities do not change over time. LSTM and GRU are both examples of general recurrent neural networks (RNNs). Here, we define general recurrent neural networks as recurrent neural networks that are not explicitly based on hidden Markov models.

[0091] Figure 3 is a time-series graph showing an example of input data used in Experiment 1. The horizontal axis of the graph in Figure 3 represents time (hours), and the vertical axis represents the values ​​of Dimension 1 and Dimension 2. In Figure 3, time 0 to 100 is class 0 data, time 100 to 200 is class 1 data, time 200 to 300 is class 2 data, and time 300 to 400 is class 3 data.

[0092] Figure 4 is a diagram showing an example of input data used in Experiment 1 in the form of a data value graph. The horizontal axis of the graph in Figure 4 represents the value of dimension 1, and the vertical axis represents the value of dimension 2. In Figure 4, data values ​​are distributed for each state of each class, and overlapping of data value distributions can be seen between classes.

[0093] Figure 5 shows the results of Experiment 1. The vertical axis of the graph in Figure 5 shows the F1 score (F1-Score). The F1 score ranges from 0 to 1, and the higher the F1 score, the better the classification accuracy can be evaluated. The horizontal axis shows the classes to be classified for each model.

[0094] Bars B111 to B114 show the F1 scores for classes 0 to 3 obtained by the classification device 10. Bar B111 shows the F1 score for class 0. Bar B112 shows the F1 score for class 1. Bar B113 shows the F1 score for class 2. Bar B114 shows the F1 score for class 3.

[0095] Bars B121 to B124 show the F1 scores for classes 0 to 3 by RLLGMN. Bar B121 shows the F1 score for class 0. Bar B122 shows the F1 score for class 1. Bar B123 shows the F1 score for class 2. Bar B124 shows the F1 score for class 3.

[0096] Bars B131 to B134 show the F1 scores for classes 0 to 3 using LSTM. Bar B131 shows the F1 score for class 0. Bar B132 shows the F1 score for class 1. Bar B133 shows the F1 score for class 2. Bar B134 shows the F1 score for class 3.

[0097] Bars B141 to B144 show the F1 scores for classes 0 to 3 by the GRU. Bar B141 shows the F1 score for class 0. Bar B142 shows the F1 score for class 1. Bar B143 shows the F1 score for class 2. Bar B144 shows the F1 score for class 3.

[0098] 5, the classification device 10 exhibits better classification accuracy than RLLGMN. In particular, for class 0 and class 1, the classification device 10 exhibits significantly better classification accuracy than RLLGMN. The adaptive hidden Markov model used by the classification device 10 is more flexible than the hidden Markov model used by the RLLGMN, in which the state transition probability does not change over time, and this may have affected the classification accuracy.

[0099] Moreover, the classification device 10 exhibits better classification accuracy than the LSTM and GRU. For the complex data used in Experiment 1, which was expressed using a hidden Markov model and a Gaussian mixture distribution, the classification device 10 was able to more accurately represent the data using a flexible adaptive hidden Markov model, which is thought to have resulted in higher classification accuracy.

[0100] (Experiment 2) In Experiment 2, we conducted a motion recognition experiment using an inertial sensor. Twenty-four subjects were asked to identify six actions or states using data with 12 dimensions and a sequence length of 100. In the explanation of Experiment 2, actions and states are collectively referred to as actions.

[0101] The dimensions of the data are attitude, which represents tilt, gravity, rotation rate, and acceleration, each of which is represented by three-dimensional data. The six movements are: downstairs (dws), upstairs (ups), sitting (sit), standing (std), walking (wlk), and jogging (jog). In Experiment 2, each movement corresponds to a class.

[0102] Figure 6 is a diagram showing an example of input data used in Experiment 2. The horizontal axis of the graph in Figure 6 represents time (hours), and the vertical axis represents the values ​​of each of the 12 dimensions. The vertical axis shows the roll angle (roll), pitch angle (pitch), and yaw angle (yaw), or the x-coordinate (x), y-coordinate (y), and z-coordinate (z) values ​​for each of the four sensor measurement items mentioned above.

[0103] On the horizontal axis, time 0 to 100 is downstairs data, time 100 to 200 is upstairs data, time 200 to 300 is sitting data, time 300 to 400 is standing data, time 400 to 500 is walking data, and time 500 to 600 is jogging data. In Experiment 2, we also compared LDA (Latent Dirichlet Allocation), a classifier that does not take time series information into account.

[0104] Figure 7 shows the results of Experiment 2. The vertical axis of the graph in Figure 7 represents the F1 score, and the horizontal axis represents the actions to be identified for each model. Bars B211 to B216 show the F1 scores of the classification results for six actions by the classification device 10. Bar B211 shows the F1 score for downstairs. Bar B212 shows the F1 score for upstairs. Bar B213 shows the F1 score for sitting. Bar B214 shows the F1 score for standing. Bar B215 shows the F1 score for walking. Bar B216 shows the F1 score for jogging.

[0105] Bars B221 to B226 show the F1 scores of the six action classification results by RLLGMN. Bar B221 shows the F1 score for downstairs. Bar B222 shows the F1 score for upstairs. Bar B223 shows the F1 score for sitting. Bar B224 shows the F1 score for standing. Bar B215 shows the F1 score for walking. Bar B216 shows the F1 score for jogging.

[0106] Bars B231 to B236 show the F1 scores for the six action classification results using LSTM. Bar B231 shows the F1 score for downstairs. Bar B232 shows the F1 score for upstairs. Bar B233 shows the F1 score for sitting. Bar B234 shows the F1 score for standing. Bar B235 shows the F1 score for walking. Bar B236 shows the F1 score for jogging.

[0107] Bars B241 to B246 show the F1 scores for the six action classification results by the GRU. Bar B241 shows the F1 score for downstairs. Bar B242 shows the F1 score for upstairs. Bar B243 shows the F1 score for sitting. Bar B244 shows the F1 score for standing. Bar B245 shows the F1 score for walking. Bar B246 shows the F1 score for jogging.

[0108] Bars B251 to B256 show the F1 scores of the six action classification results by LDA. Bar B251 shows the F1 score for downstairs. Bar B252 shows the F1 score for upstairs. Bar B253 shows the F1 score for sitting. Bar B254 shows the F1 score for standing. Bar B255 shows the F1 score for walking. Bar B256 shows the F1 score for jogging.

[0109] 7, the classification device 10 exhibits stable and good classification accuracy for all classes. It can be evaluated that the classification device 10 exhibits better classification accuracy for almost all classes than any of the comparison models. It can be considered that the classification device 10 achieved higher classification accuracy because it was able to represent data more accurately using a flexible adaptive hidden Markov model. In particular, the classification device 10 exhibits classification accuracy that is significantly better than LDA. This difference in classification accuracy is thought to demonstrate the effectiveness of taking into account the temporal nature of the data.

[0110] (Experiment 3) In Experiment 3, we conducted a motion discrimination experiment using EMG data. A total of 10-channel electromyography sensors were attached to the subjects' arms, and they were tasked with identifying a total of 12 hand shapes or wrist movements. In the explanation of Experiment 3, hand shapes and wrist movements are collectively referred to as "movements."

[0111] In Experiment 3, electromyographic sensor data was measured for each of the four subjects. The electromyographic sensor data was full-wave rectified and passed through a 2 Hz (Hertz) low-pass filter. The sampling frequency was 200 Hz. In Experiment 3, the models compared were RLLGMN, LSTM, and GRU.

[0112] Figure 8 shows the 12 actions in Experiment 3. The 12 actions to be identified in Experiment 3 consist of eight hand shapes and four wrist movements. The eight hand shapes are M0: Thumbs up, M1: V-sign, M2: Three-sign (number 3 sign), M3: Four-sign (number 4 sign), M4: Paper hand (rock-paper-scissors), M5: Grasp (rock-paper-scissors), M6: Pointing (index finger raised), and M7: Hand sword (fingers together). The four wrist movements are M8: Extension (bending the wrist toward the palm), M9: Flexion (bending the wrist toward the back of the hand), M10: Radial flexion (bending the wrist toward the thumb), and M11: Ulnar flexion (bending the wrist toward the little finger).

[0113] Figure 9 shows an example of the positions of the myoelectric potential sensors in Experiment 3. As shown in Figure 9, sensors for channels 1 to 8 were attached to the forearm midway between the wrist and elbow, and sensors for channels 9 and 10 were attached to the forearm closer to the wrist.

[0114] Fig. 10 is a diagram showing an example of the sensor measurement signals in Experiment 3. The horizontal axis of the graph in Fig. 10 represents time (hours of the day). The horizontal axis is in seconds. The vertical axis represents the signal values ​​of each of the 10 channels. The horizontal axis shows 100 signals for each operation from M0 to M12.

[0115] Figure 11 shows the results of Experiment 3. The vertical axis of the graph in Figure 11 represents the F1 score, and the horizontal axis represents the actions to be identified for each model. Bars B311 to B31c show the F1 scores of the classification results for the 12 actions by the classification device 10. Bars B311 to B31c show the F1 scores of the classification results for the actions M0 to M11, respectively. Bars B321 to B32c show the F1 scores of the classification results of 12 actions by RLLGMN. Bars B321 to B32c show the F1 scores of the classification results of actions M0 to M11, respectively.

[0116] Bars B331 to B33c show the F1 scores of the classification results for 12 actions by LSTM. Bars B331 to B33c show the F1 scores of the classification results for actions M0 to M11, respectively. Bars B341 to B34c show the F1 scores of the GRU's classification results for 12 actions. Bars B341 to B34c show the F1 scores of the classification results for actions M0 to M11, respectively.

[0117] From the experimental results shown in FIG. 11, it can be evaluated that the classification device 10 exhibits better classification accuracy in all classes than any of the comparison models. It is believed that the classification accuracy was higher in the classification device 10 because the data could be represented more accurately using a flexible adaptive hidden Markov model. Also, while there were only four subjects in Experiment 3 and the amount of training data was small, it is believed that the classification device 10 can learn sufficiently even with a small amount of data.

[0118] (Experiment 4) In Experiment 4, an imbalanced data experiment was conducted using the data used in Experiment 3. In Experiment 4, we reduced the data for six of the 12 classes by 90% from the data used in Experiment 3, creating data with a discrepancy in the number of data points between classes. Specifically, we reduced the data for each of the actions M1, M3, M5, M7, M9, and M11 by 90%.

[0119] Normally, if a model is trained using such imbalanced data, the model tends to train biasedly toward the majority class, resulting in a decrease in the accuracy of classification for the minority class. Therefore, data balancing is generally performed, such as weighting the minority class. However, in Experiment 4, data balancing was not performed, and the model was trained using the imbalanced data as is, and the classification accuracy was compared.

[0120] Figure 12 shows the results of Experiment 4. The vertical axis of the graph in Figure 12 represents the F1 score, and the horizontal axis represents the actions to be identified for each model. Bars B411 to B41c show the F1 scores of the classification results for the 12 actions by the classification device 10. Bars B411 to B41c show the F1 scores of the classification results for the actions M0 to M11, respectively. Bars B421 to B42c show the F1 scores of the classification results of 12 actions by RLLGMN. Bars B421 to B42c show the F1 scores of the classification results of actions M0 to M11, respectively.

[0121] Bars B431 to B43c show the F1 scores of the classification results for 12 actions by LSTM. Bars B431 to B43c show the F1 scores of the classification results for actions M0 to M11, respectively. Bars B441 to B44c show the F1 scores of the GRU's classification results for 12 actions. Bars B441 to B44c show the F1 scores of the classification results for actions M0 to M11, respectively.

[0122] In the experimental results shown in Fig. 12, the comparison model has a significantly lower classification accuracy than the experimental results shown in Fig. 11. In contrast, it can be evaluated that the classification device 10 has a smaller decrease in classification accuracy than the comparison model. It can be considered that the classification device 10 achieved higher classification accuracy because it was able to represent data more accurately using a flexible adaptive hidden Markov model.

[0123] Furthermore, since the classification device 10 has adaptive hidden Markov models for the number of classes, and the adaptive hidden Markov models are probability models in which the state transition probabilities change over time, it is unlikely to be affected by imbalances in the amount of data, and it is thought that there is a possibility that the device will be able to sufficiently learn even for classes with a small amount of data.

[0124] As described above, the model calculation unit 100 calculates the likelihood of transitioning to each state based on the input time series data, based on a state transition model in which the class to be identified is associated with a state of a state transition model in which the state transition probability changes over time, and in which data is generated based on a probability distribution associated with the destination state, and calculates, for each class to be identified, the posterior probability that data will be classified into that class based on the input time series data, based on the calculated likelihood. The class selection unit 170 selects one of the classes based on the calculated posterior probability.

[0125] According to the identification device 10, by using a state transition model in which the state transition probability changes over time, it is possible to more accurately represent complex time-series data, which is expected to result in better identification accuracy.

[0126] Furthermore, the component likelihood calculation unit 130 calculates the likelihood that input data will occur at the time of calculation target for each component that is a normal distribution that constitutes a Gaussian mixture distribution that indicates the probability distribution of data that will occur. The GMM likelihood calculation unit 140 calculates the likelihood that the input data occurs at the time of calculation for each Gaussian mixture distribution based on the likelihood for each component. The HMM likelihood calculation unit 150 calculates the likelihood that the input time series data up to the calculation target time will occur for each state of the adaptive hidden Markov model based on the likelihood for each Gaussian mixture distribution calculated by the GMM likelihood calculation unit 140. The posterior probability calculation unit 160 calculates the posterior probability that data will be classified into a class based on the input time series data, based on the likelihood for each state of the adaptive hidden Markov model calculated by the HMM likelihood calculation unit 150.

[0127] According to the classification device 10, calculations for classifying input time series data can be performed hierarchically, and the calculations at each stage can be relatively simple. Furthermore, according to the classification device 10, classification using an adaptive hidden Markov model can be implemented in a neural network relatively easily.

[0128] Furthermore, the Gaussian mixture distribution is expressed by a power formula of the product of the weighting coefficient and the transformed input data. According to the classification device 10, the weighting coefficients can be treated as learning parameters that are not subject to statistical constraints. This makes it easier for the classification device 10 to learn, and is expected to improve classification accuracy.

[0129] Furthermore, the state transition probability is expressed by an equation in which likelihood information at a time earlier than the time of calculation and likelihood information at the time of calculation are each weighted using a function whose value changes over time. According to the classification device 10, the degree of learning between past information and current information can be adjusted over time during training of the classification device 10. This is expected to enable the classification device 10 to perform classification with higher accuracy even for relatively complex time-series data.

[0130] FIG. 13 is a diagram illustrating an example of the configuration of a computer according to the embodiment. In the configuration shown in FIG. 13, a computer 700 includes a CPU 710, a main memory device 720, an auxiliary memory device 730, an interface 740, and a non-volatile recording medium 750.

[0131] The above-described identification device 10 may be implemented in a computer 700. In this case, the operations of the above-described processing units are stored in the form of a program in the auxiliary storage device 730. The CPU 710 reads the program from the auxiliary storage device 730, loads it into the main storage device 720, and executes the above-described processing in accordance with the program.

[0132] Furthermore, the CPU 710 allocates a storage area in the main storage device 720 for the identification device 10 to perform processing in accordance with a program. Communication between the identification device 10 and other devices is performed by an interface 740 having a communication function and performing communication under the control of the CPU 710. Interaction between the identification device 10 and a user is performed by the interface 740 having an input device and an output device, and presenting information to the user via the output device under the control of the CPU 710 and accepting user operations via the input device.

[0133] One or more of the above-described programs may be recorded on nonvolatile recording medium 750. In this case, interface 740 may read the programs from nonvolatile recording medium 750. CPU 710 may then directly execute the programs read by interface 740, or may temporarily store the programs in main storage device 720 or auxiliary storage device 730 and then execute them.

[0134] It is also possible to record a program for realizing all or part of the functions of the identification device 10 on a computer-readable recording medium, and have a computer system load and execute the program to perform processing of each part. Note that the term "computer system" here includes the OS (Operating System) and hardware such as peripheral devices. Furthermore, "computer-readable recording media" refers to portable media such as flexible disks, optical magnetic disks, ROMs (Read Only Memory), and CD-ROMs (Compact Disc Read Only Memory), as well as storage devices such as hard disks built into computer systems. The program may be one that realizes part of the aforementioned functions, or may be one that can realize the aforementioned functions in combination with a program already stored in the computer system.

[0135] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and design changes and the like are also included within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0136] 10 Identification device 100 Model Calculation Section 110 Nonlinear transformation section 120 Input section 130 Component likelihood calculation part 140 GMM likelihood calculation part 150 HMM likelihood calculation unit 160 Posterior Probability Calculation Unit 170 Class Selection Section 210 Recursive Calculation Unit 211 Gate Function Calculation Unit 212 Subtraction section 213, 214 Multiplication section 215 Addition section 216 Power calculation unit

Claims

1. a model calculation unit that calculates, for each state based on input time series data, a likelihood of transition to that state, based on the state transition model in which a class to be identified is associated with a state of the state transition model in which state transition probabilities change over time, and in which data is generated based on a probability distribution associated with the state of a transition destination, and calculates, for each class to be identified based on the likelihood, a posterior probability that data will be classified into that class based on the input time series data; a class selection unit that selects one of the classes based on the calculated posterior probability; An identification device comprising:

2. The model calculation unit a component likelihood calculation unit that calculates the likelihood of input data occurring at a calculation target time for each normal distribution that constitutes the Gaussian mixture distribution that indicates the probability distribution; a Gaussian mixture distribution likelihood calculation unit that calculates, for each Gaussian mixture distribution, a likelihood that input data will occur at a calculation target time based on the likelihood for each normal distribution; an adaptive hidden Markov model likelihood calculation unit that calculates, for each state of the state transition model, a likelihood that input time series data up to a calculation target time will occur based on the likelihood for each Gaussian mixture distribution; a posterior probability calculation unit that calculates the posterior probability based on the likelihood for each state of the state transition model; Equipped with The identification device according to claim 1 .

3. The Gaussian mixture distribution is expressed by a power formula of the product of a weighting coefficient and the transformed input data: The identification device according to claim 2 .

4. The state transition probability is expressed by an equation in which likelihood information at a time earlier than the time of calculation and likelihood information at the time of calculation are weighted using a function whose value changes over time. The identification device according to any one of claims 1 to 3.

5. The computer a state transition model in which a class to be identified corresponds to a state of the state transition model in which the state transition probability changes over time, and data is generated based on a probability distribution associated with the state of the transition destination, the likelihood of transitioning to that state is calculated for each state based on the input time series data, and a posterior probability is calculated for each class to be identified based on the likelihood that data will be classified into that class based on the input time series data, Select one of the classes based on the calculated posterior probability. The identification method includes:

6. On the computer, a state transition model in which a class to be identified corresponds to a state of the state transition model in which state transition probabilities change over time, and data is generated based on a probability distribution associated with the state of a transition destination, calculating a likelihood of transitioning to that state for each state based on input time series data, and calculating a posterior probability for each class to be identified based on the likelihood that data will be classified into that class based on the input time series data; Selecting one of the classes based on the calculated posterior probability; A program that executes the following.

Images