Model learning device, model learning method, and model learning program
The model learning device and method enhance data analysis by connecting DMD and complex NMF through geometric and arithmetic constraints, creating a model with higher expressive power and time evolution traits, addressing the limitations of existing models.
Patent Information
- Application Number
- PCT/JP2023/018983
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-22
- Publication Date
- 2024-11-28
AI Technical Summary
Current models lack higher expressive power than Dynamic Mode Decomposition (DMD) and do not incorporate time evolution characteristics similar to DMD, while complex Non-negative Matrix Factorization (NMF) does not share these time evolution traits.
A model learning device and method that acquires observation data matrices for DMD and complex NMF, imposing geometric and arithmetic constraints to estimate parameters, connecting DMD and complex NMF through equality constraints, thereby constructing a model with higher expressive power and time evolution characteristics.
Enables data analysis with a model that surpasses DMD in expressive power and captures time evolution characteristics, allowing for effective parameter estimation.
Smart Images

Figure JP2023018983_28112024_PF_FP_ABST
Abstract
Description
Model learning device, model learning method, and model learning program
[0001] The present invention relates to a model learning device, a model learning method, and a model learning program.
[0002] Dynamic mode decomposition (DMD) has been applied to the analysis of various nonlinear dynamical systems. DMD is a data-driven method for analyzing nonlinear dynamical systems based on the Koopman operator theory (see, for example, Non-Patent Document 1). By applying DMD, complex time series with nonlinearity are decomposed into multiple "modes." Each mode is composed of spatially correlated structures with the same time-evolving behavior (growing or decaying while oscillating at a certain frequency). This makes it possible to extract spatially organized structures from data. Therefore, DMD is used in various research fields, such as fluid mechanics, epidemiology, power systems, and sports analysis.
[0003] In a context different from DMD, a method called nonnegative matrix factorization (NMF) has been successful in audio source separation, document classification, and the like, and is widely used (see, for example, Non-Patent Document 2).
[0004] Furthermore, there is a complex NMF, which is an advanced method of NMF (see, for example, Non-Patent Document 3).
[0005] Yoshinobu Kawahara, Operator-Theoretic Data Analysis of Nonlinear Dynamical Systems and Dynamic Mode Decomposition: Focusing on Fundamentals and Formulations Using Positive Definite Kernels, Journal of the Institute of Mathematical Sciences, Vol. 2167, pp. 40-50, July 2020. Hiroshi Sawada, Fundamentals of Nonnegative Matrix Factorization (nmf) and Its Applications to Data / Signal Analysis, The Journal of the Institute of Electronics, Information and Communication Engineers, Vol. 95, No. 9, pp. 829-833, September 2012. Hirokazu Kameoka, Nobutaka Ono, Kunio Kashino, and Shigeki Sagayama. Complex nmf: A new sparse representation for acoustic signals. In 2009 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 3437-3440. IEEE, 2009. Naoya Takeishi. Data-Driven Analysis of Dynamical Systems Based on the Koopman Operator: A Machine Learning Perspective. PhD thesis, Universityof Tokyo, 2018.Naoya Takeishi, Yoshinobu Kawahara, and Takehisa Yairi. Sparse nonnegative dynamic mode decomposition. In 2017 IEEE International Conference on Image Processing, pp. 2682-2686. IEEE, 2017.
[0006] However, there is no model that has a higher expressive power than the DMD and that gives the complex NMF characteristics related to the way it evolves over time, like the DMD.
[0007] The present invention has been made in light of the above-mentioned circumstances, and an object of the present invention is to provide a technique that enables data analysis by constructing a model that has higher expressive power than DMD and that gives complex NMF characteristics related to the way time evolves like DMD, and estimating the parameters of the model.
[0008] In order to solve the above problem, a model learning device according to one embodiment of the present invention includes an acquisition unit that acquires a matrix representing observation data used in dynamic mode decomposition (DMD), a parameter estimation unit that estimates parameters of a model constructed by imposing an equality constraint on one of the non-negative matrices that are parameters of complex non-negative matrix factorization (complex NMF) and an arithmetic constraint on a phase parameter of the complex NMF, thereby connecting the DMD and the complex NMF, by solving a constrained optimization problem using the observation data as input, and an output control unit that outputs the estimated parameters.
[0009] According to one aspect of the present invention, a model is constructed that has higher expressive power than DMD and that gives complex NMF characteristics related to the way time evolves like DMD, and data can be analyzed by estimating its parameters.
[0010] Fig. 1 is a block diagram showing an example of the hardware configuration of a model learning device according to an embodiment. Fig. 2 is a block diagram showing the software configuration of the model learning device according to an embodiment in relation to the hardware configuration shown in Fig. 1. Fig. 3 is a flowchart showing an example of how the model learning device estimates and outputs model parameters. Fig. 4 is a flowchart explaining in more detail an example of the processing of step ST102.
[0011] Next, an embodiment of the present invention will be described below with reference to the drawings. (Koopman Operator Theory) First, the Koopman operator theory will be described.
[0012] Dynamic mode decomposition is a method for approximately calculating the spectrum of the Koopman operator corresponding to a dynamical system. Let X be the state space. Let the state at time t be x.t ∈X. A (nonlinear) dynamical system x on X t = f(x t-1 ) and observables g:X→C belonging to the function space G are given, the Koopman operator K:G→G is defined as the following infinite-dimensional linear operator: (Kg)(x) = g(f(x)) (1) where the symbol C represents a complex space. Furthermore, from the following equation, the Koopman operator can be considered to describe the time evolution of the observables. g(x t ) = g(f(x t-1 )) = (Kg)(x t-1 ) = (K 2 g) (x t-2 ) = ... = (K t g) (x 0 )
[0013] Suppose there is a subspace H ⊂ G of G that is closed with respect to K (i.e., K g ∈H if g∈H). Given n observables {g 1 , ..., g n}, and then let g = (g 1 , ..., g n ) T : X → C n In this case, the time evolution of g (one step) can be written as (kg)(x) = g(f(x)). Here, k is a finite-dimensional linear operator obtained by restricting the Koopman operator K to the space H. The eigenfunctions and eigenvalues of the linear operator k are defined as φ i : X → C n and λ i ∈C (kφ i (x) = λ i φ i (x) holds). The observable (set of observables) g is the eigenfunction {φ i} n i=1 A certain coefficient {ν i} n i=1 can be expressed as follows using
[0014] g(x) = Σ ni=1 φ i (x)v i (2)
[0015] Also, (kg)(x) = Σ n i=1 λ i φ i (x)v i Therefore, the following mode decomposition system of the observables at time t can be obtained.
[0016] g(x t ) = Σ n i=1 λ t i w i , w i =φ i (x 0 )v i (3) where {w i} n i=1 are called Koopman modes, and the absolute value of each eigenvalue |λ i | and argument ∠λ i That's the mode lol i The damping rate and vibration frequency are determined.
[0017] (DMD and Nonnegative DMD (NDMD)) Next, the DMD and nonnegative DMD will be described.
[0018] Dynamic mode decomposition (DMD) is a method for (approximately) calculating the mode decomposition system of Equation (3) based on observed data. Assume that the observed data is given as a matrix of size I×(T+1) below.
[0019] Y = {g(x 0 ), ..., g(x T )∈C I×(T+1) Hereafter, for simplicity, the (i, j)th element of the matrix Y is expressed as y ij Here, the following two symbols are used:
[0020]
[0021]
[0022] In addition, the following two matrices Y are used as submatrices of the matrix Y: 0 and Y 1 Define
[0023] Y 0 = {g(x 0 ), ..., g(x T-1 )∈C I×T Y 1 = {g(x 1 ), ..., g(x T )∈C I×T
[0024] In general, in a method called strict DMD, which is used as a DMD, for example, as disclosed in Non-Patent Document 1, the matrix Y 0 By applying (truncated) singular value decomposition to i} n i=1 and the eigenvalues {λ i} n i=1 get.
[0025] It is also known that the DMD can be formulated as an optimization problem as shown below, as in Non-Patent Document 1, for example. By reformulating it as an optimization problem, it is possible to construct a method in which constraints and regularization terms are added to the values taken by the modes and eigenvalues, as in Non-Patent Document 5, for example. The modes and eigenvalues are expressed by the following parameters in polar coordinate format.
[0026]
[0027] where j is the imaginary unit (j 2 =-1),
[0028]
[0029] is the radius (radius vector) of the mode and eigenvalue, φ ir , ψ r ∈R is a parameter representing the angle (phase). In the following description, these symbols will be collectively written as follows:
[0030]
[0031] where:
[0032]
[0033] and R represents the number of effective modes.
[0034] Without loss of generality, the radius parameter
[0035]
[0036] may be non-negative, because coordinates expressed by negative radii can be expressed by rotating coordinates expressed by positive radii by 180 degrees (i.e., adding π to the angle).
[0037] Using the parameters in polar coordinate format above, the right hand side of equation (3) can be transformed as follows:
[0038]
[0039] For example, as described in Non-Patent Document 5, DMD can be considered as a method for estimating parameters by solving the following optimization problem:
[0040]
[0041] Here, the objective function L is defined by the following equation.
[0042]
[0043] For example, Non-Patent Document 5 proposes a method called non-negative DMD by introducing a regularization term and a non-negative constraint into this optimization problem as follows: For example, in the non-negative DMD of Non-Patent Document 5, φ is fixed as a vector with all elements being zero, and L using a scalar-valued hyperparameter is used as follows: 1 I'm considering norm regularization.
[0044]
[0045] In this embodiment, the formulation shown below is a broader setting that includes the above equation.
[0046]
[0047] where:
[0048]
[0049] Is, L q is a regularization term using a norm,
[0050]
[0051] is a hyperparameter. When 0<q<2, this regularization term has the effect of encouraging the parameter ρ to be sparse. The above ρ≧0 means that all
[0052]
[0053] In ρ ir ≧0.
[0054] (NMF and Complex NMF (CNMF)) Next, NMF and complex NMF will be described. As described above, the complex NMF disclosed in Non-Patent Document 3 is an advanced method of the NMF disclosed in Non-Patent Document 2.
[0055] While a conventional NMF handles data represented as a matrix with real-valued elements, a complex NMF is designed to handle audio signals represented as a complex spectrum, and therefore uses a matrix with complex-valued elements as input.
[0056] Let Y∈C be a matrix whose elements take on complex values. I×J , and the element is y ij (Note that we have intentionally written it in the same way as the matrix representing the DMD observation data.) Complex NMF has two non-negative matrices as parameters:
[0057]
[0058] and phase
[0059]
[0060] Using these three, the matrix Y is approximated in the following factorization form:
[0061]
[0062] The parameters are estimated by minimizing the following loss function:
[0063]
[0064] The loss function above is defined as follows:
[0065]
[0066] (Method in the embodiment) Next, the method in the embodiment will be described.
[0067] The method in this embodiment is positioned between DMD and (complex) NMF, and is constructed by theoretically analyzing the relationship between the DMD and NMF subtypes, specifically the aforementioned DMD, non-negative DMD (NDMD) (see, for example, Non-Patent Document 5), NMF, and complex NMF.
[0068] The theoretical analysis described below connects the above-mentioned method through two constraints as follows:
[0069] The decomposition form of DMD corresponds to a special case of the factorization form of complex NMF, which is an advanced method of NMF, and the two become equivalent by adding two constraints to complex NMF: (Constraint 1) the rows of the factor matrix are a geometric progression, and (Constraint 2) the phase is an equidifference progression (details will be given later).
[0070] Under the condition that the above two constraints are satisfied, the equivalence of the objective function and optimization problem between the nonnegative DMD and the complex NMF, in which the nonnegative constraint that the elements of the DMD's factor matrix are nonnegative (i.e., ρ≧0 as explained in the above equation (8)) is added to the DMD, is also proven. Note that it is known that the NMF can be considered as a special case of the complex NMF, as in Non-Patent Document 3, for example. Therefore, the above theoretical analysis clarifies the connection between the DMD, nonnegative DMD (NDMD), NMF, and complex NMF.
[0071] Based on this theoretical analysis, this embodiment constructs a model positioned between DMD and complex NMF and a method for estimating its parameters. The constructed methods are a method in which only Constraint 1 or Constraint 2 is added to the complex NMF as a strict constraint, and a method in which the complex NMF is estimated so that either or both of Constraints 1 and 2 are loosely (although not strictly) satisfied. In this way, a model with higher expressive power than the DMD and which gives the complex NMF characteristics (like the DMD) regarding the way it evolves over time is constructed, and its parameters are estimated.
[0072] (Theoretical Analysis of DMD and Complex NMF) Next, a theoretical analysis of DMD and complex NMF will be described.
[0073] The method constructed in this embodiment is based on the theory of equivalence between DMD and complex NMF. The theoretical analysis of this embodiment will be described below. In the analysis, first, two constraints, (i) geometric progression constraint (GP constraint) and (ii) arithmetic difference constraint (AP constraint), are defined. (i) Geometric progression constraint: A geometric progression (GP) constraint on the factor matrix B is set to each column {b tr} t∈T The constraint that is a geometric progression. That is, the common ratio ξ for each r r ∈R such that b t,r and b t-1,r is defined as a constraint that satisfies the following equation.
[0074]
[0075] (ii) Arithmetic Constraint: An arithmetic progression (AP) constraint on the phase θ is applied to the phase parameter
[0076]
[0077] The constraint that the difference sequence is equal. That is, the common difference τ r ∈R such that θ i,t,r and θ i,t-1,r is defined as a constraint that satisfies the following equation.
[0078]
[0079] In the following, the parameters representing the common ratio and tolerance are
[0080]
[0081] and
[0082]
[0083] Using the above two constraints, the following lemma can be used to show that DMD and complex NMF are connected. Lemma 1 (Equivalence of the Decomposition Forms): When the factorization form of the complex NMF (Equation (9)) is introduced with the equality constraint (Equation (11)) and the arithmetic difference constraint (Equation (12)), it coincides with the approximation form of the DMD (Equation (5)).
[0084] Proof: From the constraints (11) and (12), b t,r = b 0,r (ξ r ) t and θ i,t,r = tτ r +θ i,0,r Therefore, the complex NMF factorization form of equation (9) can be expanded as follows:
[0085]
[0086] Therefore, by making the following substitutions, the equation (5) which is an approximation of the DMD can be derived.
[0087]
[0088] Lemma 1 shows that the approximation of the DMD is a special case of the factorization form of the complex NMF. Furthermore, if we look carefully at Lemma 1, we can see that the radial parameter ρ of the DMD is a part of the parameters of the complex NMF.
[0089]
[0090] From now on, the radial parameter expressed by these two terms will be called
[0091]
[0092] and is defined as follows:
[0093]
[0094] where diag(b 0 ) is a diagonal matrix of size R × R, with the rth diagonal element being b 0,r Similarly, the common ratio ξ r and tolerance τ r It can also be seen that corresponds to the parameters related to the singular values of the DMD.
[0095] For example, in Non-Patent Document 4, the similarity between the approximation form / factorization form of NMF and the complex NMF is pointed out (without mathematical proof). However, in this embodiment, it has been mathematically proven that they match exactly by adding the geometric constraint and the arithmetic constraint.
[0096] Furthermore, from Lemma 1, it can be immediately shown that the loss function of DMD and the loss function of complex NMF are the same.
[0097] Proposition 1 (Equivalence of the (Data-Fitting) Losses): Assume that the equality constraint (Equation (11)) and the arithmetic constraint (Equation (12)) are satisfied. In this case, the loss function D of the complex NMF and the loss function L of the DMD are the same.
[0098] Proof: Some of the phase parameters of the complex NMF are listed below.
[0099]
[0100] From Lemma 1, the following equation holds:
[0101]
[0102] The following correspondence can also be shown for the regularization term:
[0103] Lemma 2 (Equivalence of the Regularizations): If the factor matrix B satisfies the equality constraint (Equation (11)) and the leading column b 0 Let the values of be all non-zero as follows:
[0104]
[0105] In this case, the following equation holds:
[0106]
[0107] where the symbol
[0108]
[0109] is matrix element-wise division (Hadamard division), b q 0 is b 0 is raised to the qth power for each element, that is, expressed by the following formula.
[0110]
[0111] Proof: It is clear that the following formula holds:
[0112]
[0113] Thus, the following results for complex NMF and non-negative DMD are obtained:
[0114] Proposition 2 (Equivalence of Optimization Problem): In the optimization problem (Equation (10)) corresponding to the complex NMF, the factor matrix B is 0 is a string whose elements are all 1
[0115]
[0116] By adding the constraints that the equation (11) is satisfied and the factor parameters satisfy the arithmetic constraint (12), the optimization problem corresponds to the optimization problem (8) corresponding to the non-negative DMD.
[0117] Proof: By Proposition 1 and Lemma 2, the objective function of complex NMF can be expanded as follows:
[0118]
[0119] The obtained equation is the objective function of the non-negative DMD, and can be regarded as the parameter ρ replaced with the matrix A. Therefore, it can be seen that the non-negativity constraint on the matrix A and the non-negativity constraint on the parameter ρ represent the same constraint condition.
[0120] The above theoretical analysis has revealed that DMD, non-negative DMD (NDMD), NMF, and complex NMF are connected by the geometric constraint and the arithmetic constraint.
[0121] (Formulation of this embodiment) Next, the formulation of this embodiment will be described.
[0122] Considering the above theoretical analysis results, it is possible to construct a model positioned between the complex NMF and the (non-negative) DMD. For example, if we consider a formulation that introduces a geometric constraint (without adding an arithmetic constraint) to the complex NMF as shown below, we can obtain a model that allows a high degree of freedom for the value of the phase θ as in the complex NMF, while allowing a b tr A model can be constructed that expresses the development of an environment by expanding or contracting over time.
[0123]
[0124] Similarly, we can consider a complex NMF with only arithmetic constraints:
[0125]
[0126] For parameter estimation, any method for solving constrained optimization problems can be used, such as the (stochastic) projected gradient method, Newton's method, the L-BFGS method, Adam, etc. Furthermore, it is also possible to construct and solve the problem using an algorithm that uses auxiliary variables or latent variables, such as the auxiliary function method or the EM (Expectation-Minimization) algorithm.
[0127] Furthermore, in the above two formulations, the arithmetic constraint or the geometric constraint is strictly satisfied. However, by considering a formulation in which the constraint condition is converted into a regularization term and added to the objective function, it is possible to construct a method that does not require strict satisfaction. For example, suppose that the geometric constraint in equation (14) is expressed by the following regularization term (hereinafter referred to as the geometric regularization term or GP regularization):
[0128]
[0129] where β is a hyperparameter. This regularization term is designed to have a value of zero when the geometric constraint is satisfied, and to take a larger value as the elements of matrix B deviate from the numerical values determined by the geometric constraint. By formulating this as the following optimization problem in addition to the objective function of complex NMF, a method can be constructed that "loosely" requires that the geometric constraint be satisfied.
[0130]
[0131] Similarly, when replacing the arithmetic constraint with a regularization term in equation (15), if the regularization term is designed as follows,
[0132]
[0133] The following methods can be constructed:
[0134]
[0135] (Estimation Algorithm) In this embodiment, an example will be shown in which an algorithm for solving equation (16) is constructed using the auxiliary function method (also called the majorization-minimization (MM) method). In the following, for simplicity, the objective function in the optimization problem of equation (16) is written as F, and the parameters are collectively written as Θ = {A, B, θ, ξ}. That is, the following equation holds:
[0136]
[0137] In the auxiliary function method (MM method), an estimation algorithm is derived using a function called an auxiliary function or majoring function. + The following can be used as:
[0138]
[0139] where:
[0140]
[0141] are auxiliary variables,
[0142]
[0143] is an (arbitrary) positive variable that satisfies the following relationship:
[0144]
[0145] This auxiliary function F + is an upper bound of the objective function F and satisfies the following equation:
[0146]
[0147] The equality condition is given by the following formula:
[0148]
[0149] In the auxiliary function method, the auxiliary function is used to indirectly minimize the objective function F as follows, and the minimum value of the objective function is found:
[0150]
[0151] The minimization of the auxiliary variables in step 2 can use equation (20) as the update equation. The minimization of the parameters in step 1 can be performed using the following update equation.
[0152] Update A and B: Auxiliary function F + A ir The partial derivative with respect to is given by the following equation:
[0153]
[0154] Here, the above formula can be transformed into the following formula, which is utilized.
[0155]
[0156] In addition, s * ijr is, s ijr represents the complex conjugate of , and Re(·) is a function that returns the real part of a complex value.
[0157] Therefore, the necessary condition for the minimum
[0158]
[0159] From this, the update formula for A is obtained as follows:
[0160]
[0161] Similarly, the following update equation can be derived for B:
[0162]
[0163] Updating θ(C): By grouping the terms that do not depend on the phase parameter θ into constant terms (denoted as Const. in the equation) and using the addition theorem of trigonometric functions, the auxiliary function F + can be summarized as follows:
[0164]
[0165] However, the symbol {v itr} is defined as follows: In the following equation, Im(•) is a function that returns the imaginary part of a complex value.
[0166]
[0167] Therefore, the auxiliary function F + is cos(θ itr -v itr ) = 1, i.e., cos(θ itr ) = cos(v itr ) and sin(θ itr ) = sin(v itr ) is minimized. Therefore, the matrix C(c itr =exp(jθ itr )) the following update equation is derived:
[0168]
[0169] Note that, by substituting equation (23) into equations (21) and (22), it can be confirmed that by setting the initial values of A and B to non-negative values, the updated values also become non-negative, and the non-negativity of A and B is preserved.
[0170] Update of ξ: Auxiliary function F + ξ r The partial derivative with respect to is given by the following equation:
[0171]
[0172] Therefore, the necessary condition for a local minimum is
[0173]
[0174] This leads to the following update formula:
[0175]
[0176] Updating U: The parameter U is an (arbitrary) positive variable and is not necessarily updated. However, for example, in the complex NMF disclosed in Non-Patent Document 3, it is recommended to update it according to the following formula:
[0177]
[0178] In complex NMF, when this update formula for U is introduced, the update formulas for matrices A and B become (almost) the same as the update formula for normal NMF. As a result, the update formula for parameters Θ = {A, B, θ, ξ} (and parameter U) is obtained. This allows us to derive an algorithm for estimating parameters using auxiliary functions in this embodiment.
[0179] (Configuration) Next, the hardware and software configurations of a model learning device for executing the above-described method will be described. FIG. 1 is a block diagram showing an example of the hardware configuration of a model learning device 1 according to an embodiment. The model learning device 1 is a computer that analyzes input data and generates and outputs output data. For example, the model learning device 1 is installed in any location set by the user who manages the model learning device 1.
[0180] As shown in FIG. 1 , the model learning device 1 includes a control unit 10, a program storage unit 20, a data storage unit 30, a communication interface 40, and an input / output interface 50. The control unit 10, the program storage unit 20, the data storage unit 30, the communication interface 40, and the input / output interface 50 are communicatively connected to one another via a bus. Furthermore, the communication interface 40 may be communicatively connected to an external device via a network. Furthermore, the input / output interface 50 is communicatively connected to an input device 2, an output device 3, and a user device 4.
[0181] The control unit 10 controls the model learning device 1. The control unit 10 includes a hardware processor such as a central processing unit (CPU). For example, the control unit 10 may be an integrated circuit capable of executing various programs.
[0182] The program storage unit 20 may use, as a storage medium, a combination of nonvolatile memory that can be written to and read from at any time, such as an erasable programmable read-only memory (EPROM), a hard disk drive (HDD), or a solid state drive (SSD), and a nonvolatile memory such as a read-only memory (ROM). The program storage unit 20 stores programs necessary for executing various processes. That is, the control unit 10 can realize various controls and operations by reading and executing the programs stored in the program storage unit 20.
[0183] The data storage unit 30 is a storage that uses a combination of nonvolatile memory such as a HDD or memory card that can be written to and read from at any time, and volatile memory such as a RAM (Random Access Memory), as a storage medium. The data storage unit 30 is used to store data acquired and generated in the process of the control unit 10 executing programs and performing various processes.
[0184] The communication interface 40 includes one or more wired or wireless communication modules. For example, the communication interface 40 includes a communication module that establishes a wired or wireless connection with an external device via a network. The communication interface 40 may also include a wireless communication module that establishes a wireless connection with an external device such as a Wi-Fi access point or base station. Furthermore, the communication interface 40 may also include a wireless communication module that establishes a wireless connection with an external device using short-range wireless technology. In other words, the communication interface 40 may be any general communication interface that can communicate with an external device and send and receive various information under the control of the control unit 10.
[0185] The input / output interface 50 is connected to the input device 2, the output device 3, the user device 4, etc. The input / output interface 50 is an interface that enables transmission and reception of information between the input device 2, the output device 3, and the user device 4. The input / output interface 50 may be integrated with the communication interface 40. For example, the model learning device 1 and at least one of the input device 2, the output device 3, and the user device 4 may be wirelessly connected using short-range wireless technology or the like, and information may be transmitted and received using the short-range wireless technology.
[0186] The input device 2 may include, for example, a keyboard or a pointing device for the user to input various information to the model learning device 1. The input device 2 may also include a reader for reading data to be stored in the program storage unit 20 or the data storage unit 30 from a memory medium such as a USB memory, or a disk device for reading such data from a disk medium.
[0187] The output device 3 includes a display or the like that displays the results of calculations performed by the control unit 10, the virtual space, etc. The output device 3 also includes a printer or the like that prints the information displayed on the display.
[0188] The user device 4 is a device that includes at least one of a camera that captures the user's movements, a sensor device with an acceleration sensor, a light-emitting device, a light-emitting and color-changing device, etc. That is, the user device 4 may include both a camera and a light-emitting device, or may include only a sensor device with an acceleration sensor. That is, the user device 4 is a device used to collect observation data. The observation data may be data observed in any field, such as fluid dynamics, epidemiology, power systems, or sports. The user device 4 may be any device that can acquire observation data and output the observation data to the model learning device 1.
[0189] Fig. 2 is a block diagram showing the software configuration of the model learning device 1 according to the embodiment in relation to the hardware configuration shown in Fig. 1. The control unit 10 includes an input data acquisition unit 101, a parameter estimation unit 102, and an output control unit 103.
[0190] The input data acquisition unit 101 acquires a matrix Y representing observation data. The matrix Y representing observation data may be the observation data used in the DMD described above. The input data acquisition unit 101 stores the acquired matrix Y representing the observation data in the input data storage unit 301.
[0191] The parameter estimation unit 102 includes an initialization control unit 1021, a parameter update unit 1022, a calculation iteration count update unit 1023, a change amount calculation unit 1024, and a termination condition determination unit 1025. The parameter estimation unit 102 is an estimation unit that estimates a parameter Θ of a model based on a matrix Y that represents observation data stored in the input data storage unit 301.
[0192] The initialization control unit 1021 initializes a parameter Θ. The initialization control unit 1021 also similarly initializes a variable δ that indicates the amount of change in the objective function. Furthermore, the initialization control unit 1021 sets various conditions, such as a threshold ε (a predetermined threshold) for the termination condition, the number of iterations, and the maximum number of iterations. The initialization control unit 1021 may store the various set conditions in the data storage unit 30.
[0193] The parameter update unit 1022 updates the parameters A, B, C (or θ), and ξ according to the above-described formulas (21), (22), (23), and (24). Furthermore, the parameter update unit 1022 updates the parameters S, A, and U according to formulas (20) and (25). Here, A means A bar shown in formula (19) etc. The parameter update unit 1022 stores the updated parameters in the parameter storage unit 302.
[0194] The calculation repetition count update unit 1023 updates the number of repetitions. For example, the calculation repetition count update unit 1023 updates the number of repetitions by adding 1 to the number of repetitions stored in the data storage unit 30.
[0195] The change amount calculation unit 1024 calculates the change amount |F old -F new | is calculated and the calculated value is stored in a variable δ. Here, the objective function value calculated using the parameters before updating is F old , the objective function value calculated using the updated parameters is F new It states:
[0196] The termination condition determination unit 1025 determines whether the termination condition is satisfied. For example, the termination condition determination unit 1025 determines whether the number of iterations exceeds the maximum number of iterations. Alternatively, the termination condition determination unit 1025 determines whether a variable δ indicating the amount of change in the objective function value is smaller than a threshold ε of the termination condition.
[0197] The data storage unit 30 includes an input data storage unit 301 and a parameter storage unit 302. The input data storage unit 301 is used to store the observation data and the like acquired by the input data acquisition unit 101. The parameter storage unit 302 is used to store various conditions used by the parameter estimation unit 102, estimated parameters, and the like.
[0198] 3 is a flowchart showing an example of how model learning device 1 estimates and outputs model parameters. The control unit 10 of model learning device 1 reads and executes a program stored in program storage unit 20, thereby realizing the operation of this flowchart.
[0199] This operation starts when an instruction is given by the user who manages the model learning device 1 or when observation data is received from the user device 4.
[0200] In step ST101, the input data acquisition unit 101 acquires a matrix Y representing observation data. The matrix Y representing the observation data may be the observation data used in the DMD described above. The input data acquisition unit 101 stores the acquired matrix Y representing the observation data in the input data storage unit 301.
[0201] For example, there are cases where the data input from the user device 4 is not in the form of matrix Y representing the observation data to be used by the DMD. In such cases, the input data acquisition unit 101 converts the received data into matrix Y representing the observation data to be used by the DMD. Alternatively, the data acquired by the user device 4 may be converted into matrix Y representing the observation data and output to the model learning device 1.
[0202] In step ST102, the parameter estimation unit 102 estimates the parameters Θ of the model based on the matrix Y representing the observation data stored in the input data storage unit 301.
[0203] 4 is a flowchart illustrating an example of the process of step ST102 in more detail. In step ST201, the initialization control unit 1021 initializes a parameter Θ. The initialization control unit 1021 also initializes a variable δ indicating the amount of change in the objective function. Furthermore, the initialization control unit 1021 sets a threshold ε (a predetermined threshold), the number of iterations, and the maximum number of iterations as termination conditions.
[0204] In step ST202, the parameter update unit 1022 updates the parameters A, B, C (or θ), and ξ, which are the first parameter group, in accordance with the above-described equations (21), (22), (23), and (24). The parameter update unit 1022 stores the updated parameters in the parameter storage unit 302.
[0205] In step ST203, the parameter update unit 1022 updates the parameters S, A, and U, which are the second parameter group, in accordance with equations (20) and (25). The parameter update unit 1022 stores the updated parameters in the parameter storage unit 302.
[0206] In step ST204, the calculation repetition count updating unit 1023 updates the number of repetitions. For example, the calculation repetition count updating unit 1023 updates the number of repetitions by adding 1 to the number of repetitions.
[0207] In step ST205, the change amount calculation unit 1024 calculates the change amount |F old -F newand stores the calculated value in a variable δ. Note that the variable δ may be stored in the parameter storage unit 302.
[0208] In step ST206, the termination condition determination unit 1025 determines whether the termination condition is satisfied. For example, the termination condition determination unit 1025 determines whether the number of iterations exceeds the maximum number of iterations. Alternatively, the termination condition determination unit 1025 determines whether a variable δ indicating the amount of change in the objective function value is smaller than a threshold ε of the termination condition. For example, if it is determined that the number of iterations exceeds the maximum number of iterations, or if it is determined that the variable δ is smaller than the threshold ε, the termination condition determination unit 1025 ends the processing in the parameter estimation unit 102 and proceeds to step ST103 shown in FIG. 3.
[0209] On the other hand, if it is determined that the number of repetitions does not exceed the maximum number of repetitions, or if it is determined that the variable δ is greater than the threshold ε, the termination condition determination unit 1025 sets the value of the variable δ to 0. Then, the processing returns to step ST202.
[0210] 3 , in step ST103, the output control unit 103 acquires the parameter Θ stored in the parameter storage unit 302 and controls the display of the output device 3 to display the acquired parameter Θ. The output control unit 103, which also operates as a transmission control unit, may output the parameter Θ to the user device 4 via the input / output interface 50. Alternatively, the output control unit 103, which also operates as a transmission control unit, may transmit the parameter Θ to an external device via the communication interface 40.
[0211] [Effects] According to the embodiment, a model is constructed that has higher expressive power than DMD and that gives complex NMF characteristics related to the way time evolves like DMD, and data can be analyzed by estimating its parameters.
[0212] [Other Embodiments] In the above embodiment, the auxiliary function method is used to estimate the parameter θ that minimizes equation (18). However, any method for solving a constrained optimization problem, such as the (stochastic) projected gradient method, Newton's method, the L-BFGS method, Adam's method, etc. may also be used.
[0213] Furthermore, in the model learning device 1 in the above embodiment, the operation of each component can be constructed as a program, and installed and executed on a computer used as the user device 4 or a computer used as an external device.
[0214] Furthermore, the techniques described in the above embodiments can be stored as a program (software means) that can be executed by a computer on a storage medium such as a magnetic disk (e.g., a floppy disk, a hard disk, etc.), an optical disk (e.g., a CD-ROM, a DVD, an MO, etc.), or a semiconductor memory (e.g., a ROM, a RAM, a flash memory, etc.), and can also be distributed by transmitting it via a communication medium. The program stored on the medium also includes a configuration program that configures the software means (including not only execution programs but also tables and data structures) that the computer executes. The computer that realizes this device loads the program stored on the storage medium and, in some cases, configures the software means using the configuration program, and executes the above-mentioned processing by controlling the operation of this software means. Note that the term "storage medium" as used herein is not limited to storage media for distribution, but also includes storage media such as magnetic disks and semiconductor memories installed inside the computer or in devices connected via a network.
[0215] In short, this invention is not limited to the above-described embodiments, and various modifications can be made in the implementation stage without departing from the spirit of the invention. Furthermore, the embodiments may be implemented in combination as appropriate as possible, and in such cases, the combined effects can be obtained. Furthermore, the above-described embodiments include inventions at various stages, and various inventions can be extracted by appropriately combining the disclosed multiple constituent elements.
[0216] DESCRIPTION OF SYMBOLS 1...Model learning device 10...Control unit 101...Input data acquisition unit 102...Parameter estimation unit 1021...Initialization control unit 1022...Parameter update unit 1023...Calculation repetition count update unit 1024...Change amount calculation unit 1025...End condition determination unit 103...Output control unit 20...Program storage unit 30...Data storage unit 301...Input data storage unit 302...Parameter storage unit 40...Communication interface 50...Input / output interface 2...Input device 3...Output device 4...User device
Claims
1. A model learning device comprising: an acquisition unit that acquires a matrix representing observed data used in dynamic mode decomposition (DMD); a parameter estimation unit that estimates parameters of a model constructed by connecting the DMD and the complex NMF by imposing an equality constraint on one of the non-negative matrices that are parameters of complex non-negative matrix decomposition (complex NMF) and an arithmetic constraint on the phase parameter of the complex NMF, by solving a constrained optimization problem using the observed data as input; and an output control unit that outputs the estimated parameters.
2. The model learning device of claim 1, wherein the connection between the DMD and the complex NMF is based on the equivalence of objective functions and optimization problems of the non-negative DMD and the complex NMF with the non-negative constraint that elements of the factor matrix of the DMD are non-negative values.
3. The model learning device according to claim 1, wherein the geometric constraint is a constraint that each column of the non-negative matrix is a geometric progression.
4. The model learning device according to claim 1, wherein the arithmetic constraint is a constraint that the phase parameters follow an arithmetic difference progression.
5. The model learning device according to claim 1, wherein the model is constructed by introducing the equality constraint or the arithmetic difference constraint into the complex nonnegative matrix decomposition.
6. The model learning device according to claim 5, wherein the model is a model formulated as an optimization problem in which the equal ratio constraint or the arithmetic difference constraint is replaced in the form of a regularization term and added to an objective function of the complex NMF.
7. A model learning method executed by a processor of a model learning device, comprising: acquiring a matrix representing observed data used in dynamic mode decomposition (DMD); estimating parameters of a model constructed by connecting the DMD and the complex NMF by imposing an equality constraint on one of the non-negative matrices that are parameters of complex non-negative matrix decomposition (complex NMF) and an arithmetic constraint on a phase parameter of the complex NMF, by solving a constrained optimization problem using the observed data as input; and outputting the estimated parameters.
8. A model learning program having instructions to be executed by a processor of a model learning device, the instructions comprising: acquiring a matrix representing observed data used in dynamic mode decomposition (DMD); estimating parameters of a model constructed by connecting the DMD and the complex NMF by imposing an equality constraint on one of the non-negative matrices that are parameters of complex non-negative matrix decomposition (complex NMF) and an arithmetic constraint on a phase parameter of the complex NMF, by solving a constrained optimization problem using the observed data as input; and outputting the estimated parameters.
