Data analysis device, method and program

Abstract

A data analysis device according to one embodiment of the present invention has: an input unit for accepting input of index data that is a set of indices of data being analyzed, a set of elements of the data being analyzed, and magnitude comparison data that indicates a magnitude relationship between the elements of the data being analyzed; a first estimation unit for estimating, on the basis of the set of elements inputted through the input unit, a parameter for minimizing the upper bound of error between a loss function used in estimating the factors of the data being analyzed and an approximation of the loss function; and a second estimation unit for estimating the factors of the data being analyzed, on the basis of the index data and magnitude comparison data inputted through the input unit and the parameter estimated by the first estimation unit.

Description

[Technical field]

[0001] An embodiment of the present invention relates to a data analysis device, a method, and a program. [Background technology]

[0002] In recent data analysis, a method called matrix factorization (MF) or tensor factorization (TF) has been widely used (see, for example, Non-Patent Document 1). A tensor is a high-dimensional array with three or more axes.

[0003] When the data to be analyzed can be expressed as a matrix or tensor, such as documents, purchase history, user attribute information, or movie rating history, it is possible to automatically extract patterns in the data or complement missing values ​​in the data by factoring matrix-expressed data into the product of multiple matrices using MF, or factoring tensor-expressed data into the product of multiple matrices using TF. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Hisashi Kashima. Machine Learning for Relational Data. Journal of the Robotics Society of Japan, Vol.33, No.2, pp.72-76,2015. Summary of the Invention [Problem to be solved by the invention]

[0005] However, in recent data analysis, it is sometimes necessary to analyze data in which the index of an element of a matrix or tensor does not correspond to the value of that element, which is created as a result of the data collection method or processing for privacy protection. For example, in the case of extracting a pattern of a user's movie rating tendency, in a normal MF, the movie rating history is expressed as a matrix.

[0006]

number

[0007] Assuming that the matrix is ​​available, the pattern extraction is performed by applying MF to the input. For example, the row index i corresponds to a user, the column index j corresponds to a movie, and the matrix element value x ij is a matrix showing the rating value of movie j by user i.

[0008] However, such a matrix representation of movie rating history is not necessarily usable. For example, the fact that user i rated or watched movie j may be recorded and used, but the rating value is not recorded or made public in a form linked to movie j from the viewpoint of privacy protection, and the distribution of ratings of movie j by user i, for example, there are z1 movies rated "1", z2 movies rated "2", etc., may be available. In this case, the rating of movie j by user i cannot be observed, so there is no correspondence between the index (i, j) of the matrix element and the value of that element, i.e., x ij Since it is not known what the value of is, it is expressed as data that cannot be expressed in a matrix.

[0009] Similarly, when dealing with user attribute information where row index i corresponds to each attribute item such as weight and annual income, and column index j corresponds to a user, from the perspective of privacy protection, the rating value is not recorded or published in a form linked to user j, and may be available in the form of a distribution of the value of item i, for example, there are z1 users who weigh 60 kg, there are z2 users who weigh 61 kg, etc. In this case, as in the case of movie rating history, the information of user j regarding item i cannot be observed, and therefore it is not possible to establish a correspondence between the index of a matrix element and the value of that element, resulting in data that cannot be expressed in a matrix.

[0010] The present invention has been made in light of the above-mentioned circumstances, and has as its object to provide a data analysis device, method, and program that are capable of appropriately analyzing data to be analyzed regardless of whether there is a correspondence between the index and elements of the data to be analyzed. [Means for solving the problem]

[0011] A data analysis apparatus according to one embodiment of the present invention comprises an input unit that accepts input of index data, which is a set of indexes of data to be analyzed, a set of elements of the data to be analyzed, and size comparison data indicating the size relationship of the elements of the data to be analyzed, a first estimation unit that estimates parameters that minimize an upper bound of an error between a loss function used to estimate a factor of the data to be analyzed and an approximation of the loss function, based on the set of elements input by the input unit, and a second estimation unit that estimates a factor of the data to be analyzed based on the index data, the size comparison data input by the input unit, and the parameters estimated by the first estimation unit.

[0012] A data analysis method according to one embodiment of the present invention is a method performed by a data analysis device, and includes accepting input of index data, which is a set of indexes of data to be analyzed, a set of elements of the data to be analyzed, and size comparison data indicating the size relationship of the elements of the data to be analyzed; estimating parameters that minimize an upper bound of the error between a loss function used to estimate a factor of the data to be analyzed and an approximation of the loss function, based on the input set of elements; and estimating the factor of the data to be analyzed based on the input index data, the size comparison data, and the estimated parameters. Effect of the Invention

[0013] According to the present invention, data to be analyzed can be appropriately analyzed regardless of the presence or absence of correspondence between the index and element of the data to be analyzed. [Brief description of the drawings]

[0014] [Figure 1] FIG. 1 is a diagram showing an application example of a data analysis device according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a flow chart showing an example of an outline of the operation of the data analysis device according to one embodiment of the present invention. [Diagram 3] FIG. 3 is a flowchart showing an example of an operation of the hyper-parameter estimation unit. [Figure 4] FIG. 4 is a flowchart showing an example of the operation of the parameter estimation unit. [Diagram 5] FIG. 5 is a block diagram showing an example of a hardware configuration of a data analysis device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In this embodiment, we will explain a method that can automatically extract patterns in data and estimate values ​​of matrix elements, such as MF or TF, even if the data is given in a format in which the index of a matrix or tensor element cannot correspond to the value of that element.

[0016] The conventional MF (TF) cannot analyze data in which the indices of matrix (tensor) elements do not correspond to the values ​​of those elements, as described above. This is because MF assumes that the input data is expressed in a matrix, as follows:

[0017] MF is the input matrix

[0018]

number

[0019] The factor matrices, which correspond to the factor matrices with i rows and r columns and the factor matrices with r rows and j columns, respectively, are

[0020]

number

[0021] and,

[0022]

number

[0023] Patterns are extracted by estimating as the result of factorization.

[0024] The estimation of the factor matrix is ​​done by the matrix

[0025]

number

[0026] and the product of the factor matrices

[0027]

number

[0028] This is done so that the values ​​of and are similar.

[0029]

number

[0030] The (i, j) element of

[0031]

number

[0032] The loss function used as a measure of similarity above, that is, the smaller the value, the more similar the two matrices are, is the following L2 divergence:

[0033]

number

[0034] Or the Bregman Divergence (BD) below:

[0035]

number

[0036] (For example, see "Tamara G Kolda and Brett W Bader. Tensor decompositions and applications. SIAM review, Vol.51, No.3, pp. 455-500, 2009." (Reference 1), "Russ R Salakhutdinov and Andriy Mnih. Probabilistic matrix factorization. In Advances in neural information processing systems, pp. 1257-1264, 2008." (Reference 2), and "Ajit P Singh and Geoffrey J Gordon. Relational learning via collective matrix factorization. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 650-658, 2008." (Reference 4)).

[0037]

number

[0038] In addition,

[0039]

number

[0040] is a convex function,

[0041]

number

[0042] is its derivative. The function

[0043]

number

[0044] By changing , BD can express various functions. For example,

[0045]

number

[0046] corresponds to the squared error when

[0047]

number

[0048] corresponds to logistic loss when

[0049]

number

[0050] When , it corresponds to I-divergence (also called generalized KL (Kullback-Leibler) divergence),

[0051]

number

[0052] When , it corresponds to the Itakura-Saito pseudo-distance. Determining the loss function to use is equivalent to making an assumption on the probability distribution that generates the matrix elements. Specifically, using the squared error, I-divergence, or Itakura-Saito pseudo-distance corresponds one-to-one to the assumption that the matrix elements are generated according to normal distribution, poisson distribution, or exponential distribution, respectively.

[0053] The factor matrix is ​​estimated by minimizing the loss function represented above. Various methods such as Stochastic Gradient Descent (SGD) or Newton's method can be used to optimize this loss function. Reference 4 above describes the derivation of an algorithm that repeatedly updates the factor matrix alternately based on Newton's method.

[0054] In addition, when applying TF, we can consider low-rank approximations of tensors rather than matrices. For example, a 3rd order tensor

[0055]

number

[0056] is approximated by CP decomposition (Canonical Polyadic decomposition), that is, by decomposing it into factors with a basis number R, the element x of the tensor ijk and its factor matrix

[0057]

number

[0058] and

[0059]

number

[0060] Approximation by

[0061]

number

[0062] It is sufficient to estimate that and are similar values.

[0063] The above MF (TF) cannot be applied to data that lacks matrix (tensor) expression, such as the data to be analyzed this time. This is because the data is not expressed as a matrix (tensor), so it is not possible to calculate the value of the loss function, nor is it possible to calculate the above Newton update formula. Therefore, in this embodiment, the following new method is presented.

[0064] Methods for dealing with a related problem, different from the method of this embodiment, include the method described in "Z. Shen, L. Du, X. Shen, and Y. Shen. Interval-valued matrix factorization with applications. In ICDM, pp. 1037-1042. IEEE, 2010." (Reference 3) and the method described in "Yuto Yamaguchi and Kohei Hayashi. Tensor decomposition with missing indices. In International Joint Conference on Artificial Intelligence, pp.3217-3223, 2017." (Reference 6). In the method described in Reference 3, the values ​​of matrix elements themselves cannot be observed, but information on a certain interval of values, for example, x ij When information is available, such as the value of is between 1 and 3, a method has been proposed to use that information to perform MF.

[0065] The problem considered in this embodiment is different from the problem where information on a certain interval of values ​​is given, and therefore this method cannot be applied. Reference 6 above proposes a method that can perform tensor decomposition even when there is a missing tensor index. In this method, a pair of tensor index and element (i, j, k, x ijk Even if some of the indices in , for example j, are missing, the tensor decomposition is applied by estimating the value of the missing index.

[0066] The method considered in this embodiment differs from the method described in Reference 6 above in that it is a method that can analyze data without the need to estimate missing values ​​and establish correspondence between indices and matrix elements.

[0067] In addition, in deriving the objective function used for parameter estimation in this embodiment, an approximation of the same kind as that used to derive the objective function in the regression method described in "Liyuan Xu, Gang Niu, Junya Honda, and Masashi Sugiyama. Uncoupled regression from pairwise comparison data. In Proceedings of the 33rd International Conference on Neural Information Processing Systems, pp. 3992-4002, 2019." (Reference 5) was used. However, the problem addressed by the method described in Reference 5 here is a regression problem of estimating a function f that determines the relationship y=f(x) between an input x and an output y, which is a different problem from the matrix decomposition or tensor decomposition considered in this embodiment.

[0068] In this embodiment, a method is described that can perform matrix factorization or tensor factorization even for data in which the index of a matrix or tensor element does not correspond to the value of the element, and the representation as a matrix or tensor is insufficient. Hereinafter, this method is referred to as uncoupled matrix factorization (UMF) and uncoupled tensor factorization (UTF).

[0069] (formulation) Next, we will explain the formulation, which is the problem setting of the method described in this embodiment. Below, we will explain the UMF formulation, but the UTF formulation is almost the same if you replace matrices with tensors based on the UMF formulation. The size of the matrix is ​​n I ×n J In the case of the movie rating history data mentioned above, n I is the number of users, n J represents the number of movies. The set of matrix indices is

[0070]

number

[0071] This is hereafter referred to as index data. U represents the number of indexes in this data. In the case of the movie rating history data mentioned above,

[0072]

number

[0073] is user i m The movie m It corresponds to a collection of data that records whether a user has rated or viewed a program.

[0074] Furthermore, the set of distributions for the elements of each row of the matrix, i.e., the set of probability distributions, is

[0075]

number

[0076] The distribution of elements in each row above is the Probability Density Function (PDF) of the matrix elements conditioned (or fixed) on row i and marginalized with respect to column j.

[0077] In the case of the movie rating history data mentioned above, f X|I=i Let (x) be the probability density function of movie rating value x by a certain user i, which can be constructed from prior knowledge, estimated from aggregate data that shows that there are z1 movies rated "1" and z2 movies rated "2", or the set of values ​​of the elements of each row of the matrix.

[0078]

number

[0079] n V represents the number of elements in this data. m represents the row index, and x m is the i m It represents the value of an element in a certain column of a row, but the column is unknown. As will be described later, when using aggregated data or a set of element values ​​for each row, the method can be applied without explicitly estimating this probability density function.

[0080] In addition to the above, the data showing the magnitude relationship of the matrix elements is

[0081]

number

[0082] This will be referred to as size comparison data from now on. mis the row index,

[0083]

number

[0084] and,

[0085]

number

[0086] and are both column indexes and element values.

[0087]

number

[0088] The value of

[0089]

number

[0090] It indicates that the value of element is greater than the value of element .

[0091]

number

[0092] It is not something that has been observed. R represents the number of pairs of matrix elements that are compared. Such data can be obtained, for example, by having a user evaluate a movie j and judge whether it is better than another movie j', and recording the result of this judgment. This type of data is often easier for users to answer than giving a score such as 1 or 2, and it is also easier to collect the data.

[0093] Also, when the elements of a matrix are not movie ratings but rather represent the number of visits to a store, answering that you visit a certain store more than another store rather than entering the actual number of visits will hide the fact that you have a favorite store that you visit very frequently, and will allow you to answer with less information about yourself. UMF is an index data

[0094]

number

[0095] , a set of probability distributions

[0096]

number

[0097] , and size comparison data

[0098]

number

[0099] The input is three data, and the factor matrix

[0100]

number

[0101] Hereafter, this factor matrix is ​​collectively referred to as

[0102]

number

[0103] The estimate of each element using the factor matrix is

[0104]

number

[0105] It is written as follows.

[0106] (Loss function) Next, we will explain how to derive the loss function for the above estimation. In defining the loss function, the random variable whose realization value corresponds to the row index is written as I, and the random variable whose realization value corresponds to the column index is written as J. Furthermore, the random variables whose realizations correspond to the values ​​of the matrix elements are written as X. The probability distribution that these random variables follow is P I,J,X It is written as follows. Furthermore, the conditional probability distribution when I=i is used and the probability density function of this probability distribution are P J,X|I=i , f J,X|I=i It is assumed that it is written as follows. The conditional distribution is further marginalized with respect to J or X, and the distribution is P X|I=i , P J|I=i The probability distribution P X|I=i The probability density function of f X|I=i , the cumulative density function is F X|I=i By definition,

[0107]

number

[0108] It is.

[0109] The loss function used to estimate the parameter θ using the random variables above is defined as the following equation (1) using the Bregman divergence above.

[0110]

number

[0111] However, in equation (1),

[0112]

number

[0113] is the probability distribution P I,J,X This is an expected value operation by

[0114]

number

[0115] is defined by the following equation (2).

[0116]

number

[0117] However, in equation (2),

[0118]

number

[0119] are probability distributions P J,X|I=i , P J|I=i The difficulty with this loss function is caused by the last term on the right-hand side of equation (2),

[0120]

number

[0121] This is because, although this term is defined by the joint distribution of the column index J and the element X, in the problem setting considered in this embodiment, the index and the element are not associated and cannot be observed simultaneously, so that it cannot be calculated even if, for example, sample approximation is performed. Therefore, hereinafter, we consider approximately evaluating this term.

[0122] Here, we newly define a set of random variables (J+ , J - ) is introduced. This set is the set of elements (i, J + ) is the value of (i, J - ) and is defined as follows:

[0123]

number

[0124] Here, (J, X) and (J′, X′) both have probability P J,X|I=i From this definition, the above-mentioned size comparison data

[0125]

number

[0126] We will later use the fact that we can consider J to be the realization of this random variable. + and J. - The probability density functions that follow are

[0127]

number

[0128] and

[0129]

number

[0130] It is written as "J" + and J. - The expected value operation regarding the outcome of

[0131]

number

[0132] Using this random variable, it can be shown that the following equations (3) and (4) hold true.

[0133]

number

[0134] However, F X|I=i is the cumulative density function for the element values ​​with row i held fixed

[0135]

number

[0136] It is.

[0137] We will now explain the proof of formula (3). + By the definition of f J+|I=i can be expanded as follows:

[0138]

number

[0139] where Z is a normalization constant defined as follows:

[0140]

number

[0141] By integration by parts, we can derive Z = 1 / 2. Using this result, we can derive equation (3) as follows:

[0142]

number

[0143] The derivation of equation (4) is similar. This completes the proof.

[0144] Using equations (3) and (4), we can see that the conditional marginal distribution of the random variable X is a uniform distribution on [0, 1] (i.e., F X|I=i(x) = x), the last term on the right hand side of equation (2) becomes

[0145]

number

[0146] Considering this fact, a certain hyperparameter

[0147]

number

[0148] It is considered promising to use the approximation shown in the following equation (5).

[0149]

number

[0150] In equation (5), the value on the right side approximates the value on the left side. In the case of the uniform distribution on [0, 1] above, (w i1 , w i2 ) = (1 / 2, 0), this approximation is accurate. This is because the conditional marginal distribution of the random variable X is a uniform distribution on [a, b] (F X|I=i This can also be generalized to (x)=(xa) / (ba) for all x ∈ [a, b], where (w i1 , w i2 ) = (b / 2, a / 2). When considering a more general distribution other than a uniform distribution, the generalization error

[0151]

number

[0152] The hyperparameter (w i1 , w i2 ), which is discussed below.

[0153] Adding together equations (3) and (4), we obtain the following equation:

[0154]

number

[0155] Using this formula, a constant λ i Using this, the above equation (5) can be transformed into the following equation (6).

[0156]

number

[0157] λ i There is a degree of freedom in setting λ i =0 or λ i =(w i1 +w i2 ) / 2, etc. However, the basic rule is that i The value of is preferably equal to or greater than zero. i ≧0. Using the above formula (6), the generalization loss of formula (1)

[0158]

number

[0159] Approximation of

[0160]

number

[0161] is obtained as follows:

[0162]

number

[0163] however,

[0164]

number

[0165] The above C i is a constant that does not depend on the parameter θ. Therefore, the random variables I, J, J + and J. - By replacing the expected value of with the sample average, we obtain the empirical risk as shown in the following equation (7).

[0166]

number

[0167] is obtained.

[0168]

number

[0169] Here, C is a constant that does not depend on the parameter θ. Since these quantities, except for the constant C, can be calculated from data, they can be used as the objective function for estimating the parameter θ. Therefore, from now on, the empirical loss

[0170]

number

[0171] The objective function, expressed as the following equation (8), is obtained by removing the constant C from

[0172]

number

[0173] This section describes how to optimize the above.

[0174]

number

[0175] Ω in Eq. (8) A ,Ω B are the factor matrices, respectively.

[0176]

number

[0177] It represents the regularization term of the L1 norm or the L2 norm, and any value can be used. In the next section, we will consider the empirical loss

[0178]

number

[0179] We present a parameter estimation method, i.e., an estimation algorithm, that minimizes

[0180] (Estimation algorithm) Any optimization method can be used to estimate the parameter θ, such as stochastic gradient descent, (quasi-)Newton method, or alternating least squares method (ALS). In the following, we consider the case where the objective function is differentiable, but even if the objective function has points that are not differentiable, such as when the L1 norm is used as a regularization term, it can be optimized in the same way by using the subgradient method. Objective function

[0181]

number

[0182] The partial derivative of is given by the following formula:

[0183]

number

[0184] However, the above

[0185]

number

[0186] is the index data

[0187]

number

[0188] From i m = i, and the above

[0189]

number

[0190] is the index data

[0191]

number

[0192] From, j m = j, and the above

[0193]

number

[0194] is a comparison data

[0195]

number

[0196] From i m All data for which =i is extracted,

[0197]

number

[0198] is a comparison data

[0199]

number

[0200] from,

[0201]

number

[0202] All the data that is

[0203]

number

[0204] is a comparison data

[0205]

number

[0206] from,

[0207]

number

[0208] This is the value obtained by extracting all the data for which . Using these values, we can derive an update equation for estimation, for example, as follows:

[0209] The above gradient descent method will now be explained. In this gradient descent method, the parameter θ is updated in the steepest descent direction. The update formula can be specifically written as the following formulas (9) and (10).

[0210]

number

[0211] Here, ηi and γj are learning rates. By repeatedly updating the parameter θ according to these equations (9) and (10), it is possible to estimate the parameter θ. Note that, unlike the stochastic gradient descent method, the parameters for all rows i and columns j are not updated every time, but the parameters for row i selected stochastically are

[0212]

number

[0213] and the parameters for column j

[0214]

number

[0215] It is also possible to update only the above.

[0216] The above Newton method will now be explained. In this Newton method, the update formula is derived using up to two partial derivatives. In general, the length of time required to calculate the Hessian matrix obtained by two partial derivatives is an issue with the Newton method. If i ≠ i', then

[0217]

number

[0218] If j ≠ j′, then

[0219]

number

[0220] By utilizing this and the fact that it has a similar property as a regularization term, that is, the property that the second partial differential is zero if i ≠ i' or j ≠ j', and using an alternating update approach in which one factor matrix is ​​fixed and the other factor matrix is ​​updated, the parameter θ of the factor matrix to be updated can be updated independently for each row or column, and the Hessian matrix of the parameter θ to be updated can be a matrix of size R × R. This allows the parameter θ to be updated efficiently.

[0221]

number

[0222] is updated, the Hessian matrix of size R×R

[0223]

number

[0224] , and

[0225]

number

[0226] The Hessian matrix when is updated

[0227]

number

[0228] Each element of is obtained as follows:

[0229]

number

[0230] Using this, the parameter θ can be updated as shown in the following equations (11) and (12).

[0231]

number

[0232] where ηi and γj are learning rates. By repeatedly updating the parameter θ according to these equations (11) and (12), a local optimum solution of the objective function can be reached. Also, like the stochastic gradient descent method, when updating the parameter θ, the parameter θ for all rows i or columns j is not updated every time, but the parameter θ for the row i and column j selected stochastically is updated every time.

[0233]

number

[0234] It is also possible to update only the

[0235] As mentioned above, BD is a convex function that

[0236]

number

[0237] Since various loss functions can be expressed by changing θ, by using the update formula of the gradient descent method or Newton method described above, it is possible to derive the update formula of the parameter θ when using any loss function that can be expressed in BD, such as the squared error, logistic loss, I-divergence, or Itakura-Saito pseudo distance described above.

[0238] (Hyperparameter estimation) Finally, the hyperparameter {w i1 , w i2 This hyperparameter is estimated by the function Err i It can be determined by minimizing

[0239]

number

[0240] This function is

[0241]

number

[0242] of

[0243]

number

[0244] To be precise, any factor matrix

[0245]

number

[0246] and for component (i, j)

[0247]

number

[0248] Under the condition that

[0249]

number

[0250] Therefore, the above function Err i The approximation error can be reduced by minimizing the function Err i involves integral calculations, but by assumption, the distribution f X|I=i Since (x) is known, it can be calculated as a discrete approximation. Specifically, for example, we can calculate the whole possible value of x as

[0251]

number

[0252] For example, the 0.01 quantile of x is

[0253]

number

[0254] Then, the 0.99th quantile is

[0255]

number

[0256] As,

[0257]

number

[0258] By minimizing the right-hand side of the following equation (14), it can be estimated using any optimization method.

[0259]

number

[0260] Since this equation (14) is defined as the sum of absolute values, it is possible to use a method that can handle the existence of non-differentiable points in the objective function, such as the subgradient method, or linear programming.

[0261] When using the subgradient method, the function

[0262]

number

[0263] W i =(w i1 , wi2)

[0264]

number

[0265] Using any vector g belonging to

[0266]

number

[0267] We can then iteratively update the parameter θ according to the following, where γ′ is the learning rate.

[0268] In the above, the probability distribution f X|I=i We have shown how to estimate hyperparameters when (x) is known, but the set of element values ​​for each row of the matrix

[0269]

number

[0270] It is also possible to estimate hyperparameters using

[0271]

number

[0272] From i m The value where i is the

[0273]

number

[0274] If this is written, the function Err i We can consider the approximation shown in the following equation (16).

[0275]

number

[0276] however,

[0277]

number

[0278] is the cumulative density function F X|I=i Empirical approximation of

[0279]

number

[0280] It is.

[0281]

number

[0282] teeth

[0283]

number

[0284] The number of data items is also expressed by the function Err i It is possible to approximate the above. For optimizing the above formula (16), any optimization method can be used, such as the subgradient method, as in the case of optimizing the above formula (14).

[0285] This embodiment makes it possible to perform matrix decomposition or tensor decomposition even for data in which the index of a matrix or tensor element does not correspond to the value of that element and the representation as a matrix or tensor is insufficient. This makes it possible to extract latent patterns from data in which the representation as a matrix or tensor is insufficient and to estimate values ​​of matrix elements.

[0286] (Explanation of operation) The configuration of the data analysis device 1 according to this embodiment and the operation of each unit will be described below. FIG. 1 is a diagram showing an application example of a data analysis device according to an embodiment of the present invention. 1, a data analysis device 1 according to an embodiment of the present invention includes an input data processing unit 10, a hyperparameter estimation unit 20, a parameter estimation unit 30, a parameter processing unit 40, a recording unit 50, and an input / output unit 60. The input / output unit 60 is capable of inputting and outputting data to and from an external device 2. The recording unit 50 includes an input data recording unit 51 , a hyper-parameter recording unit 52 , and a parameter recording unit 53 .

[0287] Next, there will be described an outline of the operation of the data analysis device 1. Fig. 2 is a flowchart showing an example of the outline of the operation of the data analysis device according to one embodiment of the present invention. First, the input data processing unit 10 of the data analysis device 1 receives index data

[0288]

number

[0289] , a set of probability distributions

[0290]

number

[0291] , and size comparison data

[0292]

number

[0293] is input from the external device 2 via the input / output unit 60 (step S1).

[0294] Next, the hyper-parameter estimation unit 20 estimates the hyper-parameters

[0295]

number

[0296] is estimated (step S2). Next, the parameter estimation unit 30 estimates the parameter θ (step S3). Next, the parameter processing unit 40 outputs the estimated parameter θ to the external device 2 via the input / output unit 60 (step S4).

[0297] The specific operations of S1 to S4 above will be described below. First, a specific example of S1 will be described. The input data processing unit 10 receives index data

[0298]

number

[0299] , a set of probability distributions

[0300]

number

[0301] , and size comparison data

[0302]

number

[0303] is input and stored in the input data recording section 51 of the recording section 50.

[0304] Next, a specific example of S2 will be described. The hyperparameter estimation unit 20 estimates a set of probability distributions stored in the input data recording unit 51.

[0305]

number

[0306] The hyper-parameter w is estimated by inputting the hyper-parameter w and minimizing the objective function of the above equation (14) by the following methods (a) to (d). The hyper-parameter estimation unit 20 stores the estimated hyper-parameter w in the hyper-parameter recording unit 52 of the recording unit 50. FIG. 3 is a flowchart showing an example of the operation of the hyper-parameter estimation unit. FIG. 3 shows the hyper-parameter w when the subgradient method is used. i An example of updating i for i=1, …, n I By performing this step once each time, the hyper-parameter w to be estimated can be obtained.

[0307] (a) The hyperparameter estimation unit 20 estimates the parameter w i (step S21). As a variable used as a termination condition for the process in step S2, the hyper-parameter estimation unit 20 initializes the hyper-parameter w i Similarly, a variable δ indicating the maximum change range of the update amount of is initialized, and the threshold ε of the above-mentioned termination condition and the maximum number of iterations of the calculation, here the maximum value of the number of iterations of the calculation related to the update of the variable δ, are set (step S22).

[0308] (b) The hyper-parameter estimation unit 20 updates the parameter w according to the above formula (15). At this time, the hyper-parameter estimation unit 20 updates the hyper-parameter w before the update. i and the updated hyperparameter w i The maximum absolute difference between

[0309]

number

[0310] is recorded in the variable δ (step S23). However, here, the pre-update hyper-parameter w i The elements of

[0311]

number

[0312] The updated hyperparameter w i The elements of

[0313]

number

[0314] It is described as follows.

[0315] (c) In response to step S23, the hyper-parameter estimation unit 20 updates the count value of the number of times the calculation is repeated (step S24). (d) If the count value of the number of calculation iterations exceeds the predetermined maximum number of iterations or the variable δ indicating the maximum change width of the update amount is smaller than the predetermined threshold ε (Yes in step S25), the hyper-parameter estimation unit 20 ends the process of step S2. Otherwise (No in step S25), the hyper-parameter estimation unit 20 sets the variable δ to δ←0 and returns to step S22.

[0316] Next, a specific example of S3 will be described. The parameter estimation unit 30 estimates the index data stored in the input data recording unit 51.

[0317]

number

[0318] , Large and small comparison data

[0319]

number

[0320] , and the hyper-parameter w stored in the hyper-parameter recording unit 52, and estimates the parameter θ by minimizing the objective function of the above equation (8) by the following methods (a) to (e). The parameter estimation unit 30 stores the estimated parameter θ in the parameter recording unit 53 of the recording unit 50. Fig. 4 is a flowchart showing an example of the operation of the parameter estimation unit. Fig. 4 shows an example of updating the parameter θ when the Newton method is used.

[0321] (a) The parameter estimation unit 30 initializes the parameter θ (step S31). As a variable used as a termination condition for the process in step S3, the parameter estimation unit 30 similarly initializes a variable δ indicating the maximum change range of the update amount of the parameter θ, and sets a threshold ε and a maximum number of iterations of the calculation for the termination condition, here the maximum number of iterations of the calculation for updating the variable δ (step S32).

[0322] (b) The parameter estimation unit 30 estimates the factor matrix

[0323]

number

[0324] is updated according to the above formula (11). At this time, the parameter estimation unit 30 updates the factor matrix before and after the update

[0325]

number

[0326] The maximum absolute value of the difference between

[0327]

number

[0328] If is greater than the current variable δ, the parameter estimation unit 30 sets this variable δ as

[0329]

number

[0330] (Step S33). However, the factor matrix before the update is

[0331]

number

[0332] The elements of

[0333]

number

[0334] After updating, the above elements are

[0335]

number

[0336] It is described as follows.

[0337] (c) The parameter estimation unit 30 calculates the factor matrix

[0338]

number

[0339] is updated according to the above formula (12). At this time, the parameter estimation unit 30 calculates the factor matrix

[0340]

number

[0341] The maximum absolute value of the difference between

[0342]

number

[0343] If is greater than the current variable δ, set this variable δ to

[0344]

number

[0345] (Step S34). However, the factor matrix before the update is

[0346]

number

[0347] The elements of

[0348]

number

[0349] After updating, the above elements are

[0350]

number

[0351] It is described as follows.

[0352] (d) In response to step S34, the parameter estimation unit 30 updates the count value of the number of times the calculation is repeated (step S35).

[0353] (e) If the count value of the number of calculation repetitions exceeds the predetermined maximum number of repetitions, or if the variable δ indicating the maximum change range of the update amount is smaller than the predetermined threshold ε (Yes in step S36), the parameter estimation unit 30 ends the process related to step S3; otherwise (No in step S36), the parameter estimation unit 30 sets the variable δ to δ←0 and returns to step S32.

[0354] Next, a specific example of S4 will be described. The parameter processing unit 40 outputs the parameter θ stored in the parameter recording unit 53 to the external device 2 via the input / output unit 60. In the above embodiment, an algorithm based on Newton's method is used to estimate the parameter θ that minimizes the objective function of the above equation (8), but any other method, such as the gradient descent method, may be used. Also, in the above embodiment, a set of probability distributions is used to estimate the hyperparameters.

[0355]

number

[0356] Instead of using the set of element values ​​for each row of the matrix,

[0357]

number

[0358] Alternatively, aggregated data may be used. In this case, the hyperparameters can be estimated by minimizing (optimizing) the above formula (16) instead of the objective function of the above formula (14). Any method can be used for this optimization. In addition, while the above shows an example of applying UMF, which performs matrix decomposition on data that lacks a matrix representation, it is also possible to apply UTF, which performs tensor decomposition on data that lacks a tensor representation, in a similar framework.

[0359] FIG. 5 is a block diagram showing an example of a hardware configuration of a data analysis device according to an embodiment of the present invention. 5, the data analysis device 1 according to the embodiment is configured, for example, by a server computer or a personal computer, and has a hardware processor 111A such as a CPU (Central Processing Unit). A program memory 111B, a data memory 112, an input / output interface 113, and a communication interface 114 are connected to the hardware processor 111A via a bus 115.

[0360] The communication interface 114 includes, for example, one or more wireless communication interface units, and enables transmission and reception of information to and from a communication network NW. As the wireless interface, for example, an interface that adopts a low-power wireless data communication standard such as a wireless LAN (Local Area Network) is used.

[0361] An input device 200 and an output device 300 used by a user or the like, which are attached to the data analysis apparatus 1, are connected to the input / output interface 113. The input device 200 and the output device 300 correspond to the external device 2 shown in FIG. The input / output interface 113 takes in operation data input by a user or the like through an input device 200 such as a keyboard, a touch panel, a touchpad, a mouse, or the like, and outputs output data to an output device 300 including a display device using liquid crystal or organic EL (Electro Luminescence), etc., for display. The input device 200 and the output device 300 may be devices built into the data analyzing apparatus 1, or may be input devices and output devices of other information terminals capable of communicating with the data analyzing apparatus 1 via the network NW.

[0362] The program memory 111B is a non-transient tangible storage medium that is a combination of a non-volatile memory that can be written to and read from at any time, such as a hard disk drive (HDD) or a solid state drive (SSD), and a non-volatile memory such as a read only memory (ROM), and stores programs necessary for executing various control processes, etc. according to one embodiment.

[0363] The data memory 112 is a tangible storage medium that is a combination of, for example, the above-mentioned nonvolatile memory and a volatile memory such as a RAM (Random Access Memory), and is used to store various data acquired and created during various processing steps.

[0364] A data analysis device 1 according to one embodiment of the present invention can be configured as a data processing device having an input data processing unit 10, a hyperparameter estimation unit 20, a parameter estimation unit 30, a parameter processing unit 40, and an input / output unit 60 shown in FIG. 1 as processing function units based on software.

[0365] Each information storage unit and recording unit 50 used as a working memory by each unit of the data analysis device 1 can be configured by using the data memory 112 shown in Fig. 5. However, these configured storage areas are not essential components within the data analysis device 1, and may be areas provided in a storage device such as an external storage medium such as a Universal Serial Bus (USB) memory, or a database server located in the cloud.

[0366] The processing function units in each of the input data processing unit 10, the hyper parameter estimation unit 20, the parameter estimation unit 30, the parameter processing unit 40, and the input / output unit 60 can be realized by reading and executing a program stored in the program memory 111B by the hardware processor 111A. Note that some or all of these processing function units may be realized in various other forms, including integrated circuits such as an application specific integrated circuit (ASIC) or a field-programmable gate array (FPGA).

[0367] The methods described in each embodiment may be stored as a program (software means) that can be executed by a computer on a recording medium such as a magnetic disk (floppy disk, hard disk, etc.), an optical disc (CD-ROM, DVD, MO, etc.), or a semiconductor memory (ROM, RAM, flash memory, etc.), and may be distributed by transmitting the program via a communication medium. The programs stored on the medium include a setting program that configures the software means (including not only execution programs but also tables and data structures) that the computer executes. The computer that realizes the present device reads the program recorded on the recording medium, and in some cases, configures the software means by the setting program, and executes the above-mentioned processing by controlling the operation of the software means. The recording medium in this specification is not limited to a recording medium for distribution, but also includes a storage medium such as a magnetic disk or semiconductor memory provided inside the computer or in a device connected via a network.

[0368] The present invention is not limited to the above-mentioned embodiment, and can be modified in various ways without departing from the gist of the present invention. The embodiments may be combined as appropriate, and in that case, the combined effect can be obtained. Furthermore, the above-mentioned embodiment includes various inventions, and various inventions can be extracted by combinations selected from the multiple components disclosed. For example, if the problem can be solved and the effect can be obtained even if some components are deleted from all the components shown in the embodiment, the configuration from which the components are deleted can be extracted as an invention. [Explanation of symbols]

[0369] 1. Data analysis equipment 2...External device 10...Input data processing section 20...Hyperparameter estimation unit 30...Parameter estimation section 40...Parameter processing unit 50…Recording section 51... Input data recording section 52...Hyperparameter recording section 53...Parameter recording section 60...Input / output section

Claims

1. an input unit that receives input of index data, which is a set of indexes of analysis target data, a set of elements of the analysis target data, and size comparison data indicating the size relationship of the elements of the analysis target data; a first estimation unit that estimates a parameter that minimizes an upper bound of an error between a loss function used to estimate a factor of the analysis target data and an approximation of the loss function, based on the set of elements input by the input unit; a second estimation unit that estimates a factor of the analysis target data based on the index data input by the input unit, the magnitude comparison data, and the parameter estimated by the first estimation unit; A data analysis device comprising:

2. The index data is is a set of matrix or tensor indices, The elements of the analysis target data are: are the elements of each row of said matrix or tensor, The magnitude comparison data is The data indicates the magnitude relationship of the elements of the matrix or tensor. The data analysis device according to claim 1 .

3. The second estimation unit is optimizing a function based on a regularization term related to a factor of the analysis target data, a value based on the index data input by the input unit, and a value based on the magnitude comparison data input by the input unit and the estimated parameter, thereby estimating the factor of the analysis target data; The data analysis device according to claim 1 .

4. The first estimation unit estimating the parameters that minimize a function based on a probability density function of elements of the analysis target data; The data analysis device according to claim 1 .

5. A method performed by a data analysis device, comprising: receiving input of index data which is a set of indexes of analysis target data, a set of elements of the analysis target data, and size comparison data which indicates the size relationship of the elements of the analysis target data; estimating a parameter that minimizes an upper bound of an error between a loss function used to estimate a factor of the analysis target data and an approximation of the loss function based on the input set of elements; estimating a factor of the analysis target data based on the input index data, the magnitude comparison data, and the estimated parameter; A data analysis method comprising:

6. The index data is is a set of matrix or tensor indices, The elements of the analysis target data are: are the elements of each row of said matrix or tensor, The magnitude comparison data is The data indicates the magnitude relationship of the elements of the matrix or tensor. The data analysis method according to claim 5 .

7. estimating the factor and optimizing a function based on a regularization term related to a factor of the analysis target data, a value based on the input index data, and a value based on the input magnitude comparison data and the estimated parameter, thereby estimating the factor of the analysis target data. The data analysis method according to claim 5 .

8. A data analysis processing program that causes a processor to function as each of the units of the data analysis device according to claim 1 .

Images