Data analysis device, method and program

Abstract

A data analysis device according to one embodiment has: an input unit for accepting input of index data that is a set of indices for data being analyzed, a set of elements of the data being analyzed, and magnitude comparison data indicating the magnitude relationship between the elements of the data being analyzed; a first estimation unit for estimating a parameter to minimize the upper boundary of an error between a loss function used in estimation of factors for the data being analyzed and an approximation of the loss function, on the basis of the set of elements inputted through the input unit; and a second estimation unit for estimating factors for the data being analyzed under a condition that satisfies a non-negativity constraint, on the basis of the index data and magnitude comparison data that were inputted through the input unit and the parameter that was estimated by the first estimation unit.

Description

[Technical field]

[0001] An embodiment of the present invention relates to a data analysis device, a method, and a program. [Background technology]

[0002] In recent years, a method called Nonnegative Matrix Factorization (NMF) or Nonnegative Tensor Factorization (NTF) has been widely used in data analysis (see, for example, Non-Patent Documents 1 and 2). A tensor is a high-dimensional array with three or more axes.

[0003] When the values ​​of elements of the analysis target, such as documents, purchasing history, user attribute information, movie rating history, etc., can be expressed as non-negative matrices or tensors, it is possible to automatically extract patterns in the data or complement missing values ​​in the data by factoring matrix-represented data into products of non-negative matrices using NMF, or factoring tensor-represented data into products of non-negative matrices using NTF. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Masahiro KOHJIMA, Tatsushi MATSUBAYASHI, Hiroshi SAWADA, Multiple Data Analysis and Non-negative Matrix / Tensor Factorization [I]:Multiple Data Analysis and Its Advances, The journal of the Institute of Electronics, Information and Communication Engineers, Vol.99, No.6, pp.543-550, jun 2016. [Non-Patent Document 2] Tatsushi MATSUBAYASHI, Masahiro KOHJIMA, Hiroshi SAWADA, Multiple Data Analysis and Non-negative Matrix / Tensor Factorization [II·Finish] : Tensor Data Analysis and Applications, The journal of the Institute of Electronics, Information and Communication Engineers, Vol.99, No.7, pp. 691-698, jul 2016. Summary of the Invention [Problem to be solved by the invention]

[0005] However, in recent data analysis, it is sometimes necessary to analyze data in which the index of an element of a matrix or tensor does not correspond to the value of that element, which is created as a result of the data collection method or processing for privacy protection. For example, in the case of extracting patterns of users' movie rating trends, normal NMF uses a matrix representation of movie rating history.

[0006]

number

[0007] We assume that the matrix representation of movie rating history is available, and patterns are extracted by applying NMF to the input. For example, row index i corresponds to a user, column index j corresponds to a movie, and the matrix element value x ij is a matrix showing the rating value of movie j by user i.

[0008] However, such a matrix representation of movie rating history is not necessarily usable. For example, the fact that user i rated or watched movie j may be recorded and used, but the rating value is not recorded or made public in a form linked to movie j from the viewpoint of privacy protection, and the distribution of ratings of movie j by user i, for example, there are z1 movies rated "1", z2 movies rated "2", etc., may be available. In this case, the rating of movie j by user i cannot be observed, so there is no correspondence between the index (i, j) of the matrix element and the value of that element, i.e., x ij Since it is not known what the value of is, it is expressed as data that cannot be expressed in a matrix.

[0009] Similarly, when dealing with user attribute information where row index i corresponds to each attribute item such as weight and annual income, and column index j corresponds to a user, from the perspective of privacy protection, the rating value is not recorded or published in a form linked to user j, and may be available in the form of a distribution of the value of item i, for example, there are z1 users who weigh 60 kg, there are z2 users who weigh 61 kg, etc. In this case, as in the case of movie rating history, the information of user j regarding item i cannot be observed, and therefore it is not possible to establish a correspondence between the index of a matrix element and the value of that element, resulting in data that cannot be expressed in a matrix.

[0010] The present invention has been made in light of the above-mentioned circumstances, and has as its object to provide a data analysis device, method, and program that are capable of appropriately analyzing data to be analyzed regardless of whether there is a correspondence between the index and elements of the data to be analyzed. [Means for solving the problem]

[0011] A data analysis apparatus according to one embodiment of the present invention comprises an input unit that accepts input of index data, which is a set of indexes of data to be analyzed, a set of elements of the data to be analyzed, and size comparison data indicating the size relationship of the elements of the data to be analyzed, a first estimation unit that estimates parameters that minimize an upper bound of an error between a loss function used to estimate a factor of the data to be analyzed and an approximation of the loss function, based on the set of elements input by the input unit, and a second estimation unit that estimates parameters of the data to be analyzed under conditions that satisfy non-negative constraints, based on the index data, the size comparison data input by the input unit, and the parameters estimated by the first estimation unit.

[0012] A data analysis method according to one embodiment of the present invention is a method performed by a data analysis device, and includes accepting input of index data, which is a set of indexes of data to be analyzed, a set of elements of the data to be analyzed, and size comparison data indicating the size relationship of the elements of the data to be analyzed; estimating parameters that minimize an upper bound of the error between a loss function used to estimate a factor of the data to be analyzed and an approximation of the loss function, based on the input set of elements; and estimating factors of the data to be analyzed under conditions that satisfy non-negative constraints, based on the input index data, the size comparison data, and the estimated parameters. Effect of the Invention

[0013] According to the present invention, data to be analyzed can be appropriately analyzed regardless of the presence or absence of correspondence between the index and element of the data to be analyzed. [Brief description of the drawings]

[0014] [Figure 1] FIG. 1 is a diagram showing an application example of a data analysis device according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a flow chart showing an example of an outline of the operation of the data analysis device according to one embodiment of the present invention. [Diagram 3] FIG. 3 is a flowchart showing an example of an operation of the hyper-parameter estimation unit. [Figure 4] FIG. 4 is a flowchart showing an example of the operation of the parameter estimation unit. [Diagram 5] FIG. 5 is a block diagram showing an example of a hardware configuration of a data analysis device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In this embodiment, we will explain a method that can automatically extract patterns in data and estimate the values ​​of matrix elements, such as NMF or NTF, even if the data is given in a format in which the index of a matrix or tensor element cannot correspond to the value of that element.

[0016] Conventional NMF (NTF) cannot analyze data in which the indices of matrix (tensor) elements do not correspond to the values ​​of those elements, as described above. This is because NMF assumes that the input data is expressed in a matrix, as shown below.

[0017] NMF is an input matrix

[0018]

number

[0019] A factor matrix with non-negative elements that corresponds to the factor matrix with i rows and r columns and the factor matrix with r rows and j columns, respectively, for low rank approximation of

[0020]

number

[0021] and

[0022]

number

[0023] Patterns are extracted by estimating as the result of factorization.

[0024] The estimation of the factor matrix is ​​done by the matrix

[0025]

number

[0026] and the product of the factor matrices

[0027]

number

[0028] This is done so that the values ​​of and are similar.

[0029]

number

[0030] The (i, j) element of

[0031]

number

[0032] The loss function used as a measure of similarity above, that is, the smaller the value, the more similar the two matrices are, is the L2 divergence function shown below.

[0033]

number

[0034] Or Bregman Divergence (BD)

[0035]

number

[0036] (For example, see "InderjitS Dhillon and Suvrit Sra. Generalized nonnegative matrix approximations with bregman divergences. In Proceedings of the 18th International Conference on Neural Information Processing Systems, pp. 283-290, 2005." (hereinafter, Reference 1) and the above-mentioned Non-Patent Document 1).

[0037]

number

[0038] In addition,

[0039]

number

[0040] is a convex function,

[0041]

number

[0042] is its derivative. The function

[0043]

number

[0044] By changing , BD can express various functions. For example,

[0045]

number

[0046] corresponds to the squared error when

[0047]

number

[0048] When

[0049]

number

[0050] When , it corresponds to I-divergence (also called generalized KL (Kullback-Leibler) divergence),

[0051]

number

[0052] When , it corresponds to the Itakura-Saito pseudo-distance. Determining the loss function to be used is equivalent to making an assumption on the probability distribution that generates the matrix elements. Specifically, using the above squared error, I-divergence, or Itakura-Saito pseudo-distance corresponds one-to-one to the assumption that the matrix elements are generated according to normal distribution, Poisson distribution, or exponential distribution, respectively.

[0053] The factor matrix is ​​estimated by minimizing the loss function shown above, subject to the constraint that each element of the factor matrix must be a non-negative value. Various methods can be used for this optimization, such as Stochastic Gradient Descent (SGD), projected gradient method, and majorization minimization. For example, the majorization minimization method, which minimizes BD, performs optimization by repeatedly updating the factor matrix shown below (see Reference 1 above).

[0054]

number

[0055] however,

[0056]

number

[0057] is the element-wise product (Hadamard product),

[0058]

number

[0059] In matrix

[0060]

number

[0061] Represents the (i, r) component of.

[0062] Note that when applying NTF, we need to consider low-rank approximations of tensors rather than matrices. For example, a 3rd order tensor

[0063]

number

[0064] is approximated by CP decomposition (Canonical Polyadic decomposition), that is, by decomposing it into factors with a basis number R, the element x of the tensor ijk and its factor matrix

[0065]

number

[0066] and

[0067]

number

[0068] Approximation by

[0069]

number

[0070] It is sufficient to estimate that and are similar values.

[0071] The above NMF (NTF) cannot be applied to data that lacks a matrix (tensor) representation, such as the data to be analyzed this time. Because the data is not represented as a matrix (tensor), it is not possible to calculate the value of the loss function, nor is it possible to calculate the update formula for the auxiliary function. Therefore, in this embodiment, the following new method is presented.

[0072] A method for dealing with a related problem, different from the method in this embodiment, is "Masahiro Kohjima, Tatsushi Matsubayashi, and Hiroyuki Toda. Generalized interval valued nonnegative matrix factorization. In IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 3412-3416, 2019. (Reference 2) and the method described in "Yuto Yamaguchi and Kohei Hayashi. Tensor decomposition with missing indices. In International Joint Conference on Artificial Intelligence, pp.3217-3223, 2017." (Reference 4). In the method described in Reference 2, the values ​​of the matrix elements themselves cannot be observed, but information on a certain range of values, for example, x ij When information is available, such as the value of is between 1 and 3, a method has been proposed to use that information to perform NMF.

[0073] The problem considered in this embodiment is different from the problem where information on a certain interval of values ​​is given, and therefore this method cannot be applied. Reference 4 above proposes a method that can perform tensor decomposition even when there is a missing tensor index. In this method, a pair of tensor index and element (i, j, k, x ijk Even if some of the indices in , for example j, are missing, the tensor decomposition is applied by estimating the value of the missing index.

[0074] The method considered in this embodiment differs from the method described in Reference 4 above in that it is a method that can analyze data without the need to estimate missing values ​​and establish correspondence between indices and matrix elements.

[0075] In addition, in deriving the objective function used for parameter estimation in this embodiment, an approximation of the same kind as that used to derive the objective function in the regression method described in "Liyuan Xu, Gang Niu, Junya Honda, and Masashi Sugiyama. Uncoupled regression from pairwise comparison data. In Proceedings of the 33rd International Conference on Neural Information Processing Systems, pp. 3992-4002, 2019." (Reference 3) was used. However, the problem addressed by the method described in Reference 3 here is a regression problem of estimating a function f that determines the relationship y=f(x) between an input x and an output y, which is a different problem from the nonnegative matrix decomposition or nonnegative tensor decomposition considered in this embodiment.

[0076] In this embodiment, a method is described that can perform nonnegative matrix factorization or nonnegative tensor factorization even for data that does not have a correspondence between the index of a matrix or tensor element and the value of the element, and is insufficient to be expressed as a matrix or tensor. Hereinafter, this method is described as Uncoupled Nonnegative Matrix Factorization (UNMF) and Uncoupled Nonnegative Tensor Factorization (UNTF).

[0077] (formulation) Next, we will explain the formulation, which is the problem setting of the method described in this embodiment. Below, we will explain the formulation of UNMF, but the formulation of UNTF is almost the same if matrices are replaced with tensors based on the formulation of UNMF. The size of the matrix is ​​n I ×n J In the case of the movie rating history data mentioned above, n I is the number of users, n Jrepresents the number of movies. The set of matrix indices is

[0078]

number

[0079] This is hereafter referred to as index data. U represents the number of indexes in this data. In the case of the movie rating history data mentioned above,

[0080]

number

[0081] is user i m The movie m It corresponds to a collection of data that records whether a user has rated or viewed a program.

[0082] Furthermore, the set of distributions for the elements of each row of the matrix, i.e., the set of probability distributions, is

[0083]

number

[0084] The distribution of elements in each row above is the Probability Density Function (PDF) of the matrix elements conditioned (or fixed) on row i and marginalized with respect to column j.

[0085] In the case of the movie rating history data mentioned above, f X|I=i Let (x) be the probability density function of movie rating value x by a certain user i, which can be constructed from prior knowledge, estimated from aggregate data that shows that there are z1 movies rated "1" and z2 movies rated "2", or the set of values ​​of the elements of each row of the matrix.

[0086]

number

[0087] can be used. V represents the number of elements in this data. m represents the row index, and x m is the i m It represents the value of an element in a certain column of a row, but the column is unknown. As will be described later, when using aggregated data or a set of element values ​​in each row, the method can be applied without explicitly estimating this probability density function.

[0088] In addition to the above, the data showing the magnitude relationship of the matrix elements is

[0089]

number

[0090] This will be referred to as size comparison data from now on. m is the row index,

[0091]

number

[0092] and,

[0093]

number

[0094] and are both column indexes and element values.

[0095]

number

[0096] The value of

[0097]

number

[0098] It indicates that the value of element is greater than the value of element .

[0099]

number

[0100] It is not something that has been observed. R represents the number of pairs of matrix elements that are compared. Such data can be obtained, for example, by having a user evaluate a movie j and judge whether it is better than another movie j', and recording the result of this judgment. This type of data is often easier for users to answer than giving a score such as 1 or 2, and it is also easier to collect the data.

[0101] Also, when the elements of the matrix are not movie ratings but rather represent the number of visits to a store, answering only that you visit one store more than another store, rather than inputting the actual number of visits, allows you to hide the fact that you have a favorite store that you visit very frequently and allows you to answer with less information about yourself.

[0102] UNMF is index data

[0103]

number

[0104] , a set of probability distributions

[0105]

number

[0106] , and size comparison data

[0107]

number

[0108] The input is three data, and the factor matrix

[0109]

number

[0110] Hereafter, this factor matrix is ​​collectively referred to as

[0111]

number

[0112] The estimate of each element using the factor matrix is

[0113]

number

[0114] It is written as follows.

[0115] (Loss Function) Next, we will explain how to derive the loss function for the above estimation. In defining the loss function, the random variable whose realized value corresponds to the row index is written as I, and the random variable whose realized value corresponds to the column index is written as J. Furthermore, the random variables whose realizations correspond to the values ​​of the matrix elements are written as X. The probability distribution that these random variables follow is P I,J,X It is written as follows. Furthermore, the conditional probability distribution when I = i and the probability density function of this probability distribution are P J,X|I =i, f J,X|I It is written as =i. The conditional distribution is further marginalized with respect to J or X, and the distribution is PX|I=i , P J|I=i The probability distribution P X|I=i The probability density function of f X|I=i , the cumulative density function is F X|I=i By definition,

[0116]

number

[0117] It is.

[0118] The loss function used to estimate the parameter θ using the random variables above is defined as the following equation (1) using the Bregman divergence above.

[0119]

number

[0120] However, in equation (1),

[0121]

number

[0122] is the probability distribution P I,J,X This is an expected value operation by

[0123]

number

[0124] is defined by the following equation (2).

[0125]

number

[0126] However, in equation (2),

[0127]

number

[0128] are probability distributions P J,X|I=i , P J|I=i The difficulty with this loss function is caused by the last term on the right-hand side of equation (2),

[0129]

number

[0130] This is because, although this term is defined by the joint distribution of the column index J and the element X, in the problem setting considered in this embodiment, the index and the element are not associated and cannot be observed simultaneously, so that it cannot be calculated even if, for example, sample approximation is performed. Therefore, hereinafter, we consider approximately evaluating this term.

[0131] Here, we newly define a set of random variables (J + , J - ) is introduced. This set is the set of elements (i, J + ) is the value of (i, J - ) and is defined as follows:

[0132]

number

[0133] Here, (J, X) and (J′, X′) both have probability P J,X|I=i From this definition, the above-mentioned size comparison data

[0134]

number

[0135] We will later use the fact that we can consider J to be the realization of this random variable. + and J. - The probability density functions that follow are

[0136]

number

[0137] and

[0138]

number

[0139] It is written as "J" + and J. - The expected value operation regarding the outcome of

[0140]

number

[0141] Using this random variable, it can be shown that the following equations (3) and (4) hold true.

[0142]

number

[0143] However, F X|I=i is the cumulative density function for the element values ​​with row i held fixed

[0144]

number

[0145] It is.

[0146] We will now explain the proof of formula (3). + By the definition of f J+|I=ican be expanded as follows:

[0147]

number

[0148] where Z is a normalization constant defined as follows:

[0149]

number

[0150] By integration by parts, we can derive Z = 1 / 2. Using this result, we can derive equation (3) as follows:

[0151]

number

[0152] The derivation of equation (4) is similar. This completes the proof.

[0153] Using equations (3) and (4), we can see that the conditional marginal distribution of the random variable X is a uniform distribution on [0, 1] (i.e., F X|I=i When (x)=x), the last term on the right hand side of equation (2) is

[0154]

number

[0155] Considering this fact, a certain hyperparameter

[0156]

number

[0157] It is considered promising to use the approximation shown in the following equation (5).

[0158]

number

[0159] In equation (5), the value on the right side approximates the value on the left side. In the case of the uniform distribution on [0, 1] above, (w i1 , w i2 ) = (1 / 2, 0), this approximation is accurate. This is because the conditional marginal distribution of the random variable X is a uniform distribution on [a, b] (F X|I=i This can also be generalized to (x)=(xa) / (ba) for all x∈[a, b]), by defining (wi1, wi2)=(b / 2, a / 2). When considering a more general distribution than the uniform distribution, the generalization loss is

[0160]

number

[0161] The hyperparameter (w i1 , w i2 ), which is discussed below.

[0162] Adding together equations (3) and (4), we obtain the following equation:

[0163]

number

[0164] Using this formula, a constant λ i Using this, the above equation (5) can be transformed into the following equation (6).

[0165]

number

[0166] λi There is a degree of freedom in setting λ i =0 or λ i =(w i1 +w i2 ) / 2, etc. However, the basic rule is that i The value of is preferably equal to or greater than zero. i ≧0. Using the above formula (6), the generalization loss of formula (1)

[0167]

number

[0168] Approximation of

[0169]

number

[0170] is obtained as follows:

[0171]

number

[0172] however,

[0173]

number

[0174] The above C i is a constant that does not depend on the parameter θ. Therefore, the random variables I, J, J + and J. - By replacing the expected value of with the sample average, we obtain the empirical risk as shown in the following equation (7).

[0175]

number

[0176] is obtained.

[0177]

number

[0178] Here, C is a constant that does not depend on the parameter θ. Since these quantities, except for the constant C, can be calculated from data, they can be used as the objective function for estimating the parameter θ. In the next section, we will consider the empirical loss

[0179]

number

[0180] We present a parameter estimation method, i.e., an estimation algorithm, that minimizes

[0181] (Estimation algorithm) To estimate the parameter θ, any optimization method can be used, such as stochastic projected gradient descent, multiplicative update, projected quasi-Newton method, alternating least squares method (ALS), auxiliary function method, etc.

[0182]

number

[0183] The partial derivative of can be calculated from the data as follows:

[0184]

number

[0185] however,

[0186]

number

[0187] is the index data

[0188]

number

[0189] From i m = i is extracted from all data,

[0190]

number

[0191] is the index data

[0192]

number

[0193] From, j m = j,

[0194]

number

[0195] is a comparison data

[0196]

number

[0197] From i m = i is extracted from all data,

[0198]

number

[0199] is a comparison data

[0200]

number

[0201] from,

[0202]

number

[0203] This is an extract of all the data that is

[0204]

number

[0205] is a comparison data

[0206]

number

[0207] from,

[0208]

number

[0209] Using these values, we can derive an update equation for estimation, for example, as follows:

[0210] The above projected gradient descent method is explained below. In this projected gradient descent method, parameter updates are performed in the following format, which combines updating the parameters in the steepest descent direction with a projection operation to satisfy the non-negativity constraint. The update formulas can be specifically written as the following formulas (8) and (9).

[0211]

number

[0212] where ηir, γrj are the learning rates and [·] + represents an arbitrary projection operation, for example, to satisfy the non-negativity constraint. An example of a projection is [y] using a sufficiently small positive integer ε. + =max(ε, y). The parameter θ can be estimated by repeatedly updating the parameter θ according to equations (8) and (9).

[0213] The above multiplicative update method will now be explained. In this multiplicative update method, the learning rate in the above projected gradient descent method is set so that the partial differential, here the positive component in the steepest descent direction, is the denominator, and the negative component is the numerator, to derive the update formula. Following this, the following update formulas (10) and (11) can be derived.

[0214]

number

[0215] When applying the multiplicative update method, as shown above, all values ​​on the right side of the update equation are positive, so by setting the initial value to a non-negative value, the non-negativity constraint is naturally satisfied. By repeatedly updating the parameter θ according to the above equations (10) and (11), the function

[0216]

number

[0217] However, under the assumption that certain conditions are satisfied, a local optimum of the objective function can be reached.

[0218] As mentioned above, BD is a convex function that

[0219]

number

[0220] By changing , we can express various loss functions, so we can derive various estimation formulas from the above update formula. For example,

[0221]

number

[0222] , that is, when using a function equivalent to the squared error,

[0223]

number

[0224] Since is a constant, the factor matrix when using the multiplicative update method is

[0225]

number

[0226] The update equation for is derived as follows:

[0227]

number

[0228] Similarly, we can derive update equations for any loss function that can be expressed in BD, such as the logistic loss, I-divergence, or Itakura-Saito pseudodistance mentioned above.

[0229] (Hyperparameter estimation) Finally, the hyperparameter {w i1 , w i2 This hyperparameter is estimated by the function Err i It can be determined by minimizing

[0230]

number

[0231] This function is

[0232]

number

[0233] of

[0234]

number

[0235] To be precise, any factor matrix

[0236]

number

[0237] and for component (i, j)

[0238]

number

[0239] Under the condition that

[0240]

number

[0241] Therefore, the above function Err i The approximation error can be reduced by minimizing the function Err i involves integral calculations, but by assumption, the distribution f X|I=i Since (x) is known, it can be calculated as a discrete approximation. Specifically, for example, we can calculate the whole possible value of x as

[0242]

number

[0243] For example, the 0.01 quantile of x is

[0244]

number

[0245] Let the 0.99th quantile be

[0246]

number

[0247] As,

[0248]

number

[0249] By minimizing the right-hand side of the following equation (13), it can be estimated using any optimization method.

[0250]

number

[0251] Since this equation (13) is defined as the sum of absolute values, it is possible to use a method that can handle the existence of non-differentiable points in the objective function, such as the subgradient method, or linear programming. When using the subgradient method, the function

[0252]

number

[0253] Wi =(w i1 , w i2 )

[0254]

number

[0255] Using any vector g belonging to

[0256]

number

[0257] We can then iteratively update the parameter θ according to the following, where γ′ is the learning rate.

[0258] In the above, the probability distribution f X|I=i We have shown how to estimate hyperparameters when (x) is known, but the set of element values ​​for each row of the matrix

[0259]

number

[0260] It is also possible to estimate hyperparameters using

[0261]

number

[0262] From i m The value extracted for i is

[0263]

number

[0264] If this is written, the function Err iWe can consider the approximation shown in the following equation (15).

[0265]

number

[0266] however,

[0267]

number

[0268] is the cumulative density function F X|I=i Empirical approximation of

[0269]

number

[0270] It is.

[0271]

number

[0272] teeth

[0273]

number

[0274] The number of data items is also expressed by the function Err i It is possible to approximate the above. For optimizing the above formula (15), any optimization method can be used, such as the subgradient method, as in the case of optimizing the above formula (13).

[0275] According to the present embodiment, even if the index of a matrix or tensor element does not correspond to the value of the element, and the data is insufficient in terms of matrix or tensor expression, it is possible to perform nonnegative matrix decomposition or nonnegative tensor decomposition. This makes it possible to extract latent patterns from data that is insufficient in terms of matrix or tensor expression, and to estimate values ​​of matrix elements.

[0276] (Explanation of operation) The operation of each unit of the data analysis device 1 according to this embodiment will be described below. FIG. 1 is a diagram showing an application example of a data analysis device according to an embodiment of the present invention. 1, a data analysis device 1 according to an embodiment of the present invention includes an input data processing unit 10, a hyperparameter estimation unit 20, a parameter estimation unit 30, a parameter processing unit 40, a recording unit 50, and an input / output unit 60. The input / output unit 60 is capable of inputting and outputting data to and from an external device 2. The recording unit 50 includes an input data recording unit 51 , a hyper-parameter recording unit 52 , and a parameter recording unit 53 .

[0277] Next, an overview of the operation of the data analysis device 1 will be described below. Fig. 2 is a flowchart showing an example of the overview of the operation of the data analysis device according to an embodiment of the present invention. First, the input data processing unit 10 of the data analysis device 1 receives index data

[0278]

number

[0279] , a set of probability distributions

[0280]

number

[0281] , Large and small comparison data

[0282]

number

[0283] is input from the external device 2 via the input / output unit 60 (step S1).

[0284] Next, the hyper-parameter estimation unit 20 estimates the hyper-parameters

[0285]

number

[0286] is estimated (step S2).

[0287] Next, the parameter estimation unit 30 estimates the parameter θ (step S3). Next, the parameter processing unit 40 outputs the estimated parameter θ to the external device 2 via the input / output unit 60 (step S4).

[0288] The specific operations of S1 to S4 above will be described below. First, a specific example of S1 will be described. The input data processing unit 10 receives index data

[0289]

number

[0290] , a set of probability distributions

[0291]

number

[0292] , and size comparison data

[0293]

number

[0294] is input and stored in the input data recording section 51 of the recording section 50.

[0295] Next, a specific example of S2 will be described. The hyperparameter estimation unit 20 estimates a set of probability distributions stored in the input data recording unit 51.

[0296]

number

[0297] The hyper-parameter w is estimated by inputting the hyper-parameter w and minimizing the objective function of the above equation (13) by the following methods (a) to (d). The hyper-parameter estimation unit 20 stores the estimated hyper-parameter w in the hyper-parameter recording unit 52 of the recording unit 50. FIG. 3 is a flowchart showing an example of the operation of the hyper-parameter estimation unit. FIG. 3 shows the hyper-parameter w when the subgradient method is used. i An example of updating is shown. This update is performed for i=1,…,n I By performing this step once each time, the hyper-parameter w to be estimated can be obtained.

[0298] (a) The hyperparameter estimation unit 20 estimates the parameter w i (step S21). As a variable used as a termination condition for the process in step S2, the hyper-parameter estimation unit 20 initializes the hyper-parameter w i Similarly, a variable δ indicating the maximum change range of the update amount of is initialized, and the threshold ε of the above-mentioned termination condition and the maximum number of iterations of the calculation, here the maximum value of the number of iterations of the calculation related to the update of the variable δ, are set (step S22).

[0299] (b) The hyper-parameter estimation unit 20 updates the parameter w according to the above formula (14). At this time, the hyper-parameter estimation unit 20 updates the pre-update hyper-parameter w i and the updated hyperparameter w i The maximum absolute value of the difference between

[0300]

number

[0301] is recorded in the variable δ (step S23). However, here, the pre-update hyper-parameter w i The elements of

[0302]

number

[0303] The updated hyperparameter w i The elements of

[0304]

number

[0305] It is described as follows.

[0306] (c) In response to step S23, the hyper-parameter estimation unit 20 updates the count value of the number of times the calculation is repeated (step S24). (d) If the count value of the number of calculation iterations exceeds the predetermined maximum number of iterations or the variable δ indicating the maximum change width of the update amount is smaller than the predetermined threshold ε (Yes in step S25), the hyper-parameter estimation unit 20 ends the process of step S2. Otherwise (No in step S25), the hyper-parameter estimation unit 20 sets the variable δ to δ←0 and returns to step S22.

[0307] Next, a specific example of S3 will be described. The parameter estimation unit 30 estimates the index data stored in the input data recording unit 51.

[0308]

number

[0309] , Large and small comparison data

[0310]

number

[0311] , and the hyper-parameter w stored in the hyper-parameter recording unit 52, and estimates the parameter θ by minimizing the objective function of the above equation (7) by the following methods (a) to (e). The parameter estimation unit 30 stores the estimated parameter θ in the parameter recording unit 53 of the recording unit 50. FIG. 4 is a flowchart showing an example of the operation of the parameter estimation unit. FIG. 4 shows an example of updating the parameter θ when the multiplicative update method is used.

[0312] (a) The parameter estimation unit 30 initializes the parameter θ (step S31). As a variable used as a termination condition for the process in step S3, the parameter estimation unit 30 similarly initializes a variable δ indicating the maximum change range of the update amount of the parameter θ, and sets a threshold ε of the termination condition and a maximum number of iterations of the calculation, here the maximum value of the number of iterations of the calculation related to the update of the variable δ (step S32).

[0313] (b) The parameter estimation unit 30 estimates the factor matrix

[0314]

number

[0315] is updated according to the above formula (10). At this time, the parameter estimation unit 30 updates the factor matrix before and after the update

[0316]

number

[0317] The maximum absolute value of the difference between

[0318]

number

[0319] If is greater than the current variable δ, the parameter estimation unit 30 sets this variable δ as

[0320]

number

[0321] (Step S33). However, the factor matrix before the update is

[0322]

number

[0323] The elements of

[0324]

number

[0325] After updating, the above elements are

[0326]

number

[0327] It is described as follows.

[0328] (c) The parameter estimation unit 30 calculates the factor matrix

[0329]

number

[0330] is updated according to the above formula (11). At this time, the parameter estimation unit 30 calculates the factor matrix

[0331]

number

[0332] The maximum absolute value of the difference between

[0333]

number

[0334] If is greater than the current variable δ, set this variable δ to

[0335]

number

[0336] (Step S34). However, the factor matrix before the update is

[0337]

number

[0338] The elements of

[0339]

number

[0340] After updating, the above elements are

[0341]

number

[0342] It is described as follows.

[0343] (d) In response to step S34, the parameter estimation unit 30 updates the count value of the number of times the calculation is repeated (step S35).

[0344] (e) If the count value of the number of calculation repetitions exceeds the predetermined maximum number of repetitions, or if the variable δ indicating the maximum change range of the update amount is smaller than the predetermined threshold ε (Yes in step S36), the parameter estimation unit 30 ends the process related to step S3; otherwise (No in step S36), the parameter estimation unit 30 sets the variable δ to δ←0 and returns to step S32.

[0345] Next, a specific example of S4 will be described. The parameter processing unit 40 outputs the parameter θ stored in the parameter recording unit 53 to the external device 3 via the input / output unit 60. In the above embodiment, the algorithm based on the multiplicative update method is used to estimate the parameter θ that minimizes the objective function of the above equation (7), but any other method, such as the above projected gradient descent method, may be used. Also, in the above embodiment, a set of probability distributions is used to estimate the hyperparameters.

[0346]

number

[0347] Instead of using the set of element values ​​for each row of the matrix,

[0348]

number

[0349] Alternatively, aggregated data may be used. In this case, the hyperparameters can be estimated by minimizing (optimizing) the above formula (15) instead of the above formula (13). Any method can be used to optimize K. In addition, the above shows an example of applying UNMF, which performs nonnegative matrix decomposition on data that lacks a matrix representation, but in a similar framework, it is also possible to apply UNTF, which performs nonnegative tensor decomposition on data that lacks a tensor representation.

[0350] FIG. 5 is a block diagram showing an example of a hardware configuration of a data analysis device according to an embodiment of the present invention. 5, the data analysis device 1 according to the embodiment is configured, for example, by a server computer or a personal computer, and has a hardware processor 111A such as a CPU (Central Processing Unit). A program memory 111B, a data memory 112, an input / output interface 113, and a communication interface 114 are connected to the hardware processor 111A via a bus 115.

[0351] The communication interface 114 includes, for example, one or more wireless communication interface units, and enables transmission and reception of information to and from a communication network NW. As the wireless interface, for example, an interface that adopts a low-power wireless data communication standard such as a wireless LAN (Local Area Network) is used.

[0352] An input device 200 and an output device 300 used by a user or the like, which are attached to the data analysis apparatus 1, are connected to the input / output interface 113. The input device 200 and the output device 300 correspond to the external device 2 shown in FIG. The input / output interface 113 takes in operation data input by a user or the like through an input device 200 such as a keyboard, a touch panel, a touchpad, a mouse, or the like, and outputs output data to an output device 300 including a display device using liquid crystal or organic EL (Electro Luminescence), etc., for display. The input device 200 and the output device 300 may be devices built into the data analyzing apparatus 1, or may be input devices and output devices of other information terminals capable of communicating with the data analyzing apparatus 1 via the network NW.

[0353] The program memory 111B is a non-transient tangible storage medium that is a combination of a non-volatile memory that can be written to and read from at any time, such as a hard disk drive (HDD) or a solid state drive (SSD), and a non-volatile memory such as a read only memory (ROM), and stores programs necessary for executing various control processes, etc. according to one embodiment.

[0354] The data memory 112 is a tangible storage medium that is a combination of, for example, the above-mentioned nonvolatile memory and a volatile memory such as a RAM (Random Access Memory), and is used to store various data acquired and created during various processing steps.

[0355] A data analysis device 1 according to one embodiment of the present invention can be configured as a data processing device having an input data processing unit 10, a hyperparameter estimation unit 20, a parameter estimation unit 30, a parameter processing unit 40, and an input / output unit 60 shown in FIG. 1 as processing function units based on software.

[0356] Each information storage unit and recording unit 50 used as a working memory by each unit of the data analysis device 1 can be configured by using the data memory 112 shown in Fig. 5. However, these configured storage areas are not essential components within the data analysis device 1, and may be areas provided in a storage device such as an external storage medium such as a Universal Serial Bus (USB) memory, or a database server located in the cloud.

[0357] The processing function units in each of the input data processing unit 10, the hyper parameter estimation unit 20, the parameter estimation unit 30, the parameter processing unit 40, and the input / output unit 60 can be realized by reading and executing a program stored in the program memory 111B by the hardware processor 111A. Note that some or all of these processing function units may be realized in various other forms, including integrated circuits such as an application specific integrated circuit (ASIC) or a field-programmable gate array (FPGA).

[0358] The methods described in each embodiment may be stored as a program (software means) that can be executed by a computer on a recording medium such as a magnetic disk (floppy disk, hard disk, etc.), an optical disc (CD-ROM, DVD, MO, etc.), or a semiconductor memory (ROM, RAM, flash memory, etc.), and may be distributed by transmitting the program via a communication medium. The programs stored on the medium include a setting program that configures the software means (including not only execution programs but also tables and data structures) that the computer executes. The computer that realizes the present device reads the program recorded on the recording medium, and in some cases, configures the software means by the setting program, and executes the above-mentioned processing by controlling the operation of the software means. The recording medium in this specification is not limited to a recording medium for distribution, but also includes a storage medium such as a magnetic disk or semiconductor memory provided inside the computer or in a device connected via a network.

[0359] The present invention is not limited to the above-mentioned embodiment, and can be modified in various ways without departing from the gist of the present invention. The embodiments may be combined as appropriate, and in that case, the combined effect can be obtained. Furthermore, the above-mentioned embodiment includes various inventions, and various inventions can be extracted by combinations selected from the multiple components disclosed. For example, if the problem can be solved and the effect can be obtained even if some components are deleted from all the components shown in the embodiment, the configuration from which the components are deleted can be extracted as an invention. [Explanation of symbols]

[0360] 1. Data analysis equipment 2...External device 10...Input data processing section 20...Hyperparameter estimation unit 30...Parameter estimation section 40...Parameter processing unit 50…Recording section 51... Input data recording section 52...Hyperparameter recording section 53...Parameter recording section 60...Input / output section

Claims

1. an input unit that receives input of index data, which is a set of indexes of analysis target data, a set of elements of the analysis target data, and size comparison data indicating the size relationship of the elements of the analysis target data; a first estimation unit that estimates a parameter that minimizes an upper bound of an error between a loss function used to estimate a factor of the analysis target data and an approximation of the loss function, based on the set of elements input by the input unit; a second estimation unit that estimates a factor of the analysis target data under a condition that satisfies a non-negative constraint, based on the index data input by the input unit, the magnitude comparison data, and the parameter estimated by the first estimation unit; A data analysis device comprising:

2. The index data is is a set of matrix or tensor indices, The elements of the analysis target data are: are the elements of each row of said matrix or tensor, The magnitude comparison data is The data indicates the magnitude relationship of the elements of the matrix or tensor. The data analysis device according to claim 1 .

3. The second estimation unit is optimizing a function based on a regularization term related to a factor of the data to be analyzed, a value based on the index data input by the input unit, and a value based on the magnitude comparison data input by the input unit and the estimated parameter under a condition that satisfies a non-negative constraint, thereby estimating the factor of the data to be analyzed; The data analysis device according to claim 1 .

4. The first estimation unit estimating the parameters that minimize a function based on a probability density function of elements of the analysis target data; The data analysis device according to claim 1 .

5. A method performed by a data analysis device, comprising: receiving input of index data which is a set of indexes of analysis target data, a set of elements of the analysis target data, and size comparison data which indicates the size relationship of the elements of the analysis target data; estimating a parameter that minimizes an upper bound of an error between a loss function used to estimate a factor of the analysis target data and an approximation of the loss function based on the input set of elements; estimating a factor of the analysis target data under a condition that satisfies a non-negative constraint based on the input index data, the magnitude comparison data, and the estimated parameters; A data analysis method comprising:

6. The index data is is a set of matrix or tensor indices, The elements of the analysis target data are: are the elements of each row of said matrix or tensor, The magnitude comparison data is The data indicates the magnitude relationship of the elements of the matrix or tensor. The data analysis method according to claim 5 .

7. estimating the factor and optimizing a function based on a regularization term related to a factor of the data to be analyzed, a value based on the input index data, and a value based on the input magnitude comparison data and the estimated parameter under a condition that satisfies a non-negative constraint, thereby estimating the factor of the data to be analyzed. The data analysis method according to claim 5 .

8. A data analysis processing program that causes a processor to function as each of the units of the data analysis device according to claim 1 .

Images