Learning device, inference device, learning method, inference method, and program

Abstract

A learning device according to an aspect disclosed herein comprises: a selection unit that selects a sub-tensor from a given tensor; a first latent tensor creation unit that creates a first latent tensor representing a feature of the sub-tensor on the basis of a first model parameter and the sub-tensor; a second latent tensor creation unit that creates a second latent tensor by converting each element of the first latent tensor, while taking into consideration the values of other elements, on the basis of a second model parameter and the first latent tensor; a prediction unit that predicts a predicted tensor representing a predicted value of a predetermined task on the basis of a third model parameter and the second latent tensor; and an updating unit that updates the first model parameter, the second model parameter, and the third model parameter so that the prediction accuracy of the predicted tensor increases.

Description

Learning device, inference device, learning method, inference method, and program

[0001] The present disclosure relates to a learning device, an inference device, a learning method, an inference method, and a program.

[0002] Observation data obtained by observing various objects can be expressed in tensor form. A large amount of observation data is required to analyze the observation data expressed in tensor form with high accuracy, but there may be cases where a sufficient amount of observation data cannot be prepared.

[0003] A method called meta-learning has been proposed as a method that can achieve high performance in a target task by utilizing training data from different tasks, even when only a small amount of training data can be prepared for the target task (e.g., Non-Patent Document 1).

[0004] Finn, Chelsea, Pieter Abbeel, and Sergey Levine. "Model-agnostic meta-learning for fast adaptation of deep networks." Proceedings of the 34th International Conferenceon Machine Learning, 2017.

[0005] However, existing meta-learning methods have the problem that they cannot be used for data represented in tensor format.

[0006] The present disclosure has been made in consideration of the above points, and aims to provide a technology that can analyze data expressed in tensor format with high accuracy.

[0007] A learning device according to one aspect of the present disclosure includes a selection unit that selects a subtensor from a given tensor; a first latent tensor creation unit that creates a first latent tensor that represents characteristics of the subtensor based on first model parameters and the subtensor; a second latent tensor creation unit that creates a second latent tensor based on second model parameters and the first latent tensor by converting each element of the first latent tensor while taking into account values ​​of other elements; a prediction unit that predicts a prediction tensor that represents a predicted value of a predetermined task based on third model parameters and the second latent tensor; and an update unit that updates the first model parameters, the second model parameters, and the third model parameters so as to increase the prediction accuracy of the prediction tensor.

[0008] A technology is provided that can analyze data expressed in tensor format with high accuracy.

[0009] Fig. 1 is a diagram illustrating an example of the hardware configuration of a tensor learning device during learning; Fig. 2 is a diagram illustrating an example of the functional configuration of a tensor learning device during learning; Fig. 3 is a flowchart illustrating an example of a learning process according to the present embodiment; Fig. 4 is a diagram illustrating an example of the functional configuration of a tensor learning device during testing; and Fig. 5 is a flowchart illustrating an example of a test process according to the present embodiment.

[0010] An embodiment of the present invention will be described below with reference to the drawings. Hereinafter, observation data obtained by observing some object (e.g., a dynamic system, etc.) expressed in tensor format will be referred to as "tensor data" or simply "tensor." Note that observation data of any format, such as matrix data, time-series data, language data, graph data, etc., can be expressed in tensor format.

[0011] Furthermore, when auxiliary information exists for the observation data, the observation data and the auxiliary information can be expressed together as a tensor by combining data in which the observation data is expressed in tensor format with data in which the auxiliary information is expressed in tensor format. Hereinafter, when auxiliary information exists for the observation data, it is assumed that the observation data and the auxiliary information are expressed together as a tensor. Note that auxiliary information refers to auxiliary information other than information obtained by observing an object. For example, if the observation data represents the flow of people at each location at each date and time, a specific example of auxiliary information would be information indicating whether each day is a public holiday.

[0012] In the following embodiment, a tensor learning device 10 is described that, when given one or more tensors (particularly, multiple tensors that may have different sizes), learns a model that can analyze tensors of a desired size with high accuracy. Also described is a case where a trained model is used to analyze tensors of a desired size with high accuracy.

[0013] The size of a tensor refers to the number of modes of a tensor and the number of dimensions of each mode. Hereinafter, a tensor to be analyzed for a target task will be referred to as a "target tensor." The size of the target tensor may be any desired size and does not need to be uniquely determined. For example, when multiple target tensors are to be analyzed, the sizes of the target tensors may be different. Furthermore, analysis refers to performing various operations on the target tensor to obtain prediction results for the target task. Examples of tasks include various machine learning problems such as missing value imputation, classification, regression, clustering, and anomaly detection. Below, we will explain, as an example, a case where the target task is missing value imputation (i.e., a task of predicting and imputing unobserved values). However, missing value imputation as the target task is merely an example, and the following embodiment can be similarly applied to any machine learning problem other than missing value imputation.

[0014] Here, the tensor learning device 10 according to this embodiment has a "learning time" during which a model that accurately complements missing values ​​of a target tensor is learned, and a "test time" during which missing values ​​of the target tensor are accurately complemented using a trained model. The learning time may be called, for example, "training." The testing time may be called, for example, "inference," "estimation," "prediction," etc.

[0015] During learning, the tensor learning device 10 includes a learning data set E={T １ , ..., T Ｄ} is given. T ｄ is the dth tensor. Each tensor T ｄ is composed of real-valued elements and its size is N ｄ１ ×N ｄ２ ×...×N ｄＭ It is assumed that N ｄｍ is the dth tensor T ｄ The dimension of the m-th mode is M, and M represents the number of modes. For simplicity, each tensor T ｄ The number of modes M is common to all tensors, but it may be different for each tensor. ｄ The number of modes is M ｄ Each tensor T ｄ may have missing values ​​(i.e., each tensor T ｄ Elements of are allowed to have missing values.

[0016] At the time of testing, the tensor learning device 10 is provided with test data that includes at least the target tensor X. Hereinafter, since missing value imputation is assumed, the test data is assumed to include a mask tensor S in addition to the target tensor X. The target tensor X is a tensor of a desired size that is composed of elements that take real values ​​and that were not included in the training dataset E. Note that when there are multiple test data, the size of the target tensor X included in each test data may be different. The mask tensor S is a tensor that indicates which elements of the target tensor X are missing values, and is composed of elements that take 0 or 1, and its size is the same as that of the target tensor X. For example, if the size of the target tensor X is N１ ×N ２ ×...×N Ｍ Then, among the elements of the target tensor X, the dimension of the mth mode is n ｍ The element X(n １ , ..., n Ｍ ), among the elements of the mask tensor S, the dimension of the m-th mode is n ｍ The elements that are １ , ..., n Ｍ ) In this case, X(n １ , ..., n Ｍ ) is not a missing value, S(n １ , ..., n Ｍ ) = 1, X(n １ , ..., n Ｍ ) is a missing value, S(n １ , ..., n Ｍ ) = 0.

[0017] [During Learning] The tensor learning device 10 during learning will be described below.

[0018] <Example of Hardware Configuration of Tensor Learning Apparatus 10 During Learning> An example of the hardware configuration of the tensor learning apparatus 10 during learning will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the hardware configuration of the tensor learning apparatus 10 during learning.

[0019] 1, the tensor learning device 10 during learning includes an input device 101, a display device 102, an external I / F 103, a communication I / F 104, a random access memory (RAM) 105, a read-only memory (ROM) 106, an auxiliary storage device 107, and a processor 108. Each of these pieces of hardware is connected to each other via a bus 109 so as to be able to communicate with each other.

[0020] The input device 101 is, for example, a keyboard, a mouse, a touch panel, a physical button, etc. The display device 102 is, for example, a display, a display panel, etc. Note that the tensor learning device 10 does not necessarily have to include at least one of the input device 101 and the display device 102.

[0021] The external I / F 103 is an interface with an external device such as a recording medium 103a. Examples of the recording medium 103a include a CD (Compact Disc), a DVD (Digital Versatile Disk), an SD memory card (Secure Digital memory card), and a USB (Universal Serial Bus) memory card.

[0022] The communication I / F 104 is an interface for connecting to a communication network. The RAM 105 is a volatile semiconductor memory (storage device) that temporarily stores programs and data. The ROM 106 is a non-volatile semiconductor memory (storage device) that can store programs and data even when the power is turned off. The auxiliary storage device 107 is a non-volatile storage device such as a hard disk drive (HDD), a solid state drive (SSD), or a flash memory. The processor 108 is a variety of arithmetic devices such as a central processing unit (CPU) or a graphic processing unit (GPU).

[0023] 1 is an example, and the hardware configuration of the tensor learning device 10 is not limited to this. For example, the tensor learning device 10 may have multiple auxiliary storage devices 107 or multiple processors 108, may not have some of the hardware shown in the figure, or may have various hardware other than the hardware shown in the figure.

[0024] <Example of Functional Configuration of Tensor Learning Apparatus 10 During Learning> An example of the functional configuration of the tensor learning apparatus 10 during learning will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of the functional configuration of the tensor learning apparatus 10 during learning.

[0025] As shown in FIG. 2 , the tensor learning device 10 during learning includes an input unit 201, an embedding unit 202, a tensor transformation unit 203, a prediction unit 204, and a learning unit 205. These units are realized, for example, by a process in which one or more programs installed in the tensor learning device 10 are executed by the processor 108 or the like. The tensor learning device 10 during learning also includes a training dataset storage unit 206 and a model parameter storage unit 207. These storage units are realized, for example, by a storage area of ​​the auxiliary storage device 107 or the like. However, for example, at least one of the training dataset storage unit 206 and the model parameter storage unit 207 may be realized by a storage area of ​​a storage device (e.g., a storage device provided in a database server) communicatively connected to the tensor learning device 10.

[0026] The input unit 201 inputs a training dataset E stored in the training dataset storage unit 206. The input unit 201 also inputs a tensor T ｄ ∈E is randomly selected, and the tensor T ｄ Randomly select a subtensor X from the tensor T ｄ (i.e., the number of modes is less than the size of the tensor T ｄ The following, and the number of dimensions of each mode is tensor T ｄ In the following, we define the size of the subtensor X as N １ ×N ２ ×...×N Ｍ It should be noted that the size of the subtensor X may be different from the size of the target tensor to be analyzed during testing. Furthermore, the input unit 201 determines the size N １ ×N ２ ×...×N Ｍ A mask tensor S is randomly generated.

[0027] The embedding unit 202 uses the model parameters stored in the model parameter storage unit 207, and receives the subtensor X and the mask tensor S as input to create a first latent tensor Z that represents the features of the subtensor X and the mask tensor S. The first latent tensor Z is composed of elements that take real values, and its size is N １ ×N ２ ×...×N Ｍ ×H.

[0028] The tensor transformation unit 203 receives the first latent tensor Z as input and transforms the first latent tensor Z to create a second latent tensor Z' using the model parameters stored in the model parameter storage unit 207. In other words, the tensor transformation unit 203 transforms the first latent tensor Z into a second latent tensor Z' using the model parameters stored in the model parameter storage unit 207. The second latent tensor Z' is composed of elements that take real values, and its size is N １ ×N ２ ×...×N Ｍ ×H'. The second latent tensor Z' is created by transforming each element of the first latent tensor Z while taking into account the values ​​of the other elements.

[0029] The prediction unit 204 uses the model parameters stored in the model parameter storage unit 207 to input the second latent tensor Z' and predicts the tensor X' after missing values ​​are imputed. The tensor X' after missing values ​​are imputed is a tensor obtained by predicting and imputing values ​​of elements considered to be missing values ​​of the subtensor X (i.e., elements of the subtensor X that correspond to elements of the mask tensor S that take the value 0). The tensor X' after missing values ​​are imputed is made up of elements that take real values, and its size is the same as that of the subtensor X.

[0030] The learning unit 205 updates the model parameters stored in the model parameter storage unit 207 so as to improve the prediction accuracy by the prediction unit 204. That is, the learning unit 205 updates the model parameters using a known optimization method such as a gradient method so as to minimize the error between the subtensor X and the corresponding missing value-complemented tensor X′.

[0031] The learning dataset storage unit 206 stores the given learning dataset E={T １ , ..., T Ｄ} is stored.

[0032] The model parameter storage unit 207 stores unlearned model parameters. Hereinafter, a set of model parameters will be represented by θ, and θ will also be simply referred to as a “model parameter.”

[0033] <Learning Process> An example of the learning process according to this embodiment will be described with reference to Fig. 3. Fig. 3 is a flowchart showing an example of the learning process according to this embodiment.

[0034] The input unit 201 receives a learning dataset E={T １ , ..., T Ｄ} is input (step S101).

[0035] The input unit 201 receives the tensor T ｄ ∈E is selected at random (step S102).

[0036] The input unit 201 receives the tensor T selected in step S102. ｄ In the following, the size of the subtensor X is set to N １ ×N ２ ×...×N Ｍ It should be noted that the size of the subtensor X selected in this step may be different or the same for each repetition of steps S102 to S108. For simplicity, it is assumed below that the size of the subtensor X selected in this step is the same for each repetition.

[0037] The input unit 201 receives a subtensor of size N, which is the same size as the subtensor X selected in step S103. １ ×N ２ ×...×N ＭA mask tensor S is randomly generated (step S104). This mask tensor S indicates which element of the subtensor X selected in step S103 is to be treated as a missing value. １ , ..., n Ｍ ) = 0 １ , ..., n Ｍ ) can be treated as having a defect.

[0038] The embedding unit 202 uses the model parameters stored in the model parameter storage unit 207 to create a first latent tensor Z from the subtensor X selected in step S103 and the mask tensor S generated in step S104 (step S105). The embedding unit 202 may create the first latent tensor Z using any function. For example, when creating the first latent tensor Z using a linear function, the embedding unit 202 can create the first latent tensor Z using the following formula (1).

[0039] Here, concat ｍ (・,・) is the mth mode of combining two tensors, × ｍ is the m-mode product. １ is a 2×H linear transformation matrix. １ is included in the model parameter θ. This allows us to create a model with real-valued elements and a size of N １ ×N ２ ×...×N Ｍ A first latent tensor Z of x × H is created. This first latent tensor Z can be considered as a tensor in the feature space of the subtensor X and the mask tensor S. Note that since the mask tensor S does not exist in tasks other than missing value imputation, Z = X × Ｍ＋１ W １ The first latent tensor Z may be created by:

[0040] The tensor transformer 203 uses the model parameters stored in the model parameter storage unit 207 to create a second latent tensor Z' from the first latent tensor Z created in step S105 (step S106). At this time, the tensor transformer 203 creates the second latent tensor Z' by transforming each element of the first latent tensor Z while taking into account the values ​​of the other elements. The tensor transformer 203 may create the second latent tensor Z' using any function that can accept a tensor of any size as input. For example, when creating the second latent tensor Z' using a self-attention mechanism, the tensor transformer 203 can create the second latent tensor Z' by repeating the following steps 1 to 3 for each of m = 1, ..., M, and then performing the following step 4.

[0041] Step 1: The tensor transform unit 203 calculates the query tensor Q using the following equations (2) to (4). （ｍ） , Key Tensor K （ｍ） , value tensor V （ｍ） Create a.

[0042] Here, Z （ｍ－１） is the latent tensor created in step 3 below when m-1. （０） = Z (i.e., when m = 1, Z （ｍ－１） is the first latent tensor Z. ) Also, W ２ （ｍ） is a size H consisting of real-valued elements. （ｍ） ×H Ｑ （ｍ） , the linear transformation matrix of W ３ （ｍ） is a size H consisting of real-valued elements. （ｍ） ×H Ｑ （ｍ） The linear transformation matrix of W ４ （ｍ） is a size H consisting of real-valued elements. （ｍ） ×H Ｖ （ｍ） is a linear transformation matrix of H （１） When m≧2, Z （ｍ－１） If is calculated, H（ｍ） = H Ｖ （ｍ－１） , and Z according to the formula (7) described later. （ｍ－１） If is calculated, H （ｍ） = H Ｏ （ｍ－１） In addition, W ２ （ｍ） , W ３ （ｍ） , W ４ （ｍ） is included in the model parameter θ. This allows us to create a model of size N １ ×N ２ ×...×N Ｍ ×H Ｑ （ｍ） Query tensor Q （ｍ） and a size N consisting of real-valued elements １ ×N ２ ×...×N Ｍ ×H Ｑ （ｍ） Key Tensor K （ｍ） and a size N consisting of real-valued elements １ ×N ２ ×...×N Ｍ ×H Ｖ （ｍ） The value tensor V （ｍ） The following is obtained.

[0043] Step 2: The tensor transformation unit 203 transforms the query tensor Q （ｍ） and Key Tensor K （ｍ） Attention matrix A based on the similarity （ｍ） For example, when the inner product is used as the similarity, the tensor transform unit 203 creates an attention matrix A by the following equation (5): （ｍ） can be created.

[0044] Here, softmax(·) is the softmax function, and t is the symbol for transposition. Also, the following are the query tensor Q （ｍ） m-mode matrix of key tensor K （ｍ） represents the m-mode matrix of

[0045] Note that the query tensor Q （ｍ）The m-mode matrix of １ ...N ｍ－１ N ｍ＋１ ...N Ｍ H Ｑ （ｍ） ×N ｍ Similarly, the key tensor K （ｍ） The m-mode matrix of １ ...N ｍ－１ N ｍ＋１ ...N Ｍ H Ｑ （ｍ） ×N ｍ This results in a matrix of N ｍ ×N ｍ Attention matrix A （ｍ） is obtained.

[0046] Step 3: The tensor transformation unit 203 converts the value tensor V （ｍ） and attention matrix A （ｍ） The potential tensor Z （ｍ） That is, the tensor transform unit 203 creates a latent tensor Z （ｍ） Create a.

[0047] This allows us to create a matrix of size N consisting of real-valued elements. １ ×N ２ ×...×N Ｍ ×H Ｖ （ｍ） Tensor Z of （ｍ） is obtained.

[0048] The tensor transform unit 203 converts the latent tensor Z （ｍ） You may also create a

[0049] Here, mTA ｒ （ｍ） is the rth mTA （ｍ） Also, W ５ （ｍ） is an H consisting of real-valued elements. Ｖ （ｍ） R x H Ｏ （ｍ） is a linear transformation matrix of W ５ （ｍ）is included in the model parameter θ.

[0050] Step 4: The tensor transform unit 203 converts the latent tensor Z （Ｍ） Let Z′ be the second latent tensor. （Ｍ） is calculated, H'=H Ｖ （Ｍ） , and Z according to the above formula (7) （Ｍ） is calculated, H'=H Ｏ （Ｍ） This results in a second latent tensor Z′, in which each element of the first latent tensor Z is transformed while taking into account the values ​​of the other elements (specifically, the value of each element of the first latent tensor Z is weighted so that the element that contributes more to the prediction of the missing value has a higher value).

[0051] In addition, W ２ （ｍ） , W ３ （ｍ） , W ４ （ｍ） , W ５ （ｍ） Each of these may be a common value for all modes, or may be a different value for each mode.

[0052] The prediction unit 204 predicts the missing value-complemented tensor X' from the second latent tensor Z' using the model parameters stored in the model parameter storage unit 207 (step S107). The prediction unit 204 may predict the missing value-complemented tensor X' from the second latent tensor Z' using any model as the prediction model. For example, when a linear regression model is used as the prediction model, the prediction unit 204 can predict the missing value-complemented tensor X' using the following equation (8).

[0053] where 1 is the size N where all elements are 1. １ ×N ２ ×...×N Ｍ × 1 tensor (i.e., a tensor corresponding to the bias term). ６ is a (H'+1) × 1 linear transformation matrix consisting of elements that take real values. Ｍ＋１ (Z', 1) is a real-valued size N１ ×N ２ ×...×N Ｍ Note that W is a tensor of size H' × (H' + 1). ６ is included in the model parameter θ. This allows us to create a model of size N １ ×N ２ ×...×N Ｍ After imputing the missing values, the tensor X' is obtained.

[0054] The learning unit 205 updates the model parameters θ stored in the model parameter storage unit 207 so as to improve the prediction accuracy of the tensor X′ after missing values ​​are complemented (step S108). That is, the learning unit 205 uses the error between the subtensor X and the tensor X′ after missing values ​​are complemented to update the model parameters θ using a known optimization method such as a gradient method so as to minimize the error. Various errors can be used as the error between the subtensor X and the tensor X′ after missing values ​​are complemented. For example, a tensor of size N where all elements are 1 can be used. １ ×N ２ ×...×N Ｍ Let I be the tensor of the above equation. Then, we take the square of the Frobenius norm of the tensor obtained by multiplying I-S by X-X' (Hadamard product) element by element. This corresponds to the sum of squares of the difference between the missing value and its correct answer.

[0055] The learning unit 205 determines whether to terminate the learning process (step S109). For example, the learning unit 205 may determine to terminate the learning process if a predetermined termination condition is met, and may determine not to terminate the learning process if a predetermined termination condition is not met. Examples of the predetermined termination condition include the number of repetitions of steps S102 to S108 exceeding a predetermined number of times, the update of the model parameter θ converging, and the objective function (error) converging. Note that if it is determined not to terminate the learning process, the process returns to step S102.

[0056] By executing the above learning process, the model parameter θ stored in the model parameter storage unit 207 is updated, and the updated model parameter θ is obtained as the trained model parameter θ. Note that a model configured by the embedding unit 202, the tensor transformation unit 203, and the prediction unit 204 that use the trained model parameter θ is the trained model.

[0057] <Modifications> Modifications of the above learning process will be described below. Note that the following modifications can also be applied to the test process described later in the same way.

[0058] Variation 1 When creating the second latent tensor Z′ in step S106, in procedure 1 above, the following equations (2′) to (4′) may be used instead of the above equations (2) to (4).

[0059] However, for all m, H （ｍ） =H.

[0060] In this case, for example, after executing the above steps 1 to 3 in parallel for each m = 1, ..., M, the latent tensor Z （１） , ..., Z （Ｍ） The average tensor of the second latent tensor Z′ may be used.

[0061] Modification 2 When creating the second latent tensor Z′ in step S106, the latent tensor Z （Ｍ） may be vectorized (fiberized) in the Mth mode, and these vectors may be converted by a neural network to obtain the second latent tensor Z'. Any neural network can be used as the neural network, and for example, a feedforward network configured with a fully connected layer can be used. Note that the parameters of the neural network are included in the model parameter θ.

[0062] Modification 3 When creating the second latent tensor Z′ in step S106, for each of m=1, . . . , M, the latent tensor Z′ obtained in step 3 above is （ｍ）can be transformed by a neural network. That is, the latent tensor Z （ｍ） are vectorized (fiberized) in the mth mode, and these vectors are transformed by a neural network to form the latent tensor Z （ｍ） Any neural network can be used as the neural network, for example, a feedforward network configured with a fully connected layer can be used. Note that the parameters of the neural network are included in the model parameter θ.

[0063] [Test Time] The tensor learning device 10 during testing will be described below. Note that the tensor learning device 10 during testing may also be called, for example, an "inference device" or a "prediction device." Furthermore, although the following assumes that the tensor learning device 10 during testing is implemented by the same device or system as the tensor learning device 10 during learning, the tensor learning device 10 during testing and the tensor learning device 10 during learning may also be implemented by different devices or systems.

[0064] <Example of Hardware Configuration of Tensor Learning Apparatus 10 During Testing> The tensor learning apparatus 10 during testing can be realized by the same hardware configuration as the tensor learning apparatus 10 during learning, and therefore a description thereof will be omitted.

[0065] <Example of Functional Configuration of Tensor Learning Apparatus 10 During Testing> An example of the functional configuration of the tensor learning apparatus 10 during testing will be described with reference to Fig. 4. Fig. 4 is a diagram illustrating an example of the functional configuration of the tensor learning apparatus 10 during testing.

[0066] As shown in FIG. 4 , the tensor learning device 10 at the time of testing includes an input unit 201, an embedding unit 202, a tensor transformation unit 203, a prediction unit 204, and an output unit 208. Each of these units is realized, for example, by a process in which one or more programs installed in the tensor learning device 10 are executed by the processor 108 or the like. The tensor learning device 10 at the time of testing also includes a test data storage unit 209 and a trained model parameter storage unit 210. Each of these storage units is realized, for example, by a storage area such as the auxiliary storage device 107. However, for example, at least one of the test data storage unit 209 and the trained model parameter storage unit 210 may be realized by a storage area of ​​a storage device (e.g., a storage device provided in a database server) communicatively connected to the tensor learning device 10.

[0067] The input unit 201 inputs test data stored in the test data storage unit 209. The test data includes a target tensor X and a corresponding mask tensor S. Hereinafter, the test data will be represented as (X, S).

[0068] The embedding unit 202 uses the learned model parameters stored in the learned model parameter storage unit 210, and takes the target tensor X and the mask tensor S as input to create a first latent tensor Z that represents the characteristics of the target tensor X and the mask tensor S.

[0069] The tensor transformation unit 203 uses the learned model parameters stored in the learned model parameter storage unit 210 to input the first latent tensor Z and create a second latent tensor Z′ by transforming the first latent tensor Z.

[0070] The prediction unit 204 uses the trained model parameters stored in the trained model parameter storage unit 210 to predict the missing value-complemented tensor X′ using the second latent tensor Z′ as input.

[0071] The output unit 208 outputs the missing value-complemented tensor X′ to a predetermined output destination. Any output destination can be determined as the predetermined output destination, and examples of the predetermined output destination include the display device 102 such as a display, a storage area such as the auxiliary storage device 107, another program, another device connected for communication, etc.

[0072] The test data storage unit 209 stores the given test data (X, S).

[0073] The trained model parameter storage unit 210 stores trained model parameters θ trained during training.

[0074] <Test Processing> An example of test processing according to this embodiment will be described with reference to Fig. 5. Fig. 5 is a flowchart showing an example of test processing according to this embodiment.

[0075] The input unit 201 inputs test data (X, S) stored in the test data storage unit 209 (step S201).

[0076] The embedding unit 202 uses the trained model parameters stored in the trained model parameter storage unit 210 to create a first latent tensor Z from the target tensor X and mask tensor S included in the test data (X, S) input in step S201 (step S202). Note that the embedding unit 202 uses the trained model parameters instead of the model parameters to create the first latent tensor Z by processing similar to step S105 in FIG. 3 .

[0077] The tensor transform unit 203 creates a second latent tensor Z' from the first latent tensor Z created in step S202 above, using the trained model parameters stored in the trained model parameter storage unit 210 (step S203). Note that the tensor transform unit 203 creates the second latent tensor Z' by processing similar to step S106 in FIG. 3, using the trained model parameters instead of the model parameters.

[0078] The prediction unit 204 predicts the missing value-complemented tensor X' from the second latent tensor Z' using the trained model parameters stored in the trained model parameter storage unit 210 (step S204). Note that the prediction unit 204 predicts the missing value-complemented tensor X' by using the trained model parameters instead of the model parameters, through processing similar to that of step S107 in FIG. 3.

[0079] The output unit 208 outputs the missing value-complemented tensor X′ predicted in step S204 to a predetermined output destination (step S205).

[0080] By executing the above test process, a missing value-complemented tensor X′ in which missing values ​​are predicted and complemented with high accuracy is obtained.

[0081] [Evaluation] To evaluate the tensor learning device 10 according to this embodiment, model parameters θ were learned using training data E consisting of four tensors, and then missing value imputation was performed on a target tensor X whose missing values ​​were known. The comparison methods used were a neural network-based method (NN), tensor train decomposition (TTD), CP decomposition (CPD), and Tucker decomposition (TD). The evaluation index used was the error between the tensor X′ after missing value imputation and the target tensor X whose missing values ​​were imputed with known missing values. The evaluation results are shown in Table 1 below.

[0082] The tensor learning device 10 according to this embodiment is a proposed method.

[0083] As shown in Table 1 above, it was confirmed that the tensor learning device 10 according to this embodiment can achieve a lower error than the comparative methods.

[0084] As described above, the tensor learning device 10 according to the present embodiment can learn a model that can analyze tensors of a target task with high accuracy using multiple tensors of different sizes as training data. Therefore, even if only a small amount of observation data expressed as tensors is available for the target task, the tensor of the target task can be analyzed with high accuracy.

[0085] The present invention is not limited to the above-described specifically disclosed embodiments, and various modifications, changes, and combinations with known technologies are possible without departing from the scope of the claims.

[0086] 10 Tensor learning device 101 Input device 102 Display device 103 External I / F 103a Recording medium 104 Communication I / F 105 RAM 106 ROM 107 Auxiliary storage device 108 Processor 109 Bus 201 Input unit 202 Embedding unit 203 Tensor transformation unit 204 Prediction unit 205 Learning unit 206 Learning dataset storage unit 207 Model parameter storage unit 208 Output unit 209 Test data storage unit 210 Learned model parameter storage unit

Claims

1. A learning device comprising: a selection unit that selects a subtensor from a given tensor; a first latent tensor creation unit that creates a first latent tensor that represents characteristics of the subtensor based on first model parameters and the subtensor; a second latent tensor creation unit that creates a second latent tensor based on second model parameters and the first latent tensor by transforming each element of the first latent tensor while taking into account values ​​of other elements; a prediction unit that predicts a prediction tensor that represents a predicted value of a predetermined task based on third model parameters and the second latent tensor; and an update unit that updates the first model parameters, the second model parameters, and the third model parameters so as to increase the prediction accuracy of the prediction tensor.

2. The learning device of claim 1, wherein the task is missing value completion of a tensor, and the learning device further comprises a mask tensor generation unit that randomly generates a mask tensor that represents elements of the subtensor that are to be treated as missing values, the first latent tensor creation unit creates a first latent tensor that represents features of the subtensor and the mask tensor based on the first model parameters, the subtensor, and the mask tensor, and the update unit updates the first model parameters, the second model parameters, and the third model parameters so as to minimize an error between the subtensor and the predicted tensor.

3. The learning device described in claim 1 or 2, wherein the second latent tensor creation unit creates the second latent tensor using a self-attention mechanism based on the second model parameters and the first latent tensor.

4. The learning device described in claim 3, wherein the second latent tensor creation unit creates the second latent tensor by repeating the self-attention mechanism and fiber-by-fiber transformation of the latent tensor created by the self-attention mechanism using a neural network for each mode of the first latent tensor.

5. An inference device having: a first latent tensor creation unit that creates a first latent tensor representing the characteristics of a tensor based on trained first model parameters and a given tensor; a second latent tensor creation unit that creates a second latent tensor based on trained second model parameters and the first latent tensor by converting each element of the first latent tensor while taking into account the values ​​of the other elements; and a prediction unit that predicts a prediction tensor representing a predicted value of a specified task based on trained third model parameters and the second latent tensor.

6. A learning method performed by a computer, comprising: a selection step of selecting a subtensor from a given tensor; a first latent tensor creation step of creating a first latent tensor representing the characteristics of the subtensor based on first model parameters and the subtensor; a second latent tensor creation step of creating a second latent tensor based on second model parameters and the first latent tensor by transforming each element of the first latent tensor while taking into account the values ​​of the other elements; a prediction step of predicting a prediction tensor representing a predicted value of a predetermined task based on third model parameters and the second latent tensor; and an update step of updating the first model parameters, the second model parameters, and the third model parameters so as to increase the prediction accuracy of the prediction tensor.

7. An inference method performed by a computer, comprising: a first latent tensor creation procedure for creating a first latent tensor representing the characteristics of a given tensor based on trained first model parameters and a given tensor; a second latent tensor creation procedure for creating a second latent tensor based on trained second model parameters and the first latent tensor by transforming each element of the first latent tensor while taking into account the values ​​of the other elements; and a prediction procedure for predicting a prediction tensor representing a predicted value of a specified task based on trained third model parameters and the second latent tensor.

8. A program that causes a computer to function as the learning device according to claim 1 or the inference device according to claim 5.

Images