Information processing device, information processing method, and information processing program

Abstract

In the present invention, a reading unit reads a first model and task vectors of a plurality of models different from the first model. A task arithmetic unit performs task arithmetic by using the first model and a result of converting the task vectors according to the first model.

Description

Information processing device, information processing method, and information processing program

[0001] The present invention relates to an information processing device, an information processing method, and an information processing program for training a machine learning model.

[0002] Recently, models specialized for various tasks have been created by fine-tuning the base model. In addition, a technique called task arithmetic is known, which allows one base model to learn multiple task knowledge or forget task knowledge by adding or subtracting task vectors based on multiple models specialized for different tasks to or from another base model (see, for example, Non-Patent Document 1).

[0003] Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, Ali Farhadi, Editing Models with Task Arithmetic, [online], [Retrieved March 28, 2024], Internet <URL: https: / / arxiv.org / abs / 2212.04089>

[0004] However, conventional techniques have a problem in that task arithmetic cannot be performed between different models. For example, if the parameters of two base models do not correspond one-to-one to each other, it is difficult to correctly perform addition of the two base models.

[0005] Therefore, an object of the present invention is to perform task arithmetic between different models.

[0006] In order to solve the above-mentioned problems, the information processing device of the present invention is characterized by having a reading unit that reads a first model and task vectors of multiple models different from the first model, and a task arithmetic unit that performs task arithmetic using the first model and the result of converting the task vector according to the first model.

[0007] According to the present invention, task arithmetic can be performed between different models.

[0008] FIG. 1 is a conceptual diagram of a permutation transform. FIG. 2 is a conceptual diagram of a learning process when the step size is infinitesimal. FIG. 3 is a diagram illustrating a method for obtaining continuous information in a learning process. FIG. 4 is a diagram illustrating an example of a configuration of an information processing apparatus according to a first embodiment. FIG. 5 is a flowchart illustrating an outline of a learning transfer process executed by the information processing apparatus according to the first embodiment. FIG. 6 is a diagram illustrating an example of an algorithm. FIG. 7 is a flowchart illustrating an algorithm. FIG. 8 is a diagram illustrating an outline of task arithmetic. FIG. 9 is a conceptual diagram of task arithmetic according to a first embodiment. FIG. 10 is a flowchart illustrating a process of task arithmetic according to the first embodiment. FIG. 11 is a conceptual diagram of task arithmetic according to a second embodiment. FIG. 12 is a flowchart illustrating a process of task arithmetic according to the second embodiment. FIG. 13 is a diagram illustrating an example of a configuration of a computer that executes an information processing program.

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a description will be given of an embodiment of the present invention with reference to the drawings, but the present invention is not limited to the embodiment.

[0010] First Embodiment Overview First, an overview of an information processing apparatus according to a first embodiment will be described with reference to Fig. 1. The information processing apparatus performs learning of a machine learning model using, for example, a DNN (Deep Neural Network).

[0011] The information processing device performs learning transfer between models and task arithmetic. For example, the information processing device performs learning transfer to execute task arithmetic. However, the information processing device may obtain the results of learning transfer from another device or the like and perform task arithmetic using the obtained results. Therefore, the information processing device does not need to be equipped with a learning transfer function.

[0012] [Overview of Model Learning and Learning Transfer] First, we will explain model learning and learning transfer. For example, the initial parameter θ1 of the DNN t=0 On the other hand, the learning process of dataset D by SGD (Stochastic Gradient Descent) (θ1 t=0,…,θ1 t=T ) is given. Here, the information processing device calculates a new initial parameter (initial parameter value) θ2 t=0 When learning a dataset D, the learning process of the given θ1 (θ1 t=0 ,…,θ1 t=T ) is permuted to obtain the learning process of θ2 (θ2 t=0 ,…,θ2 t=T ) is obtained. Then, the information processing device uses the obtained permutation transformation π to obtain the initial parameter θ2 t=0 The learning process of the dataset D for (θ2 t=0 ,…,θ2 t=T ) is calculated. This gives (θ2 t=1 ,…,θ2 t=T ) to reduce the calculation time.

[0013] In other words, the initial parameter θ1 t=0 The learning process of the dataset D for (θ1 t=0 ,…,θ1 t=T ) is considered to contain information useful for learning the dataset D. Therefore, the information processing device utilizes the above information to generate new initial parameters θ2 t=0 This allows the information processing device to efficiently learn the data set D for the initial parameter θ2 t=0 The process of learning the dataset D (θ2 t=1 ,…,θ2 t=T ) to reduce the calculation time.

[0014] [Permutation Symmetry of DNN] Next, the basic concept of learning performed by the information processing device of this embodiment will be described. As a general property of DNN, NN f(x; W1, ..., W L ) has degrees of freedom of permutation symmetry. For example, any permutation matrix σ1,…,σ L-1 The relationship shown in the following equation (1) holds for the DNN parameters θ = (W1, ..., W L ) for the permutation transformation π=(σ1,…,σ L-1The result of applying (the right side of equation (1)) is written as πθ, and this is called a permutation transformation of θ by π.

[0015]

[0016] The permutation matrix is, for example, a permutation matrix as shown in Equation 2. By using the permutation matrix as described above, the information processing device can handle permutations of intermediate dimensions in the NN.

[0017]

[0018] In this embodiment, it is assumed that the learning process of the parameters θ1 and θ2 approximately matches with a certain permutation transformation. In other words, the permutation transformation π=(σ1,...,σ2) satisfies the relationship shown in the following equation (3) for any t. L-1 ) exists.

[0019]

[0020] If a value of π that satisfies the relationship shown in equation (3) is found, the learning process for parameter θ2 can be obtained inductively from the learning process for parameter θ1. In other words, the learning process for parameter θ2 can be obtained by the following equation (4).

[0021]

[0022] Therefore, the information processing device of this embodiment obtains a permutation transformation π that satisfies the relationship of the above formula (3) for any t.

[0023] [Formulation of the Proposed Technology] Based on the above discussion, the desired permutation transformation π can be calculated by solving the optimization problem P T can be considered as a solution to

[0024] Given: θ1 t=0 ,…,θ1 t=T (learning process of θ1) and θ2 t=0 (initial value of θ2)

[0025]

[0026] In addition, || ||2 in equation (5) 2 is the vector L 2 It is the norm.

[0027] However, θ2 in equation (5) t is (θ2 t=0 Since the parameters are unknown (except for T is an incomplete formulation. Therefore, the information processing device calculates each difference θ i t -θ i t-1 The optimization problem P is approximated by the gradient of the loss function L of the dataset D. T The gradient is the first-order gradient -∇θ i t-1 L, and the quadratic gradient -∇ 2 θ i t-1 L. This solves the optimization problem P T can finally be formulated as the following equation (6).

[0028]

[0029] The optimization problem P shown in Equation (6) T is a nested structure (θ 2,π t also contains the unknown variable π), it is difficult to solve directly and requires some ingenuity.

[0030] Therefore, the information processing device solves the above optimization problem P T Instead of T Solve them in order and inductively find the solutions π1, ..., π T-1 , π T = :π where each optimization problem P´ t is P´ t-1 Solution of π t-1 is inductively defined as in the following equation (7).

[0031]

[0032] Each optimization problem P' in the above equation (7) tis a simple quadratic optimization problem without nesting (where clause), and permutation transformation does not change the norm of the parameter vector. t can be reduced to a linear optimization problem. Therefore, the information processing device can use an existing linear optimization solver to find the solution π t Therefore, the information processing device can calculate the solutions π1, ..., π T-1 , π T By calculating in order, the final value obtained is π T is the optimization problem P T For example, the information processing device can solve the optimization problem using the Hungarian method and optimal transportation.

[0033] Here, in the learning process, the model parameters are updated step by step. At each step, a portion of data from dataset D is input to the model, and the parameters are updated.

[0034] The first-order gradient is obtained as a value corresponding to each step, and can therefore be considered as discrete information of the learning process. In contrast, the second-order gradient can be considered as continuous information of the learning process. The information processing device of this embodiment determines the permutation transform π using continuous information of the learning process. This allows the information processing device to determine the permutation transform π with higher accuracy than, for example, when using only discrete information. As a result, the accuracy of learning transfer is improved.

[0035] Note that the continuous information of the learning process is not limited to a quadratic gradient. For example, the continuous information of the learning process may be a higher-order gradient of quadratic or higher order. The quadratic gradient in this embodiment may be replaced with a higher-order gradient as appropriate.

[0036] The information processing device calculates the quadratic gradient based on the learning process. The information processing device can approximately calculate the quadratic gradient by assuming that the step size is infinitesimal.

[0037] 2 is a conceptual diagram of the learning process when the step size is infinitesimal. Here, the information processing device learns the model P1 by using a data set.ｔ＝2 Applying the permutation transformation π to the learning process that changes to , and then training model P2 using the same dataset, we obtain model P2. t=2 The following is required.

[0038] As shown in Figure 2, in the first step of learning, model P1 t=1 In addition, in the first step of learning, model P2 changes to model P2 t=1 In addition, in the second step of learning, model P1 t=1 Model P1 t=2 In the second step of learning, model P2 t=1 Model P2 t=2 where the learning rate α corresponds to the step size and is infinitesimal (α→0).

[0039] If the learning process of model P1 is G1 and the learning process of model P2 is G2, the information processing device approximates the first step of the learning process as shown in Figure 3(a). Figure 3 is a diagram explaining a method for obtaining continuous information on the learning process. The information processing device obtains the linear gradient of the first step of the learning process of model P1 and the linear gradient of the first step of the learning process of model P2 using (a).

[0040] Furthermore, the information processing device approximates the parameters at the second step (i.e., the step size is infinitesimal) in the direction of the obtained linear gradient as shown in (b) or (b'). Using (b) and (b'), the information processing device obtains the quadratic gradient of the first step of the learning process of model P1 and the linear gradient of the first step of the learning process of model P2.

[0041] The information processing device may also obtain the approximation of the first-order and second-order gradients shown in FIG. 3 between the first-order and second-order derivatives of the Gradient Flow solution at t=0.

[0042] [Configuration Example] Next, a configuration example of the information processing device 10 will be described with reference to Fig. 4. The information processing device 10 includes, for example, an input / output unit 11, a storage unit 12, and a control unit 13.

[0043] The input / output unit 11 is an interface that controls the input and output of various data. For example, the input / output unit 11 controls the learning process of the parameter θ1 of the model 1 (θ1 t=0 ,…,θ1 t=T ), the initial value of the parameter θ2 of model 2 0 , dataset D, etc. are accepted as input.

[0044] The storage unit 12 stores data, programs, etc. that are referenced when the control unit 13 executes various processes. The storage unit 12 is realized by a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk.

[0045] For example, the memory unit 12 stores the learning process of the parameter θ1 (θ1 t=0 ,…,θ1 t=T ), the initial value of parameter θ2 t=0 , data set D, etc. The learning process of parameter θ1 (θ1 t=0 ,…,θ1 t=T ) is the initial value of the parameter θ1 of Model 1 (θ1 t=0 ) and dataset D.

[0046] The control unit 13 is responsible for overall control of the information processing device 10. The functions of the control unit 13 are realized, for example, by a CPU (Central Processing Unit) executing a program stored in the storage unit 12.

[0047] The control unit 13 includes, for example, an input receiving unit 131, a reading unit 132, a calculation unit 133, an output processing unit 134, and a task arithmetic unit 136. An information generating unit 135 indicated by a dashed line may or may not be provided, and the cases in which it is provided will be described later.

[0048] The input receiving unit 131 is a learning process of the parameter θ1 (θ1 t=0 ,…,θ1 t=T ), the initial value of parameter θ2 t=0, the input of the data set D is accepted and stored in the storage unit 12 .

[0049] The reading unit 132 reads the learning process of the parameter θ1 (θ1 t=0 ,…,θ1 t=T ), the initial value of parameter θ2 t=0 Load.

[0050] The calculation unit 133 performs the learning process of the parameter θ1 of the model 1 (θ1 t=0 ,…,θ1 t=T ) and the initial parameter value θ2 of Model 2 0 and the learning process of the parameter θ2 using the data set D (θ2 t=0 ,…,θ2 t=T ) to find the permutation transformation that minimizes the distance.

[0051] Here, the calculation unit 133 performs the learning process of the parameter θ1 (θ1 t=0 ,…,θ1 t=T ), and the learning process of the parameter θ2 (θ2 t=0 ,…,θ2 t=T ) each difference θ i t -θ i t-1 is the gradient value (-∇θ i t-1 L) to find a permutation transformation that minimizes the distance (optimization problem P T (Solve).

[0052] In addition, the calculation unit 133 performs a learning process of the parameter θ1 (θ1 t=0 ,…,θ1 t=T ) and the learning process of the parameter θ2 (θ2 t=0 ,…,θ2 t=T ) is executed in the order of the learning process.

[0053] For example, when the number of learning processes of the parameter θ is T, the calculation unit 133 compares and minimizes partial learning processes up to the tth one, where t≦T, instead of simultaneously comparing all processes and minimizing the distance (the distance of the entire learning process). t is a permutation transformation π that minimizes the distance of the partial learning process up to the t-1th iteration. t-1 The calculation unit 133 repeats this process inductively from t=1 to t=T, and finally obtains the permutation transformation π T By this, the distance of the whole learning process is minimized.

[0054] Then, the calculation unit 133 calculates the learning process of the parameter θ2 (θ2 t=0 ,…,θ2 t=T )

[0055] Furthermore, the calculation unit 133 can use continuous information of the learning process when determining the permutation transformation. That is, the calculation unit 133 determines a transformation method for the first learning process so as to reduce the degree of deviation between the continuous information of the first learning process obtained when learning the parameters of the first model using a dataset and the continuous information of the second learning process of the second model obtained by transforming the first learning process. The transformation method is, for example, permutation transformation π. The degree of deviation is, for example, the distance between the learning processes.

[0056] The continuous information of the learning process is, for example, a quadratic gradient. The calculation unit 133 can find a transformation method for the first learning process that simultaneously minimizes the distance between the linear gradient of the first learning process and the linear gradient of the second learning process, and the distance between the quadratic gradient of the first learning process and the quadratic gradient of the second learning process.

[0057] Furthermore, the calculation unit 133 can approximately calculate the quadratic gradient by assuming an infinitesimal step size. That is, the calculation unit 133 approximates the linear gradient of the first model and the linear gradient of the second model in the second step, which is the next step after the first step when the step size is infinitesimal, based on the linear gradient of the first model and the linear gradient of the second model in the first step of the first learning process and the second learning process, and then calculates the quadratic gradient of the first model and the quadratic gradient of the second model in the first step based on the approximation result. For example, the first step and the second step correspond to the first step and the second step in FIG. 2, respectively.

[0058] The output processing unit 134 performs the learning process (θ2 t=0 ,…,θ2 t=T ) the value of the parameter of the final process (θ2 t=T ) is output.

[0059] [Example of Processing Procedure for Learning Transfer] Next, an example of processing procedure for learning transfer executed by the information processing device 10 will be described with reference to FIG. 5 .

[0060] First, the reading unit 132 of the information processing device 10 reads the learning process of Model 1 (θ t=0 ,…,θ1 t=T ) (S1). The reading unit 132 also reads the parameter initial value θ2 of model 2. t=0 Next, the calculation unit 133 calculates π (Permutation transformation) that satisfies the relationship shown in the above-mentioned formula (3) for any t (S3).

[0061] After S3, the calculation unit 133 calculates the learning process (θ t=0 ,…,θ2 t=T ) (S4). After that, the output processing unit 134 returns the value (θ t=T ) is output (S5).

[0062] [Example of Learning Transfer Algorithm] An example of an algorithm used when the information processing device executes the above-mentioned learning transfer process is shown in FIG. 6. In FIG. 6, θ i t=s θ i s Also, π t is a permutation transform. The algorithm shown in FIG. 6 will be explained using the flowchart shown in FIG.

[0063] First, the reading unit 132 of the information processing device 10 loads the learning process of Model 1 from the storage unit 12. For example, the reading unit 132 reads the learning process of Model 1 with length T (θ1 t=0 ,…,θ1 t=T ) of model 2 from the storage unit 12 (S11). t=0 is read (S12).

[0064] Next, the calculation unit 133 calculates a variable (θ t=1 ,…,θ2 t=T ) are all initialized to zero vectors (S13). Then, a variable π is prepared to store the "permutation transformation for exchanging the components of each intermediate layer of the NN" (S14). Furthermore, the calculation unit 133 initializes a loop index t to 0 (S15).

[0065] After S15, if t=T is not true (No in S16), the calculation unit 133 sets the variable g1 1 ,…,g1 t , g2 1 ,…,g2 t , h1 1 ,…,h1 t , and h2 1 ,…,h2 t (S17), and initialize the loop index s to 0 (S18). After that, if s=t is not true (No in S19), the calculation unit 133 calculates the mini-batch gradient (S20). For example, the calculation unit 133 calculates g1 s parameter θ1 s-1 Substituting the linear gradient of the mini-batch in g2 s parameter θ2 s-1Substitute the linear gradient of the mini-batch in h1 s parameter θ1 s-1 Substituting the quadratic gradient of the mini-batch in h2 s parameter θ2 s-1 Then, the calculation unit 133 adds 1 to s (S21) and returns to S19.

[0066] Here, we will explain how to calculate the quadratic gradient of the mini-batch in S20. The calculation unit 133 calculates the Hessian-vector product H i However, P i is the parameter of model i. Also, L D is the loss function of the dataset D. Also, G i is the learning process of model i.

[0067]

[0068] In this case, if the right-hand side of equation (8) is calculated separately for each model, the computational complexity becomes O((number of parameters) 2 ), which becomes enormous. Therefore, the calculation unit 133 introduces α, which represents a sufficiently small positive real number (infinitesimal), and calculates equation (8) approximately. First, the calculation unit 133 calculates equations (9) and (10) by back-propagation, and then calculates equation (11).

[0069]

[0070]

[0071]

[0072] As a result, the calculation unit 133 calculates H i H approximated as i ´. H i and H i ´ are considered to be the same.

[0073] Furthermore, the calculation unit 133 calculates H i Instead of ´, H i ' may be calculated approximately as in equation (12).

[0074]

[0075] On the other hand, if s=t in S19 (Yes in S19), the calculation unit 133 performs optimization of the subproblem (S22). i is the aforementioned H i ´ or H i That is, the calculation unit 133 solves the permutation transformation π that minimizes the sum of squares of the L2 distances of the mini-batch gradients shown in the following equation (13) using a quadratic optimization or linear optimization solver, and substitutes the result for the variable π.

[0076]

[0077] After S22, the calculation unit 133 initializes the loop index s to 0 (S23). If s=t is not satisfied (No in S24), the calculation unit 133 updates the learning process of model 2 (S25). That is, the calculation unit 133 updates the parameter θ s θ2 s-1 +π(θ1 s -θ1 s-1 After that, the calculation unit 133 adds 1 to s (S26) and returns to S24.

[0078] On the other hand, if s=t in S24 (Yes in S24), the calculation unit 133 adds 1 to t (S27) and returns to S16. In S16, if t=T (Yes in S16), the calculation unit 133 calculates the learning process of model 2 (θ ｔ=1 ,…,θ2 ｔ=T ) (S28). After that, the output processing unit 134 returns the learning process (θ ｔ=1 ,…,θ2 ｔ=T ) among the parameters of the final process (θ2 ｔ=T ) is output.

[0079] By executing the above process, the information processing device 10 obtains the learned parameters of model 2 (θ ｔ=T ) can be calculated.

[0080] [Modification of the First Embodiment Regarding Learning Transfer] The information processing device 10 stores in the storage unit 12 the learning process (θ1 t=0 ,…,θ1 t=T), instead of holding two parameters (e.g., parameter initial value θ1 t=0 , the parameter value θ1 in the final learning process t=T ) may be held.

[0081] In this case, the information processing device 10 includes an information generating unit 135 (see FIG. 4). The information generating unit 135 generates an initial value θ1 of the parameter θ1 of the model 1 stored in the storage unit 12. t=0 and the final process value of parameter θ1 t=T From the above, the intermediate learning process of Model 1 (θ1 t=1 ,…,θ1 t=T-1 For example, the information generating unit 135 linearly interpolates θ t By calculating the following equation (14), the learning process of model 1 with length T (θ t=0 ,…,θ1 t=T ) is generated.

[0082]

[0083] This allows the information processing device 10 to reduce the number of parameters θ1 stored in the storage unit 12, thereby reducing storage costs.

[0084] [Task Arithmetic] First, an overview of task arithmetic will be explained using Fig. 8. Fig. 8 is a diagram for explaining the overview of task arithmetic. Model P0 in Fig. 8 is a base model to which task vectors are added. Here, model P1 to model P n Task vector M1 corresponding to each of n base models up to T1 -P1,…, M1 Tn -P n Let us consider adding model P0 by task arithmetic. Note that such task arithmetic is performed by adding model P1 to model P n This is done when it is desired to use model P0 instead of model P1 through model P2. For example, model P0 is used to n It is assumed that the generalization performance and robustness performance are superior to those of the previous methods.

[0085] The task vector is the parameter of the model after task-specific learning (e.g., M1 T1 ) from the model before task-specific learning, i.e., the parameters of the base model (e.g., P1). Therefore, task arithmetic can be rephrased as addition and subtraction of vectors representing model parameters or learning processes. Note that model parameters may be simply referred to as the model.

[0086] Also, π1,...,π in FIG. n is the permutation transformation obtained in the learning transfer. The method for obtaining the permutation transformation between models is as described above. That is, the task arithmetic unit 136 performs task arithmetic using the result of permutation transformation of the task vector through learning transfer that minimizes the distance between the learning process of model P0 and the task vector, and the first model. The permutation transformation may also be obtained by a method other than the method of this embodiment.

[0087] The task arithmetic unit 136 can obtain the model M0 by adding each task vector to which the permutation transformation has been applied to the model P0. In this way, the task arithmetic unit 136 performs task arithmetic using the model P0 and the result of transforming the task vector according to the model P0.

[0088] (First Embodiment) A first embodiment of task arithmetic performed by the task arithmetic unit 136 will be described. As shown in Fig. 9, in the first embodiment, the task arithmetic unit 136 performs task arithmetic using permutation transformations obtained between model P0 and models P1 and P2. Fig. 9 is an image diagram of task arithmetic in the first embodiment. Note that n=2 in Fig. 9.

[0089] 10 is a flowchart showing the processing procedure of task arithmetic in the embodiment 1. As shown in FIG. 10, the reading unit 132 reads a base model P0 and a plurality of other base models P1, . . . , P n Task vectors V1,…,V n(S31). The task arithmetic unit 136 assigns 1 to i (S32).

[0090] Next, the task arithmetic unit 136 generates a model ∼P that assumes the result of the task arithmetic. i (Just above P) is model P0, weight α i , Permutation transformation π i , task vector V i After using the Permutation transformation π i (S33). The task arithmetic unit 136 calculates the permutation transformation π i can be calculated.

[0091] Next, if i=n is not satisfied (No in S34), the task calculation unit 136 adds 1 to i (S35), returns to S33, and repeats the process. On the other hand, if i=n is satisfied (Yes in S34), the output processing unit 134 calculates α corresponding to each of i=1 to i=n. i π i V i The value obtained by adding this to P0 is output (S36).

[0092] (Example 2) Example 2 of task arithmetic performed by the task arithmetic unit 136 will be described. As shown in Fig. 11, in Example 2, the task arithmetic unit 136 adds a task vector of another model to model P0, and then adds a task vector of yet another model to the result of the addition, repeating this process. Fig. 11 is an image diagram of task arithmetic in Example 2. Note that in Fig. 9, n=2.

[0093] 12 is a flowchart showing the processing procedure of task arithmetic in the second embodiment. As shown in FIG. 12, the reading unit 132 reads a model P0 and a plurality of other base models P1, . . . , P n Task vectors V1,…,V n (S41). The task arithmetic unit 136 assigns 1 to i (S42). The task arithmetic unit 136 also assigns P0 to ∼P0 (S43).

[0094] Next, the task arithmetic unit 136 calculates the model .about.P i Let P0 be the model and α be the weight. i, Permutation transformation π i , task vector V i After using the Permutation transformation π i (S44). The task arithmetic unit 136 calculates the permutation transformation π i can be calculated.

[0095] Here, the task arithmetic unit 136 adds α to ∼P0. i π i V i (S45). In this way, α is added to ~P0 every time the value of i increases. i π i V i is added up.

[0096] Next, if i=n is not true (No in S46), the task arithmetic unit 136 adds 1 to i (S47), and returns to S44 to repeat the process. On the other hand, if i=n is true (Yes in S46), the output processing unit 134 outputs ∼P0 (S48).

[0097] In this way, the task arithmetic unit 136 sets the model P0 to the initial value (S43), and then sequentially (S47) performs a process of updating the initial value for each task vector by adding the result of the conversion according to the initial value to the initial value (S45).

[0098] [Effects of Task Arithmetic in the First Embodiment] As described above, the task arithmetic unit 136 calculates π i The transformed task vector π1V1,…,π n V n Task arithmetic is performed by treating it as a pseudo task vector on P0. This enables task arithmetic between different underlying models.

[0099] Furthermore, in the first embodiment, multiple transformations are calculated independently for each base model, so when arithmetic is performed on transformed task vectors, the transformed task vectors may interfere with each other and cause adverse effects.

[0100] In contrast to this, in the second embodiment, the task arithmetic unit 136 performs task vector conversion and task arithmetic in a manner that prevents task vectors from interfering with each other.

[0101] For example, consider the case where n is 3 or greater and i is 3. In this case, in Example 1, α2π2V2 and α3π3V3 are simultaneously added to P0 (S36 in FIG. 10), which may cause V2 and V3 to interfere with each other, making it impossible to properly specialize the base models for their respective tasks.

[0102] On the other hand, in Example 2, the specialization of the base model to task V2 is first determined at i = 2, and then the base model is specialized to task V3 at i = 3, so it is expected that the base model will successfully specialize to both task V2 and task V3. Thus, according to Example 2, mutual interference and adverse effects can be avoided when performing arithmetic on task vectors.

[0103] [System Configuration, etc.] The components of each unit shown in the figure are conceptual functional units and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of each device is not limited to that shown, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc. Furthermore, all or any part of the processing functions performed by each device can be realized by a CPU and a program executed by the CPU, or can be realized as hardware using wired logic.

[0104] Furthermore, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using a known method.In addition, the information including the processing procedures, control procedures, specific names, various data and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified.

[0105] [Program] The information processing device 10 can be implemented by installing a program (information processing program) as package software or online software on a desired computer. For example, by executing the program on an information processing device, the information processing device can function as the information processing device 10. The information processing device referred to here includes mobile communication terminals such as smartphones, mobile phones, and PHS (Personal Handyphone Systems), as well as terminals such as PDAs (Personal Digital Assistants).

[0106] 13 is a diagram showing an example of a computer that executes an information processing program. The computer 1000 includes, for example, a memory 1010 and a CPU 1020. The computer 1000 also includes a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0107] The memory 1010 includes a read-only memory (ROM) 1011 and a random access memory (RAM) 1012. The ROM 1011 stores a boot program such as a basic input / output system (BIOS). The hard disk drive interface 1030 is connected to a hard disk drive 1090. The disk drive interface 1040 is connected to a disk drive 1100. A removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to a mouse 1110 and a keyboard 1120, for example. The video adapter 1060 is connected to a display 1130, for example.

[0108] The hard disk drive 1090 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. That is, the programs that define the processes executed by the information processing device 10 are implemented as program modules 1093 in which computer-executable code is written. The program modules 1093 are stored, for example, in the hard disk drive 1090. For example, the program modules 1093 for executing processes similar to those of the functional configuration of the information processing device 10 are stored in the hard disk drive 1090. Note that the hard disk drive 1090 may be replaced with an SSD (Solid State Drive).

[0109] Data used in the processing of the above-described embodiment is stored as program data 1094, for example, in the memory 1010 or the hard disk drive 1090. The CPU 1020 then reads the program module 1093 or the program data 1094 stored in the memory 1010 or the hard disk drive 1090 into the RAM 1012 as necessary and executes them.

[0110] The program module 1093 and program data 1094 may not necessarily be stored in the hard disk drive 1090, but may also be stored in a removable storage medium and read by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and program data 1094 may be stored in another computer connected via a network (such as a local area network (LAN) or a wide area network (WAN)). The program module 1093 and program data 1094 may then be read by the CPU 1020 from the other computer via the network interface 1070.

[0111] The following additional notes are provided regarding the above-described embodiments.

[0112] (Supplementary Item 1) An information processing device including a memory and at least one processor connected to the memory, wherein the processor reads a first model and task vectors of a plurality of models different from the first model, and performs task arithmetic using the first model and a result of converting the task vector in accordance with the first model. (Supplementary Item 2) The information processing device according to Supplementary Item 1, wherein the processor performs the conversion of the task vector and the task arithmetic in a way that does not cause the task vectors to interfere with each other. (Supplementary Item 3) The information processing device according to Supplementary Item 1, wherein the task arithmetic unit sets the first model to an initial value, and sequentially performs a process of updating the initial value for each of the task vectors by adding the result of converting the task vector in accordance with the initial value to the initial value. (Supplementary Item 4) The information processing device according to Supplementary Item 1, wherein the processor is configured such that the task arithmetic unit performs task arithmetic using the first model and a result of permutation transforming the task vector by learning transfer such that the distance between the learning process of the first model and the task vector is minimized. (Supplementary Item 5) A non-transitory storage medium storing a computer-executable program, wherein the program causes a computer to execute the following process: read a first model and task vectors of a plurality of models different from the first model, and perform task arithmetic using the first model and a result of transforming the task vector according to the first model.

[0113] REFERENCE SIGNS LIST 10 Information processing device 11 Input / output unit 12 Memory unit 13 Control unit 131 Input reception unit 132 Reading unit 133 Calculation unit 134 Output processing unit 135 Information generation unit 136 Task arithmetic unit

Claims

1. An information processing device comprising: a reading unit that reads a first model and task vectors of multiple models different from the first model; and a task arithmetic unit that performs task arithmetic using the first model and the result of converting the task vector according to the first model.

2. The information processing device according to claim 1, wherein said task arithmetic unit performs transformation and task arithmetic of said task vectors in a manner that prevents said task vectors from interfering with each other.

3. The information processing device according to claim 1, characterized in that the task arithmetic unit sets a first model to an initial value, and sequentially performs a process of updating the initial value for each of the task vectors by adding the result of conversion according to the initial value to the initial value.

4. The information processing device described in claim 1, characterized in that the task arithmetic unit performs task arithmetic using the result of permutation transformation of the task vector and the first model through learning transfer such that the distance between the learning process of the first model and the task vector is minimized.

5. An information processing method comprising: a reading step of reading a first model and task vectors of a plurality of models different from the first model; and a task arithmetic step of performing task arithmetic using the first model and the result of converting the task vector according to the first model.

6. An information processing program that causes a computer to execute: a read procedure for reading a first model and task vectors of multiple models different from the first model; and a task arithmetic procedure for performing task arithmetic using the first model and the result of converting the task vector according to the first model.

Images