Method, information processing device, and program for performing transfer learning while suppressing the occurrence of catastrophic forgetting

Abstract

To provide a technique for performing transfer learning while suppressing occurrence of catastrophic forgetting.SOLUTION: A method includes: in learning of a T task, adjusting a first weight coefficient based on a first gradient of the first weight coefficient determined by using a T learning set and a significance parameter of the first weight coefficient; and after the learning using the T learning set has been completed, determining, based on the T learning set, respective second gradients of a plurality of second weight coefficients including the first weight coefficient included in a first layer including the first weight coefficient in a plurality of layers, and calculating the importance parameter of the first weight coefficient to be used in learning of a T+1 task based on the respective second gradients of the plurality of second weight coefficients.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present disclosure relates to deep learning technology, and more particularly to technology for performing transfer learning while suppressing the occurrence of catastrophic forgetting. [Background technology]

[0002] For example, neural networks are used to perform various processes such as image recognition, document classification, behavior recognition, and behavior prediction. A neural network is configured by connecting multiple layers, and each layer includes one or more units. The units may also be referred to as neurons. Each unit in the first layer receives an input value of the neural network. Each unit in the remaining layers receives an output value of one or more units in the previous layer as an input value. Each unit has a weighting coefficient corresponding to each input value of the unit. Each unit multiplies each input value by the corresponding weighting coefficient and calculates an output value based on the sum of the values ​​multiplied by the weighting coefficients. The weighting coefficient corresponding to each input value of each unit may also be simply referred to as a parameter. However, in this disclosure, the term "weighting coefficient" is used. Each unit in layers other than the last layer outputs its output value to one or more units in the layer immediately following. The output value of the unit in the last layer becomes the output value of the neural network. Note that a branch may be provided after the unit in the last layer, outputting the output value of the unit in the last layer to each of multiple heads, and the output of each head may become the output value of the neural network. Each head may have one or more layers.

[0003] To use neural networks for various tasks, for example, training is performed. Training is the process of optimizing the weight coefficients of each unit for a given task using a set of training data corresponding to the task (hereinafter referred to as the training set). For example, for the task of recognizing objects A, B, and C from images, a training set containing many images of objects A, B, and C is prepared. Then, image data of the images in the training set is input sequentially into the neural network to obtain the output values ​​of the neural network (forward propagation). The errors of these output values ​​are evaluated based on a predetermined error function (also referred to as a loss function), and the error amount of each unit is calculated sequentially, starting with the unit in the last layer, using the so-called backpropagation method. The "gradient" of each weight coefficient of that unit is then calculated from the error amount of the unit. Each weight coefficient is then changed / adjusted based on the corresponding gradient (backward propagation). By repeating forward propagation and subsequent backpropagation using the training set, the weight coefficients of each unit are optimized for the task.

[0004] In deep learning, a phenomenon known as catastrophic forgetting is known. For example, in image recognition, if a neural network can distinguish between objects A, B, and C using a certain training set, and the network is trained with a different training set that includes images of objects D and E in order to further enable the network to recognize objects D and E, the accuracy of distinguishing between objects A, B, and C will deteriorate. This phenomenon is called catastrophic forgetting.

[0005] Non-Patent Document 1 (hereinafter referred to as HAT technology) discloses a technology for suppressing catastrophic forgetting. [Prior art documents] [Non-patent literature]

[0006] [Non-Patent Document 1] Joan Serra and Didac Suris and Marius Miron and Alexandros Karatzoglou.Overcoming Catastrophic Forgetting with Hard Attention to the Task. In Proc. of ICML,2018. Summary of the Invention [Problem to be solved by the invention]

[0007] Specifically, in HAT technology, when learning a new task (e.g., the task of identifying objects D and E), catastrophic forgetting is suppressed by blocking the updating of weighting coefficients that are important for previously learned tasks (e.g., the task of identifying objects A, B, and C).

[0008] Meanwhile, so-called transfer learning, which improves the performance of a newly learned task by transferring knowledge of a previously learned task to the newly learned task, has been attracting attention.

[0009] However, as disclosed in the HAT technology, blocking updates to weighting coefficients that are important for previously learned tasks can hinder transfer learning.

[0010] The present disclosure provides a technique for performing transfer learning while suppressing the occurrence of catastrophic forgetting. [Means for solving the problem]

[0011] According to one aspect of the present disclosure, there is provided a method executed by an information processing device for causing a neural network that has sequentially learned tasks from a first task to a T-1th task by sequentially using a first learning set to a T-1th learning set (T is an integer equal to or greater than 2) to learn a T-th task using the T-th learning set. The neural network includes multiple layers, each of which includes multiple units. Each of the multiple units has a weighting factor associated with each input of the unit and an importance parameter indicating the importance of the weighting factor. The method includes, during learning of the T task, adjusting the first weighting factor based on a first gradient of the first weighting factor determined using the T learning set and the importance parameter for the first weighting factor; and, after learning using the T learning set is completed, determining, based on the T learning set, second gradients of each of multiple second weighting factors including the first weighting factor that are included in a first layer of the multiple layers that includes the first weighting factor, and calculating the importance parameter for the first weighting factor to be used in learning the T+1 task based on the second gradients of each of the multiple second weighting factors. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram illustrating an example neural network used to explain the present disclosure. [Figure 2] Functional block diagram of one unit during forward propagation. [Figure 3] Functional block diagram of one unit during backward propagation. [Figure 4] FIG. 1 is a diagram showing a hardware configuration of an information processing apparatus. [Figure 5] FIG. 1 is a functional block diagram of an information processing device. [Figure 6] 10 is a flowchart of a learning process executed by a learning unit. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention as claimed, and not all combinations of features described in the embodiments are necessarily essential to the invention. Two or more of the features described in the embodiments may be arbitrarily combined. Furthermore, the same reference numerals are used for the same or similar components, and redundant explanations will be omitted.

[0014] FIG. 1 is a configuration diagram of a neural network used to explain this embodiment. FIG. 1 shows the configuration of the neural network when learning of a total of T-1 tasks, task #1 to task #T-1 (T is an integer equal to or greater than 2), is completed. The neural network has a layer section 100 and a head section 200. The head section 200 has a head provided corresponding to each task learned in the past, and therefore, in this example, has a total of T-1 heads, head #1 to head #T-1. Head #t (t is an integer from 1 to T-1) corresponds to task #t.

[0015] The layer section 100 has a total of L layers, from the first layer to the Lth layer (L is an integer equal to or greater than 3). In this embodiment, each layer has N units (N is an integer equal to or greater than 2). Hereinafter, the nth unit (n is an integer equal to or greater than 1) in the lth layer (l is an integer equal to or greater than 1) will be referred to as unit #ln. An input value of the neural network is input to each unit in the first layer. An output value of all units in the previous layer is input to each unit in the second layer to the Lth layer. The output value of each unit in the Lth layer is output to heads #1 to #T-1 of the head section 200, respectively. The configuration of heads #1 to #T-1 may be the same as that of the layer section 100. However, the number of layers and the number of units in each layer may differ from those of the layer section 100. The outputs of heads #1 to #T-1 become the outputs of the neural network.

[0016] Note that the neural network in FIG. 1 is an example, and the configuration of the neural network is not limited to that shown in FIG. 1. For example, the output of each unit in the first to L-1th layers may be input to some of the units in the next layer. The number of units in each layer does not have to be the same. Furthermore, although the neural network of this embodiment is provided with a head unit 200 having a head #t corresponding to each task, the neural network may not have a head unit 200.

[0017] As described above, the neural network in Fig. 1 is trained using (T-1) training sets. In the following description, the training set used for training task #t will be referred to as training set #t.

[0018] As a specific example that does not limit the invention, task #1 recognizes objects A1, A2, and A3 from an image, and task #2 recognizes objects B1 and B2, which are different from objects A1, A2, and A3, from the image. Similarly, tasks #3 to #T-1 recognize predetermined objects from images. Note that the processing performed by the neural network is not limited to image recognition, and can be various other processing such as document classification, behavior recognition, and behavior prediction.

[0019] Hereinafter, the present disclosure will be described using an example in which a new task #T is learned using a learning set #T. At the start of learning task #T, the weighting coefficients of the layer unit 100 and the head unit 200 are set to the values ​​at the time when learning task #T-1 is completed. In addition, for learning task #T, a head #T is added to the head unit 200. The initial values ​​of the weighting coefficients of the head #T at the start of learning task #T are set by any method.

[0020] Similar to normal learning, the learning of task #T involves inputting the data of learning set #T sequentially into the neural network to obtain the output value of the neural network (forward propagation), and updating the weight coefficient of each unit based on the output value of the neural network (backward propagation). Note that the operation of each unit in forward propagation and backward propagation is similar, so the following describes the operation of unit #ln. Note that the same applies to each unit in each head of head unit 200.

[0021] 2 is a functional block diagram for explaining the operation of unit #ln during forward propagation. In the neuron unit 101, input values ​​I l-1,1 ~I l-1,N The neuron unit 101 receives the input value I l-1,1 ~I l-1,N The corresponding weighting coefficient W l-1,1 ~W l-1,N and the input value I l-1,1 ~I l-1,N The output value O is obtained by multiplying each by the corresponding weighting coefficient and adding them together. l,n Calculate the output value O l,n are output to unit #(l+1)1 to unit #(l+1)N or head #1 to head #T.

[0022] 3 is a functional block diagram for explaining the operation of unit #ln during backward propagation. Error determination section 201 receives error amounts Δ l+1,1 ~Δ l+1,N The error determination unit 201 determines the error amount Δ l,n Calculate the error amount Δ of unit #ln l,n are output to units #(l-1)1 to #(l-1)N. This processing is the same as that in the normal backpropagation method.

[0023] The gradient determination unit 202 determines the error amount Δ l,nThe weight coefficient W held by the neuron unit 101 based on l-1,n The gradient of G l-1,n This process is similar to that of the normal backpropagation algorithm.

[0024] The importance parameter storage unit 203 stores the weighting coefficients W l-1,n The importance parameter γ associated with Max l-1,n The importance parameter γ Max l-1,n is a value greater than or equal to 0 and less than 1, and the associated weighting factor W l-1,n The more important the task is in the past (task #1 to task #T-1), the larger the value. Max l-1,n The calculation of will be described later.

[0025] The multiplier 206 calculates the weighting coefficient W l-1,n The gradient of G l-1,n to(1-γ Max l-1,n ) is the weighting coefficient W l-1,n The updated gradient G´ l-1,n The weighting coefficient adjustment unit 205 outputs the gradient G' l-1,n Based on the weighting factor W l-1,n Update.

[0026] As mentioned above, the importance parameter γ Max l-1,n is the weighting factor W l-1,n The more important the task is in the past (task #1 to task #T-1), the larger the value. Max l-1,n ) is the weighting factor W l-1,n The more important is in the past task, the smaller the value. l-1,n The more important is in the past task, the more the updated gradient G´ l-1,n becomes smaller, and the weighting factor W l-1,n This suppresses the updating of the memory, which can prevent catastrophic forgetting.

[0027] Furthermore, in the HAT technique, the update of weighting factors that are important for previously learned tasks is blocked, so that each time a new task is learned, the proportion of weighting factors that are not updated to the total weighting factors increases, and therefore the expressive ability of the neural network for new tasks decreases. Max l-1,n is less than 1, so (1-γ Max The value of l-1,n) is greater than 0. Therefore, each weight coefficient is updated even when learning a new task, which can prevent the degradation of the neural network's expressive ability.

[0028] The information processing device repeatedly performs the forward propagation and backward propagation in learning task #T. The number of repetitions may be a predetermined number. Alternatively, the number of repetitions may be until the loss value and the weighting coefficient of each unit converge.

[0029] When the learning of task #T is completed, the information processing device calculates the importance parameter γ Max l-1,n Calculate the following.

[0030] Below, the importance parameter γ used in learning task #T+1 is Max l-1,n The calculation of the importance parameter associated with one weighting factor is explained below. The calculation method for the importance parameter associated with one weighting factor is the same for each weighting factor of the neural network. Therefore, the calculation method for the importance parameter used in learning task #T+1 for one weighting factor at the time when learning task #T is completed will be explained below. For the sake of simplicity, the weighting factor to be explained below will be written as W, and the importance parameter to be calculated will be written as γ Max It is written as follows.

[0031] In the following description, the layer section 100 and the head #t are treated as a single function, and the function f t In other words, the function f tis a function that calculates the output value output by head #t when the value input to the layer section 100 is used as an argument. Note that although not explained here, the same applies to the method of calculating the importance parameter associated with the weight coefficient of the unit of each head.

[0032] The first importance γ of the weighting factor W for task #t at the end of learning task #T t is calculated by the following formula:

number

number

[0033] In addition, the first importance γ t is calculated for each layer. In other words, if the weighting coefficient W corresponds to a unit in the lth layer, the mean value and variance used to calculate the function Norm correspond to the weighting coefficients of each of the N units in the lth layer. In this example, since each of the N units in one layer is associated with N weighting coefficients, the number of weighting coefficients for which the mean value and variance are calculated is N. 2 is.

[0034] In equation (1), the function tanh normalizes the gradient of the weighting coefficient W to a value greater than or equal to 0 and less than 1.

[0035] The information processing device assigns a first importance γ 1 ~γ T The maximum value among them is γ T,Max As a result, this maximum value γ T,Max Therefore, the information processing device records the maximum value γ 1,Max ~Maximum value γ T,Max The information processing device records a total of T maximum values ​​of the maximum value γ 1,Max ~Maximum value γ T,Max The maximum value of these is used as the importance parameter γ of the weighting coefficient W used in learning task #T+1. Max Let's say.

[0036] The importance parameter of the weighting coefficient W used in task #T is the maximum value γ 1,Max ~Maximum value γ T-1,Max Therefore, the information processing device determines the maximum value γ T,Max The larger of the weighting factor W used in task #T and the weighting factor W used in task #T is used as the weighting factor W γ Max In this case, the information processing device may calculate the past maximum value γ 1,Max ~Maximum value γ T,Max It is not necessary to record all of the above, but it is sufficient to record only the latest importance parameter calculated at the time the learning process is completed and to be used in the next learning process.

[0037] Importance parameter γ of weighting coefficient W Max is the maximum value of the importance of the weighting factor W for each of tasks #1 to #T. In this embodiment, the update of the weighting factor W in learning for task #T+1 is suppressed according to this importance. More specifically, the higher the importance of at least one of tasks #1 to #T, the smaller the update amount of the weighting factor W in learning for task #T+1.

[0038] This configuration reduces the amount of update for the weighting factor W, which is important in past tasks, and can suppress the occurrence of catastrophic forgetting.

[0039] In addition, in the HAT technology, the update amount of the weighting factor W, which is important in past tasks, is set to approximately 0, so the weighting factor W is not updated when learning task #T+1, which hinders efficient knowledge transfer (transfer learning).

[0040] On the other hand, the importance parameter γ Max The value of varies continuously within a range of 0 or more and less than 1 according to the importance of the weighting factor W. Therefore, the weighting factor W is also updated during learning of task #T+1, thereby promoting knowledge transfer (transfer learning). Conversely, in this embodiment, even if the weighting factor W was important in a past task, it is updated according to its importance, which has the advantage that learning of task #T+1 can improve the performance of the past task, i.e., so-called backward transfer can occur.

[0041] Fig. 4 is a hardware configuration diagram of an information processing device 1 according to this embodiment. The information processing device 1 has one or more processors 11, a storage device 12 including volatile and nonvolatile memory, and an input / output interface 13 such as a keyboard, a mouse, and a display. The one or more processors 11 execute a program stored in the storage device 12, thereby realizing the functional blocks shown in Fig. 5.

[0042] The functional blocks of the information processing device 1 shown in Fig. 5 will be described below. The model storage unit 16 stores model information. The model information includes information such as the weight coefficients of each unit in multiple layers of the layer unit 100 and the connection relationships between each unit. The model information also includes information such as the number of heads in the head unit 200 and the weight coefficients of each unit in one or more layers of each head. For example, at the time when learning of task #T-1 is completed, the history storage unit 17 stores the maximum value γ 1,Max ~Maximum value γ T-1,Max It stores:

[0043] The learning unit 15 learns task #T by repeatedly performing the forward propagation and backward propagation described above. When the learning of task #T is completed, as described above, the learning unit 15 learns task #T by repeatedly performing the forward propagation and backward propagation described above. Max is calculated and stored in the model storage unit 16.

[0044] 6 is a flowchart of the process executed by the learning unit 15 when learning task #T. In S10, the learning unit 15 performs learning using learning set #T while adjusting the gradient of each weighting coefficient based on the importance parameter of each weighting coefficient.

[0045] When the learning using the learning set #T is completed, the learning unit 15 calculates the first importance γ 1 ~γ T In step S12, the maximum value of the first importance γ T,Max is calculated and stored in the history storage unit 17.

[0046] In step S13, the learning unit 15 calculates the maximum value γ 1,Max ~γ T,Max The maximum value of the task #T+1 is determined, and the determined maximum value is set as the importance parameter to be used in learning of the task #T+1. The importance parameter is set in the model storage unit 16.

[0047] The program that causes the information processing device to function can be stored in, for example, a non-transitory computer-readable storage medium and distributed.

[0048] The invention is not limited to the above-described embodiment, and various modifications and variations are possible within the scope of the gist of the invention.

Claims

1. A method executed by an information processing device to cause a neural network that has learned a first task to a T-1th task in sequence by using a first learning set to a T-1th learning set (T is an integer equal to or greater than 2) in sequence to learn a Tth task using the Tth learning set, the neural network includes a plurality of layers, each of the plurality of layers including a plurality of units; each of the plurality of units having a weighting factor associated with each input of the unit and an importance parameter indicating the importance of the weighting factor; The method comprises: In training the T tasks, adjusting the first weighting factor based on a first gradient of the first weighting factor determined using the T training sets and the importance parameter of the first weighting factor; After the learning using the T learning set is completed, a second gradient of each of a plurality of second weighting factors including the first weighting factor included in a first layer including the first weighting factor among the plurality of layers is determined based on the T learning set, and the importance parameter of the first weighting factor to be used in learning the T+1 task is calculated based on the second gradient of each of the plurality of second weighting factors; A method comprising:

2. 10. The method of claim 1, Adjusting the first weighting factor adjusting the first gradient of the first weighting factor determined using the T learning sets based on the importance parameter of the first weighting factor to determine a third gradient; adjusting the first weighting factor based on the third gradient; Including, the absolute value of the third gradient is less than the absolute value of the first gradient; The difference between the third gradient and the first gradient increases as the importance of the first weighting factor indicated by the importance parameter of the first weighting factor increases.

3. 10. The method of claim 1, The step of determining the importance parameter of the first weighting coefficient used in learning the T+1 task includes: normalizing the second gradient of the first weighting factor based on the second gradient of each of the plurality of second weighting factors.

4. 4. The method of claim 3, The method of claim 1, wherein the absolute value of the second gradient of the first weighting factor is normalized to a value greater than or equal to 0 and less than 1.

5. 4. The method of claim 3, The method, wherein normalizing the second gradients of the first weighting factors includes determining a mean value and a variance of the second gradients of each of the plurality of second weighting factors.

6. 4. The method of claim 3, The method, wherein the importance parameter of the first weighting factor used in learning the T+1 task is the larger of the normalized second gradient of the first weighting factor and the importance parameter of the first weighting factor used in learning the T task.

7. 4. The method of claim 3, the plurality of layers each have a first head to a T-th head for outputting the neural network corresponding to the first task to the T-th task, respectively; The step of determining the importance parameter of the first weighting coefficient used in learning the T+1 task includes: determining a fourth gradient for the t-th task of each of the plurality of second weighting factors based on an output of a t-th head (where t is an integer from 1 to T) using the T learning sets; normalizing the fourth gradient of the first weighting coefficient in the t task based on the fourth gradient of each of the plurality of second weighting coefficients in the t task; determining a maximum value among the normalized fourth gradients of each of the first task through the T tasks; The normalized second gradient of the first weighting factor is the maximum value.

8. A program that, when executed on one or more processors of an apparatus having one or more processors, causes the apparatus to perform the method of any one of claims 1 to 7.

9. An information processing device, one or more processors; a storage device for storing a program executable by said one or more processors; Equipped with the storage device further stores, for each of a plurality of units included in each of a plurality of layers of the neural network that has learned the first task to the T-1th task in order by using the first learning set to the T-1th learning set (T is an integer equal to or greater than 2), a weighting coefficient associated with each input of the unit and an importance parameter indicating the importance of the weighting coefficient; the one or more processors execute the program to cause the information processing device to execute a learning process that causes the neural network to learn the T-th task using the T-th learning set; The learning process includes: adjusting the first weighting factor based on a first gradient of the first weighting factor determined using the T learning sets and the importance parameter of the first weighting factor; After the learning using the T learning set is completed, a second gradient of each of a plurality of second weighting factors including the first weighting factor included in a first layer including the first weighting factor among the plurality of layers is determined based on the T learning set, and the importance parameter of the first weighting factor to be used in learning the T+1 task is calculated based on the second gradient of each of the plurality of second weighting factors; An information processing device comprising:

Images