Learning model generation device, learning model generation method, and program

Abstract

To provide a technique that can create a learning model with a high accuracy rate for both a source task and an additional task.SOLUTION: A learning unit (12) learns a learning data-label relationship of a source task and generates a learning model of the source task. A transfer learning unit (15) generates, based on the learning model of the source task, a model structure including a label of an additional task in the learning model of the source task, and generates a learning model of the additional task by learning a learning data-label relationship of the additional task based on the model structure.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] This invention relates to a learning model generation device, a learning model generation method, and a program. [Background technology]

[0002] Techniques for generating learning models using transfer learning have been developed over time. For example, there is the invention disclosed in Patent Document 1 below.

[0003] The information processing device described in Patent Document 1 acquires multiple training data belonging to a first category as target task training data in a first task which is the target task, and acquires multiple training data including training data belonging to the first category and training data belonging to the second category as source task training data in a second task.

[0004] The information processing device then adjusts the transformation parameters for transforming the original task training data based on the difference between a first distribution relating to multiple training data belonging to a first category of the target task training data and a second distribution relating to multiple training data belonging to a first category of the original task training data.

[0005] The information processing device then transforms the original task learning data based on the transformation parameters adjusted in the adjustment step, and generates a classifier for the first task based on the transformed original task learning data and the target task learning data. [Prior art documents] [Patent Documents]

[0006] [Patent Document 1] Japanese Patent Publication No. 2017-224156 (Published December 21, 2017) [Overview of the project] [Problems that the invention aims to solve]

[0007] The information processing device described in Patent Document 1 above enables the generation of highly accurate learning models by performing data transformations between tasks in transfer learning.

[0008] However, in transfer learning, when generating a learning model for a new task with a new task that adds different labels to labels identifiable in the original task, the information processing device described in Patent Document 1 cannot move data beyond the development environment. Therefore, it cannot move the data used to generate the learning model for the original task to the development environment where the learning model for the target task is generated. Consequently, the information processing device described in Patent Document 1 may not be able to generate a learning model under such circumstances.

[0009] One aspect of the present invention aims to provide a technology that can create a learning model with a high accuracy rate in both the original task and the additional task. [Means for solving the problem]

[0010] To solve the above problems, a learning model generation device according to one aspect of the present invention comprises: a learning unit that learns the relationship between the learning data and labels of an original task and generates a learning model for the original task; and a transfer learning unit that, based on the learning model for the original task, generates a model structure that includes the labels of an additional task in the learning model for the original task, learns the relationship between the learning data and labels of the additional task based on the model structure, and generates a learning model for the additional task.

[0011] Furthermore, in order to solve the above problems, a learning model generation method according to one aspect of the present invention involves learning the relationship between the learning data and labels of the original task to generate a learning model for the original task, generating a model structure that includes the labels of the additional task based on the learning model for the original task, and learning the relationship between the learning data and labels of the additional task based on the model structure to generate a learning model for the additional task.

[0012] Moreover, to solve the above problems, a program according to one aspect of the present invention causes a computer to execute a process of learning the relationship between learning data and labels of an original task to generate a learning model for the original task, a process of generating a model structure that includes labels of an additional task in the learning model for the original task based on the learning model for the original task, and a process of learning the relationship between learning data and labels of the additional task based on the model structure to generate a learning model for the additional task. [Advantageous Effects of Invention]

[0013] According to one aspect of the present invention, it is possible to create a learning model with a high accuracy rate for both the original task and the additional task. [Brief Description of Drawings]

[0014] [Figure 1] This is a block diagram illustrating a configuration example of a general learning model generation device (learning part) for transfer learning. [Figure 2] This is a block diagram illustrating a configuration example of a general learning model generation device (evaluation part) for transfer learning. [Figure 3] This is a flowchart for explaining the processing procedure of the learning unit shown in Figure 1. [Figure 4] This is a flowchart for explaining the processing procedure of the transfer learning unit shown in Figure 1. [Figure 5] This is a flowchart for explaining the details of the transfer learning process for the target task model shown in Figure 4. [Figure 6] This is a flowchart for explaining the details of the initialization process for transfer learning shown in Figure 5. [Figure 7] This is a flowchart for explaining the processing procedure of the evaluation unit shown in Figure 2. [Figure 8] This is a block diagram illustrating a configuration example of a learning model generation device (learning part) according to an embodiment of the present invention. [Figure 9] This is a block diagram illustrating a configuration example of a learning model generation device (evaluation part) according to an embodiment of the present invention. [Figure 10] Figure 8 is a flowchart illustrating the processing procedure of the learning unit. [Figure 11] Figure 8 is a flowchart illustrating the processing procedure of the transfer learning unit. [Figure 12] Figure 11 is a flowchart illustrating the details of the transfer learning process for the additional task model. [Figure 13] Figure 12 is a flowchart illustrating the details of the initialization process for transfer learning. [Figure 14] Figure 9 is a flowchart illustrating the processing procedure of the evaluation unit. [Figure 15] This graph shows the relationship between the identification results of the original task and the identification results of the additional task. [Modes for carrying out the invention]

[0015] First, before describing an example configuration of a learning model generation device according to one embodiment of the present invention, we will explain the definitions of terms used in transfer learning and an example configuration of a general learning model generation device for transfer learning.

[0016] (A general learning model generator for transfer learning) <Definition of Terms> In machine learning, the problem that needs to be solved is called a task (T). A task can be defined as a pair (T={γ,P(Y│X)}) of a label space (γ) and a conditional probability (P(Y│X)). Here, X and Y are the data X={x1,x2,…}∈χ in the data space (χ) and the labels Y={y1,y2,…)∈γ in the label space (γ), respectively.

[0017] In the case of a classification problem, the task involves learning from known data that has been labeled based on a predetermined finite number of labels, and then inferring which label unlabeled, unknown data belongs to.

[0018] When performing transfer learning, a learning model generated for one task is used to generate a learning model for another task. In this case, the task of the learning model from which the transfer originates is called the original task (T). s ={γ s ,P(Y s │X s )}), the task of the learning model to which the transfer will take place is the target task (T t ={γ t ,P(Y t │X t We'll call it )}).

[0019] A learning model consists of a model structure and its weight parameters.

[0020] <Example Configuration of a General Learning Model Generator for Transfer Learning> Figure 1 is a block diagram showing an example configuration of a general learning model generation device (learning unit) 100 for transfer learning. The learning model generation device 100 comprises a source task learning data storage unit 110, a learning unit 120, a source task learning model storage unit 130, a target task learning data storage unit 140, a transfer learning unit 150, and a target task learning model storage unit 160.

[0021] The original task learning data storage unit 110, the original task learning model storage unit 130, the target task learning data storage unit 140, and the target task learning model storage unit 160 are provided as a database in a storage device such as RAM (Random Access Memory), non-volatile memory, or a hard disk (not shown).

[0022] The original task learning data storage unit 110 stores the learning data of the original task and the labels for that learning data. The original task learning model storage unit 130 stores the learning model of the original task for transfer learning.

[0023] The target task learning data storage unit 140 stores the learning data for the target task and the labels for that learning data. The target task learning model storage unit 160 stores the learning model for the target task generated by transfer learning.

[0024] The learning unit 120 includes an original task learning data input unit 121, an original task model learning unit 122, and an original task learning model output unit 123. The original task learning data input unit 121 retrieves the learning data and labels of the original task from the original task learning data storage unit 110 and outputs them to the original task model learning unit 122.

[0025] The original task model learning unit 122 uses machine learning to learn the relationship between the original task's training data and labels, and temporarily stores the original task's learning model. The original task learning model output unit 123 retrieves the original task's learning model temporarily stored in the original task model learning unit 122 and stores it in the original task learning model storage unit 130.

[0026] The transfer learning unit 150 includes an input unit 151 for the original task learning model, an input unit 152 for the target task learning data, a transfer learning unit 153 for the target task model, and an output unit 154 for the target task learning model.

[0027] The original task learning model input unit 151 retrieves the learning model of the original task from the original task learning model storage unit 130 and outputs it to the target task model transfer learning unit 153. The target task learning data input unit 152 retrieves the learning data and labels of the target task from the target task learning data storage unit 140 and outputs them to the target task model transfer learning unit 153.

[0028] The target task model transfer learning unit 153 uses transfer learning to learn the relationship between the training data and labels of the target task and temporarily stores the target task's learning model. The target task learning model output unit 154 retrieves the target task's learning model temporarily stored in the target task model transfer learning unit 153 and stores it in the target task learning model storage unit 160.

[0029] Figure 2 is a block diagram showing an example configuration of a general learning model generation device (evaluation section) 100 for transfer learning. The learning model generation device 100 further includes a target task evaluation data storage unit 170, an evaluation unit 180, and a target task identification result storage unit 190.

[0030] The target task evaluation data storage unit 170 and the target task identification result storage unit 190 are provided as a database in a storage device such as RAM, non-volatile memory, or a hard disk (not shown).

[0031] The target task evaluation data storage unit 170 stores the evaluation data for the target task and the labels for that evaluation data. The target task identification result storage unit 190 stores the identification results for the target task.

[0032] Furthermore, the evaluation unit 180 includes a target task learning model input unit 181, a target task evaluation data input unit 182, a target task data identification unit 183, and a target task identification result output unit 184.

[0033] The target task learning model input unit 181 retrieves the learning model for the target task from the target task learning model storage unit 160 and outputs it to the target task data identification unit 183. The target task evaluation data input unit 182 retrieves the evaluation data for the target task from the target task evaluation data storage unit 170 and outputs it to the target task data identification unit 183.

[0034] The target task data identification unit 183 identifies evaluation data based on the target task's learning model and temporarily stores the identification results. The target task identification result output unit 184 retrieves the identification results temporarily stored in the target task data identification unit 183 and stores them in the target task identification result storage unit 190.

[0035] <Processing flow of a general learning model generator for transfer learning> Figure 3 is a flowchart illustrating the processing procedure of the learning unit 120 shown in Figure 1. First, the original task learning data input unit 121 retrieves the learning data and labels of the original task from the original task learning data storage unit 110 and outputs them to the original task model learning unit 122 (S101).

[0036] Next, the original task model learning unit 122 uses machine learning to learn the relationship between the original task's training data and labels, and temporarily stores the generated original task's learning model (S102). Then, the original task learning model output unit 123 retrieves the original task's learning model temporarily stored in the original task model learning unit 122 and stores it in the original task learning model storage unit 130 (S103), and terminates the process.

[0037] Figure 4 is a flowchart illustrating the processing procedure of the transfer learning unit 150 shown in Figure 1. First, the original task learning model input unit 151 retrieves the learning model of the original task from the original task learning model storage unit 130 and outputs it to the target task model transfer learning unit 153 (S111).

[0038] Furthermore, the target task learning data input unit 152 retrieves the learning data and labels for the target task from the target task learning data storage unit 140 and outputs them to the target task model transfer learning unit 153 (S112).

[0039] Next, the target task model transfer learning unit 153 learns the relationship between the training data and labels of the target task based on the learning model learned in the original task, and temporarily stores the generated target task learning model (S113). Details of this process will be described later.

[0040] Then, the target task learning model output unit 154 retrieves the learning model for the target task that was temporarily stored in the target task model transfer learning unit 153, stores it in the target task learning model storage unit 160 (S114), and terminates the process.

[0041] Figure 5 is a flowchart illustrating the details of the transfer learning process (S113) of the target task model shown in Figure 4. First, the target task model transfer learning unit 153 performs an initialization process for transfer learning (S121). The details of this process will be described later.

[0042] Next, the target task model transfer learning unit 153 generates a learning model for the target task using machine learning (S122). Then, the target task model transfer learning unit 153 temporarily saves the generated learning model for the target task (S123) and terminates the process.

[0043] Figure 6 is a flowchart illustrating the details of the transfer learning initialization process (S121) shown in Figure 5. First, the target task model transfer learning unit 153 selects a layer from which to extract weights from the original task's learning model generated by the learning unit 120 (S131).

[0044] Next, the target task model transfer learning unit 153 determines which layers to add to the learning model for the target task, based on the layers selected from the learning model for the original task, and initializes the weight parameters corresponding to the added layers with random numbers (S132).

[0045] Next, the target task model transfer learning unit 153 connects the newly added layers to the layers selected from the original task's learning model to form the structure of the target task's learning model (S133). Then, the target task model transfer learning unit 153 specifies a range of the weight parameters of the target task's learning model in which weight updates will not be performed through learning, fixes them to their current values ​​(S134), and terminates the process.

[0046] Figure 7 is a flowchart illustrating the processing procedure of the evaluation unit 180 shown in Figure 2. First, the target task learning model input unit 181 retrieves the learning model of the target task from the target task learning model storage unit 160 and outputs it to the target task data identification unit 183 (S141). Also, the target task evaluation data input unit 182 retrieves the evaluation data of the target task from the target task evaluation data storage unit 170 and outputs it to the target task data identification unit 183 (S142).

[0047] Next, the target task data identification unit 183 identifies the evaluation data based on the learning model of the target task and temporarily stores the identification result. Then, the target task identification result output unit 184 retrieves the identification result temporarily stored in the target task data identification unit 183 and stores it in the target task identification result storage unit 190 (S144), and terminates the process.

[0048] (Learning model generation device according to one embodiment of the present invention) <Prerequisites> The development environment for generating and evaluating learning models stores the data necessary for training and evaluation in a database. It has functions for generating learning models from the data stored in the database, and it is not possible to move data between databases of different development environments. However, it is possible to exchange the generated learning model with the evaluation results performed within the development environment.

[0049] <Definition of Terms> In machine learning, the problem that needs to be solved is called a task (T). A task can be defined as a pair (T={γ,P(Y│X)}) of a label space (γ) and a conditional probability (P(Y│X)). Here, X and Y are the data X={x1,x2,…}∈χ in the data space (χ) and the labels Y={y1,y2,…)∈γ in the label space (γ), respectively.

[0050] In the case of a classification problem task, the task refers to performing learning using known data labeled based on a predetermined finite number of labels, and inferring which label unlabeled unknown data belongs to.

[0051] When performing transfer learning, a learning model generated for a certain task is used to generate a learning model for another task. At this time, the task of the transfer-source learning model is referred to as the source task (T s ={γ s , P(Y s |X s )), and the task of the transfer-destination learning model is referred to as the target task (T t ={γ t , P(Y t |X t )).

[0052] A learning model is composed of a model structure and its weight parameters.

[0053] In the present embodiment, this is a method for generating a learning model (γ s ⊂γ t ) in which the label space of the target task includes the label space of the source task, using transfer learning in different development environments where data exchange is not possible. At this time, the task related to the labels to be added, which is the difference between the target task and the source task, is referred to as the additional task (T a ={γ a , P(Y a |X a )).

[0054] <Configuration Example of Learning Model Generation Device 1 According to One Embodiment of the Present Invention> Figure 8 is a block diagram illustrating a configuration example of a learning model generation device (learning part) 1 according to an embodiment of the present invention. The learning model generation device 1 includes a source task learning data storage unit 11, a learning unit 12, a source task learning model storage unit 13, an additional task learning data storage unit 14, a transfer learning unit 15, and a target task learning model storage unit 16.

[0055] The original task learning data storage unit 11, the original task learning model storage unit 13, the additional task learning data storage unit 14, and the target task learning model storage unit 16 are provided as a database in a storage device such as RAM, non-volatile memory, or a hard disk (not shown).

[0056] The original task learning data storage unit 11 stores the learning data of the original task and the labels for that learning data. The original task learning model storage unit 13 stores the learning model of the original task for transfer learning.

[0057] The additional task learning data storage unit 14 stores the learning data for the additional task and the labels for that learning data. The target task learning model storage unit 16 stores the learning model for the target task generated by transfer learning.

[0058] The learning unit 12 includes an original task learning data input unit 12-1, an original task model learning unit 12-2, and an original task learning model output unit 12-3. The original task learning data input unit 12-1 retrieves the learning data and labels of the original task from the original task learning data storage unit 11 and outputs them to the original task model learning unit 12-2.

[0059] The original task model learning unit 12-2 uses machine learning to learn the relationship between the original task's training data and labels, and temporarily stores the original task's learning model. The original task learning model output unit 12-3 retrieves the original task's learning model temporarily stored in the original task model learning unit 12-2 and stores it in the original task learning model storage unit 13.

[0060] The transfer learning unit 15 includes an original task learning model input unit 15-1, an additional task learning data input unit 15-2, an additional task model transfer learning unit 15-3, and an additional task learning model output unit 15-4.

[0061] The original task learning model input unit 15-1 retrieves the learning model of the original task from the original task learning model storage unit 13 and outputs it to the additional task model transfer learning unit 15-3. The additional task learning data input unit 15-2 retrieves the learning data and labels of the additional task from the additional task learning data storage unit 14 and outputs them to the additional task model transfer learning unit 15-3.

[0062] The additional task model transfer learning unit 15-3 uses transfer learning to learn the relationship between the training data and labels of the additional task and temporarily stores the training model of the additional task. The additional task training model output unit 15-4 retrieves the training model of the additional task temporarily stored in the additional task model transfer learning unit 15-3 and stores it in the target task training model storage unit 16.

[0063] Figure 9 is a block diagram showing an example configuration of a learning model generation device (evaluation section) 1 according to one embodiment of the present invention. The learning model generation device 1 further includes a source task evaluation data storage unit 17, an additional task evaluation data storage unit 18, an evaluation unit 19, and a judgment result storage unit 20.

[0064] The original task evaluation data storage unit 17, the additional task evaluation data storage unit 18, and the judgment result storage unit 20 are provided as a database in a storage device such as RAM, non-volatile memory, or a hard disk (not shown).

[0065] The original task evaluation data storage unit 17 stores the evaluation data of the original task and the labels associated with that evaluation data. The additional task evaluation data storage unit 18 stores the evaluation data of the additional task and the labels associated with that evaluation data. The judgment result storage unit 20 stores the judgment results from the evaluation unit 19.

[0066] The evaluation unit 19 includes an original task evaluation data input unit 19-1, an original task data identification unit 19-2, an original task identification result output unit 19-3, an original task identification result storage unit 19-4, a target task learning model input unit 19-5, an additional task evaluation data input unit 19-6, an additional task data identification unit 19-7, an additional task identification result output unit 19-8, an additional task identification result storage unit 19-9, an identification result input unit 19-10, an identification result determination unit 19-11, and a determination result output unit 19-12.

[0067] The original task identification result storage unit 19-4 and the additional task identification result storage unit 19-9 are provided as a database in a storage device such as RAM, non-volatile memory, or a hard disk (not shown).

[0068] The original task evaluation data input unit 19-1 retrieves the evaluation data for the original task from the original task evaluation data storage unit 17 and outputs it to the original task data identification unit 19-2. The target task learning model input unit 19-5 retrieves the learning model for the target task from the target task learning model storage unit 16 and outputs it to the original task data identification unit 19-2 and the additional task data identification unit 19-7.

[0069] The original task data identification unit 19-2 identifies the evaluation data of the original task based on the learning model of the target task and temporarily stores the identification result. The original task identification result output unit 19-3 retrieves the original task identification result temporarily stored in the original task data identification unit 19-2 and stores it in the original task identification result storage unit 19-4.

[0070] The additional task evaluation data input unit 19-6 retrieves the evaluation data for the additional task from the additional task evaluation data storage unit 18 and outputs it to the additional task data identification unit 19-7.

[0071] The additional task data identification unit 19-7 identifies evaluation data for additional tasks based on the learning model of the target task and temporarily stores the identification results. The additional task identification result output unit 19-8 retrieves the identification results of additional tasks temporarily stored in the additional task data identification unit 19-7 and stores them in the additional task identification result storage unit 19-9.

[0072] The identification result input unit 19-10 retrieves the identification result of the original task and the identification result of the additional task from the original task identification result storage unit 19-4 and the additional task identification result storage unit 19-9, and outputs them to the identification result determination unit 19-11.

[0073] The identification result determination unit 19-11 determines which learning model yielded the best identification result from among multiple learning models based on the identification result of the original task and the identification result of the additional task, and temporarily stores that determination result. The determination result output unit 19-12 retrieves the determination result temporarily stored in the identification result determination unit 19-11 and stores it in the determination result storage unit 20.

[0074] <Processing flow of the learning model generation device 1 according to one embodiment of the present invention> Figure 10 is a flowchart illustrating the processing procedure of the learning unit 12 shown in Figure 8. First, the original task learning data input unit 12-1 retrieves the learning data and labels of the original task from the original task learning data storage unit 11 and outputs them to the original task model learning unit 12-2 (S1).

[0075] Next, the original task model learning unit 12-2 uses machine learning to learn the relationship between the original task's training data and labels, and temporarily stores the generated original task's learning model (S2). Then, the original task learning model output unit 12-3 retrieves the original task's learning model temporarily stored in the original task model learning unit 12-2, stores it in the original task learning model storage unit 13 (S3), and terminates the process.

[0076] Figure 11 is a flowchart illustrating the processing procedure of the transfer learning unit 15 shown in Figure 8. First, the original task learning model input unit 15-1 retrieves the learning model of the original task from the original task learning model storage unit 13 and outputs it to the additional task model transfer learning unit 15-3 (S11).

[0077] Furthermore, the additional task learning data input unit 15-2 retrieves the learning data and labels for the additional task from the additional task learning data storage unit 14 and outputs them to the additional task model transfer learning unit 15-3 (S12).

[0078] Next, the additional task model transfer learning unit 15-3 uses transfer learning, based on the learning model learned in the original task, to learn the relationship between the learning data and labels of the additional task, and temporarily stores the generated learning model for the additional task (S13). Details of this process will be described later.

[0079] Then, the additional task learning model output unit 15-4 retrieves the learning model for the additional task that was temporarily stored in the additional task model transfer learning unit 15-3, stores it in the target task learning model storage unit 16 (S14), and terminates the process.

[0080] Figure 12 is a flowchart illustrating the details of the transfer learning process (S13) of the additional task model shown in Figure 11. First, the additional task model transfer learning unit 15-3 performs an initialization process for transfer learning (S21). The details of this process will be described later.

[0081] Next, the additional task model transfer learning unit 15-3 generates a learning model for the additional task using machine learning (S22). Then, the additional task model transfer learning unit 15-3 temporarily saves the generated learning model for the additional task (S23) and terminates the process.

[0082] Figure 13 is a flowchart illustrating the details of the transfer learning initialization process (S21) shown in Figure 12. First, the additional task model transfer learning unit 15-3 determines a model structure in which the final layer of the model structure of the learning model of the original task generated by the learning unit 12 is extended to fit the number of labels of the target task (S31).

[0083] Specifically, the system adds nodes in the final layer (hereinafter referred to as output nodes) to match the number of labels in the target task. For example, if the original task has 200 labels and the target task has 210 labels, the additional task model transfer learning unit 15-3 adds 10 output nodes. The additional task model transfer learning unit 15-3 also adds a new connection between the added output layer (output node) and the intermediate layer immediately preceding the output layer (hereinafter referred to as the final intermediate layer).

[0084] Next, the additional task model transfer learning unit 15-3 initializes the weight parameters of the learning model for the additional task with random numbers. Specifically, the additional task model transfer learning unit 15-3 initializes the weight parameters of the new connection between the final hidden layer and the output layer with random numbers.

[0085] Furthermore, the additional task model transfer learning unit 15-3 initializes the parts of the structure that overlap with the learning model of the original task using the weight parameters of the learning model of the original task (S32). Specifically, the additional task model transfer learning unit 15-3 initializes the weight parameters between the final intermediate layer and the output layer, excluding the new connection, using the weight parameters of the learning model of the original task.

[0086] Then, the additional task model transfer learning unit 15-3 specifies a range of the weight parameters of the target task's learning model in which weight updates will not be performed through learning, and fixes them to their current values ​​(S33). Specifically, the additional task model transfer learning unit 15-3 sets the weight parameters other than the connection between the final hidden layer and the output layer in a range in which weight updates will not be performed through learning, and fixes the weight parameters.

[0087] Figure 14 is a flowchart illustrating the processing procedure of the evaluation unit 19 shown in Figure 9. First, the target task learning model input unit 19-5 retrieves the learning model of the target task from the target task learning model storage unit 16 and outputs it to the original task data identification unit 19-2 and the additional task data identification unit 19-7 (S41, S46). Also, the original task evaluation data input unit 19-1 retrieves the evaluation data of the original task from the original task evaluation data storage unit 17 and outputs it to the original task data identification unit 19-2 (S42).

[0088] Next, the original task data identification unit 19-2 identifies the original task evaluation data based on the learning model of the target task and temporarily stores the identification result (S43). The original task identification result output unit 19-3 retrieves the identification result temporarily stored in the original task data identification unit 19-2 and stores it in the original task identification result storage unit 19-4 (S44).

[0089] Meanwhile, the additional task evaluation data input unit 19-6 retrieves the evaluation data for the additional task from the additional task evaluation data storage unit 18 and outputs it to the additional task data identification unit 19-7 (S45).

[0090] The additional task data identification unit 19-7 identifies evaluation data for the additional task based on the learning model of the target task and temporarily stores the identification result (S47). The additional task identification result output unit 19-8 retrieves the identification result temporarily stored in the additional task data identification unit 19-7 and stores it in the additional task identification result storage unit 19-9 (S48).

[0091] The identification result input unit 19-10 retrieves the identification result of the original task and the identification result of the additional task from the original task identification result storage unit 19-4 and the additional task identification result storage unit 19-9, and outputs them to the identification result determination unit 19-11 (S49).

[0092] The identification result determination unit 19-11 determines the identification result of the original task and the additional task of the learning model for the same target task based on thresholds. The identification result determination unit 19-11 has a first threshold (threshold: original task) for the identification result of the original task and a second threshold (threshold: additional task) for the identification result of the additional task set in advance. The identification result determination unit 19-11 then determines whether the identification result of the original task is equal to or greater than the first threshold, and whether the identification result of the additional task is equal to or greater than the second threshold (S50).

[0093] For example, the identification result determination unit 19-11 determines that the learning model for the target task meets the criteria if both the identification result of the original task and the identification result of the additional task are above a threshold. Conversely, the identification result determination unit 19-11 determines that the learning model for the target task does not meet the criteria if either the identification result of the original task or the identification result of the additional task is below a threshold, or if both are below a threshold.

[0094] Finally, the judgment result output unit 19-12 retrieves the judgment result temporarily stored in the identification result determination unit 19-11 and stores it in the judgment result storage unit 20 (S51), and terminates the process. Note that, considering the operation of the learning model, if the identification result determination unit 19-11 determines that none of the candidate identification results meet the criteria, the learning model generation device 1 changes the hyperparameters and restarts the learning model generation process.

[0095] Figure 15 is a graph showing the relationship between the identification results of the original task and the identification results of the additional task. In the graph shown in Figure 15, the horizontal axis represents the number of training epochs, and the vertical axis represents the accuracy rate (80% to 95%). The learning model generator 1 generates multiple learning models by changing the number of training epochs, which is one of the hyperparameters, and calculates the accuracy rate (identification result) of the original task and the accuracy rate (identification result) of the additional task for each learning model.

[0096] As shown in Figure 15, when the number of training epochs is small, the accuracy of the original task is high, and the accuracy of the additional task is low. As the number of training epochs increases, the accuracy of the original task gradually decreases, and the accuracy of the additional task increases. When the number of training epochs is E1 or higher, the accuracy of the additional task exceeds the second threshold (threshold: additional task). Also, when the number of training epochs is E2 or higher, the accuracy of the original task falls below the first threshold (threshold: original task). In Figure 15, the training model is judged to meet the criteria when the number of training epochs is in the range of E1 to E2.

[0097] In Figure 15, multiple learning models meet the criteria. To select the optimal learning model, one could choose the model with the highest sum of accuracy on the original task and accuracy on the additional task, or one could choose the model whose accuracy on the original task and accuracy on the additional task are the same.

[0098] In the above explanation, we created multiple learning models by changing the number of learning epochs, which is one of the hyperparameters. However, you may also create multiple learning models by changing other hyperparameters such as the decay coefficient.

[0099] Furthermore, machine learning methods such as CNN (Convolutional Neural Network), R-CNN (Regional CNN), Fast R-CNN, Faster R-CNN, RNN (Recurrent Neural Network), U-Net, and SegNet may also be used.

[0100] <Effects of Learning Model Generation Device 1> In machine learning, there are cases where database sharing is not possible when trying to generate a learning model using data located in different development environments. Under such conditions, it is impossible to generate a learning model using general machine learning or transfer learning techniques.

[0101] For example, Company P, which is contracted to generate learning models, generates learning model M1 under Company A's management environment using data held by Company A. Company P may also want to generate an improved final learning model M2 based on the generated learning model M1, using data held by Company B.

[0102] In such cases, Company A's data may only be accessible within Company A's management environment, and Company B's data only within Company B's management environment. However, even in such situations, Company P can generate the desired improved final learning model M2 by transferring the learning model to both Company A's and Company B's management environments.

[0103] Therefore, as in this embodiment, weight parameters necessary for generating the target task's learning model are extracted from the original task's learning model, and transfer learning is performed using the extracted weight parameters as initial values ​​for the learning model's parameters, while modifying the hyperparameters. This is achieved by evaluating both the original task's data and the added label data, and generating a learning model in which the evaluation results of both satisfy a predetermined threshold.

[0104] As described above, according to the learning model generation device 1 of this embodiment, the transfer learning unit 15 learns the relationship between the learning data and labels of the additional task based on the model structure of the learning model of the original task, which includes the labels of the additional task, and generates a learning model for the additional task. Therefore, the transfer learning unit 15 can create a learning model with a high accuracy rate for both the original task and the additional task.

[0105] Furthermore, the transfer learning unit 15 adds an output layer as a label for the additional task to the learning model of the original task, and generates a model structure that adds a connection between the final intermediate layer and the added output layer, thereby enabling the generation of an even more suitable learning model.

[0106] Furthermore, the transfer learning unit 15 fixes the weight parameters of the connections included in the original task, updates the weight parameters of the added connections, learns the relationship between the training data and labels of the additional task, and generates a learning model for the additional task. This allows for the generation of an even more suitable learning model.

[0107] Furthermore, since the evaluation unit 19 evaluates the learning model for the target task, which includes the learning model for the original task and the learning model for the additional task, it is possible to generate a learning model with an even higher accuracy rate.

[0108] Furthermore, the identification result determination unit 19-11 determines whether the learning model for the target task satisfies the criteria based on the identification result by the original task data identification unit 19-2 and the identification result by the additional task data identification unit 19-7, thereby enabling the generation of a learning model with an even more favorable accuracy rate.

[0109] Furthermore, when the identification result determination unit 19-11 determines that the learning model for the target task satisfies the criteria when the identification result by the original task data identification unit 19-2 is equal to or greater than the first threshold and the identification result by the additional task data identification unit 19-7 is equal to or greater than the second threshold, it is possible to generate a learning model with an even more favorable accuracy rate.

[0110] <Examples of implementation using software> The function of the learning model generation device 1 (hereinafter referred to as "the device") is a program that causes a computer to function as the device, and can be realized by a program that causes a computer to function as each control block of the device (in particular, the learning unit 12, the transfer learning unit 15, and the evaluation unit 19).

[0111] In this case, the device includes a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., memory) as hardware for executing the program. By executing the program using this control device and storage device, the functions described in each of the embodiments are realized.

[0112] The above program may be recorded on one or more computer-readable recording media, not temporary ones. These recording media may or may not be provided by the above device. In the latter case, the program may be supplied to the above device via any wired or wireless transmission medium.

[0113] Furthermore, some or all of the functions of each of the above control blocks can also be realized by logic circuits. For example, an integrated circuit in which logic circuits functioning as each of the above control blocks are formed is also included in the scope of the present invention. In addition, it is also possible to realize the functions of each of the above control blocks by, for example, a quantum computer.

[0114] 〔summary〕 The learning model generation device according to embodiment 1 of the present invention is A learning unit that learns the relationship between the training data and labels of the original task and generates a learning model for that original task, The system includes a transfer learning unit that generates a model structure that includes the labels of the additional task in the learning model of the original task based on the learning model of the original task, and then learns the relationship between the learning data and labels of the additional task based on the model structure to generate a learning model for the additional task.

[0115] A learning model generation apparatus according to embodiment 2 of the present invention is a learning model generation apparatus described in embodiment 1, The aforementioned transfer learning unit, To the learning model of the original task, a node corresponding to the label of the additional task is added to the output layer. The model structure is generated by adding the connection between the final intermediate layer and the added node.

[0116] A learning model generation apparatus according to embodiment 3 of the present invention is a learning model generation apparatus described in embodiment 2, The aforementioned transfer learning unit, The weight parameters of the connections included in the original task are fixed, and while updating the weight parameters of the added connections, the relationship between the training data and labels of the added task is learned, thereby generating a learning model for the added task.

[0117] A learning model generation apparatus according to embodiment 4 of the present invention is a learning model generation apparatus described in any of embodiments 1 to 3, The learning model generation device further includes an evaluation unit that evaluates the learning model for the target task, which includes the learning model for the original task and the learning model for the additional task.

[0118] A learning model generation device according to aspect 5 of the present invention is a learning model generation device described in aspect 4, The evaluation unit, A source task data identification unit identifies evaluation data of the source task based on the learning model of the target task, An additional task data identification unit identifies evaluation data for the additional task based on the learning model of the target task, The system includes an identification result determination unit that determines whether the learning model for the target task satisfies the criteria based on the identification result by the original task data identification unit and the identification result by the additional task data identification unit.

[0119] The learning model generation apparatus according to embodiment 6 of the present invention is the learning model generation apparatus described in embodiment 5, The identification result determination unit is The identification result by the original task data identification unit is greater than or equal to the first threshold, When the identification result by the additional task data identification unit is equal to or greater than the second threshold, it is determined that the learning model for the target task meets the criteria.

[0120] The learning model generation method according to aspect 7 of the present invention is: The relationship between the original task's training data and labels is learned to generate a learning model for that original task. Based on the learning model of the original task, a model structure is generated that includes the labels of the additional tasks in the learning model of the original task. Based on the model structure, the relationship between the training data and labels of the additional task is learned, and a training model for the additional task is generated.

[0121] The program according to aspect 8 of the present invention is: On the computer, The process involves learning the relationship between the training data and labels of the original task and generating a training model for that original task, A process to generate a model structure that includes the labels of the additional tasks in the learning model of the original task, based on the learning model of the original task. The system is configured to perform the following steps: learn the relationship between the training data and labels of the additional task based on the model structure, and generate a training model for the additional task.

[0122] [Additional Notes] The present invention is not limited to the embodiments described above, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention. [Explanation of Symbols]

[0123] 1.100 Learning Model Generator 11, 110 Original task learning data storage unit 12, 120 Learning Department 12-1, 121 Original Task Learning Data Input Section 12-2, 122 Former Task Model Learning Department 12-3, 123 Original Task Learning Model Output Unit 13, 130 Original Task Learning Model Storage Unit 14 Additional task learning data storage unit 15, 150 Transfer Learning Section 15-1 Input section of the original task-learning model 15-2 Additional Task Learning Data Input Section 15-3 Additional Task Model Transfer Learning Unit 15-4 Additional Task Learning Model Output Section 16, 160 Target Task Learning Model Storage Unit 17, 170 Target Task Evaluation Data Storage Unit 18 Additional Task Evaluation Data Storage Unit 19, 180 Evaluation Department 19-1 Former Task Evaluation Data Entry Section 19-2 Original Task Data Identification Unit 19-3 Output section for original task identification result 19-4 Original Task Identification Result Storage Unit 19-5, 181 Target Task Learning Model Input Section 19-6 Additional Task Evaluation Data Input Section 19-7 Additional Task Data Identification Unit 19-8 Additional Task Identification Result Output Unit 19-9 Additional Task Identification Result Storage Unit 19-10 Identification Result Input Section 19-11 Identification Result Determination Unit 19-12 Output section for judgment results 20. Judgment Result Storage Unit 140 Target Task Learning Data Storage Unit 182 Target Task Evaluation Data Input Section 183 Target Task Data Identification Unit 184 Target Task Identification Result Output Unit 190 Target Task Identification Result Storage Unit

Claims

1. A learning unit that learns the relationship between the training data and labels of the original task and generates a learning model for that original task, The system includes a transfer learning unit that generates a model structure that includes the labels of the additional task in the learning model of the original task, and that learns the relationship between the learning data and labels of the additional task based on the model structure to generate a learning model for the additional task. The aforementioned transfer learning unit, To the learning model of the original task, a node corresponding to the label of the additional task is added to the output layer. The model structure is generated by adding the connection between the final intermediate layer and the added node, A learning model generation device that generates a learning model for an additional task by learning the relationship between the learning data of the additional task and the labels, while fixing the weight parameters other than the connection between the final intermediate layer and the output layer included in the original task, and updating the weight parameters between the final intermediate layer and the output layer included in the original task, and the weight parameters between the final intermediate layer and the added node, thereby generating a learning model for the additional task.

2. The learning model generation device according to claim 1, further comprising an evaluation unit that evaluates a learning model for a target task that includes the learning model for the original task and the learning model for the additional task.

3. The evaluation unit, A source task data identification unit identifies evaluation data of the source task based on the learning model of the target task, An additional task data identification unit identifies evaluation data for the additional task based on the learning model of the target task, The learning model generation apparatus according to claim 2, further comprising an identification result determination unit that determines whether or not the learning model for the target task satisfies the determination criteria based on the identification result by the original task data identification unit and the identification result by the additional task data identification unit.

4. The identification result determination unit is The identification result by the original task data identification unit is greater than or equal to the first threshold, The learning model generation device according to claim 3, wherein when the identification result by the additional task data identification unit is greater than or equal to a second threshold, it is determined that the learning model for the target task satisfies the criteria.

5. Computers The relationship between the original task's training data and labels is learned to generate a learning model for that original task. Based on the learning model of the original task, a model structure is generated that includes the labels of the additional tasks in the learning model of the original task. A method for generating a learning model, which generates a learning model for an additional task by learning the relationship between the learning data and labels of the additional task based on the model structure, In generating the learning model for the aforementioned additional task, To the learning model of the original task, a node corresponding to the label of the additional task is added to the output layer. The model structure is generated by adding the connection between the final intermediate layer and the added node, A method for generating a learning model, which involves fixing the weight parameters other than the connection between the final intermediate layer and the output layer included in the original task, while updating the weight parameters between the final intermediate layer and the output layer included in the original task, and the weight parameters between the final intermediate layer and the added node, thereby learning the relationship between the learning data of the added task and the labels, and generating a learning model for the added task.

6. On the computer, The process involves learning the relationship between the training data and labels of the original task and generating a training model for that original task, A process to generate a model structure that includes the labels of the additional tasks in the learning model of the original task, based on the learning model of the original task. A program that performs the following processes: learning the relationship between the training data and labels of the additional task based on the model structure, and generating a training model for the additional task; In the process of generating the learning model for the aforementioned additional task, To the learning model of the original task, a node corresponding to the label of the additional task is added to the output layer. The model structure is generated by adding the connection between the final intermediate layer and the added node, A program that, while fixing the weight parameters other than the connection between the final intermediate layer and the output layer included in the original task, updates the weight parameters between the final intermediate layer and the output layer included in the original task, and the weight parameters between the final intermediate layer and the added node, learns the relationship between the training data of the added task and the labels, and executes a process to generate a training model for the added task.

Images