Model learning facility, model learning method and model learning program

Abstract

A model learning facility (1) comprises a fixed branch selection unit (16) that selects fixed branches as branches to be excluded from learning objects from a plurality of branches contained in a learning model; a computation graph modification unit (13) that modifies a computation graph to be used into one derived from a first computation graph utilizing the plurality of branches and a second computation graph utilizing learning object branches obtained by excluding the fixed branches from the plurality of branches; an inter-branch distance computation unit (11) that computes distances between branches, including distances between features generated by each of the plurality of branches, in a state where the computation graph to be used has been modified into the first computation graph; and a loss function computation unit (12).which calculates a total sum of losses based on a predetermined loss function and the distances between branches, a branch update unit (14) that updates weight parameters in the learning object branches based on the total sum of losses in a state in which the computation graph to be used has been modified to the second computation graph.

Description

AREA OF TECHNOLOGY

[0001] The present disclosure relates to a model learning device, a model learning method and a model learning program. TECHNICAL BACKGROUND

[0002] In cases where the amount of training data for machine learning is small, where a difficult problem is posed (as in weakly supervised learning), or similar situations, there is a possibility that a feature will be learned that is undesirable for a human. Even if it is possible to determine whether a learned feature is accurate or not through visualization using XAL (Explainable AI) (e.g., to check whether an undesired feature has been retained), it is difficult to provide feedback (e.g., to perform transfer learning) so that a learning model does not learn an inaccurate feature.Therefore, as a method for providing feedback so that the learning model does not learn an incorrect feature, a model learning procedure was proposed in which the learning of features to be obtained is controlled by modifying a loss function with respect to each data element by relearning the attention obtained through learning, and the transfer learning is repeated until a desired feature is obtained (see e.g. patent reference 1). REFERENCES ON THE STATE OF THE TECHNOLOGY PATENT REFERENCE

[0003] Patent reference 1: Publication of Japanese patent application no. 2022-79331. SUMMARY OF THE INVENTION TASK TO BE SOLVED BY THE INVENTION

[0004] In the conventional model learning method described above, it is necessary to repeat transfer learning until a desired trait is acquired. Furthermore, there is a risk of reacquiring a previously learned trait, as the transfer learning process is repeated without memorizing previously acquired traits. Therefore, the conventional model learning method is inherently inefficient.

[0005] One objective of the present disclosure, which was set out to solve the problems described above with conventional technology, is to provide a model learning device, a model learning procedure, and a model learning program that will make it possible to increase the efficiency of model learning. MEANS TO SOLVE THE PROBLEM

[0006] A model learning device according to the present disclosure is a device that performs transfer learning with respect to a learning model stored in memory. The model learning device comprises a fixed branch selection unit for selecting fixed branches from a plurality of branches contained in the learning model as branches to be excluded from learning objects; a computation graph modification unit for modifying a computation graph to be used into a first computation graph that uses the plurality of branches, and a second computation graph that uses learning object branches obtained by excluding the fixed branches from the plurality of branches;an inter-branch distance calculation unit to calculate distances between branches, including distances between features generated by each of the plurality of branches, in a state where the computation graph to be used has been modified to the first computation graph; a loss function calculation unit to calculate a total loss based on a predetermined loss function and the distances between branches; and a branch update to update weight parameters in the learning object branches based on the total loss in a state where the computation graph to be used has been modified to the second computation graph.

[0007] A model learning method according to the present disclosure is a method to be executed by a model learning device that performs transfer learning with respect to a learning model stored in memory. The model learning method comprises a step of selecting fixed branches as branches to be excluded from learning objects from a plurality of branches contained in the learning model; a step of modifying a computation graph to be used into a first computation graph that uses the plurality of branches and a second computation graph that uses learning object branches obtained by excluding the fixed branches from the plurality of branches; and a step of computing distances between branches, including distances between features generated by each of the plurality of branches, in a state.in which the calculation graph to be used was modified into the first calculation graph, a step of calculating a total sum of losses based on a predetermined loss function and the distances between branches, and a step of updating weighting parameters in the learning object branches based on the total sum of losses in a state in which the calculation graph to be used was modified into the second calculation graph. IMPACT OF THE INVENTION

[0008] According to the present disclosure, the efficiency of model learning can be increased. BRIEF DESCRIPTION OF THE DRAWINGS Fig. Figure 1 is a block diagram which schematically shows the configuration of a model learning device according to a first embodiment. Fig. Figure 2 is a representation showing an example of the hardware configuration of the model learning device according to the first embodiment. Fig. Figure 3 is a schematic representation showing the functioning of the model learning device according to the first embodiment. Fig. Figure 4 is a schematic representation showing the functionality of a model learning facility as a comparative example. Fig. Figure 5 is an explanatory illustration showing the functioning of a model learning unit at the time of forward propagation. Fig. Figure 6 is an explanatory illustration showing the functioning of a model learning unit at the time of error backpropagation. Fig. Figure 7 is a flowchart showing the operation of the model learning device according to the first embodiment. Fig. Figure 8 is a flowchart showing the operation of the model learning device according to the first embodiment at the time of model learning. Fig. Figure 9 is a block diagram which schematically shows the configuration of a model learning device according to a second embodiment. Fig. Figure 10 is a representation showing an example of the hardware configuration of the model learning device according to the second embodiment. Fig. Figure 11 is a flowchart showing the operation of the model learning device according to the second embodiment at the time of model learning. Fig. Figure 12 is a block diagram which schematically shows the configuration of a model learning device according to a third embodiment. Fig. Figure 13 is a representation showing an example of the hardware configuration of the model learning device according to the third embodiment. Fig. Figure 14 is a flowchart showing the operation of the model learning device according to the third embodiment at the time of model learning. Fig. Figure 15 is a block diagram which schematically shows the configuration of a model learning device according to a fourth embodiment. Fig. Figure 16 is a representation showing an example of the hardware configuration of the model learning device according to the fourth embodiment. Fig. Figure 17 is a flowchart showing the operation of the model learning device according to the fourth embodiment at the time of model learning. MODE FOR EXECUTING THE INVENTION

[0009] The following describes a model learning device, a model learning method, and a model learning program according to each embodiment with reference to the drawings. The following embodiments are only examples, and it is possible to combine embodiments appropriately and to modify each embodiment appropriately. <1> First embodiment<1-1> configuration

[0010] Fig. Figure 1 is a block diagram schematically showing the configuration of a model learning device 1 according to a first embodiment. The model learning device 1 according to the first embodiment is a device capable of performing a model learning procedure, such as a computer that, for example, executes a model learning program according to the first embodiment. The model learning device 1 according to the first embodiment comprises a model learning unit 10, a branch visualization unit 15, and a fixed branch selection unit 16. The model learning unit 10 comprises an inter-branch distance calculation unit 11, a loss function calculation unit 12, a computation graph modification unit 13, and a branch update unit 14. Furthermore, one or both of the fixed branch selection unit 16 and the branch visualization unit 15 can also be part of the model learning unit 10.

[0011] Fig. Figure 2 is a diagram showing an example of the hardware configuration of the model learning device 1 according to the first embodiment. The model learning device 1 comprises, for example, a processor 101, such as a CPU (Central Processing Unit), a memory 102 as a storage device, and an interface 103. The components of the model learning device 1 are, for example, equipped with a processing circuit. The processing circuit can be dedicated hardware or a CPU that executes a program (for example, a model learning program) stored in memory 102. The processor 101 implements the Fig. 1 function blocks shown.

[0012] Memory 102, for example, comprises semiconductor memory such as RAM (Random Access Memory) and non-volatile storage, such as an HDD (Hard Disk Drive). Model Learning Setup 1 can also be a combination of components equipped with a processing circuit and components equipped with a processor. Furthermore, part or all of Model Learning Setup 1 can be a server computer on a network. The model learning program is provided by downloading it over a network or via a storage medium that holds information, such as a USB drive.

[0013] In the example in Fig. Memory 102 stores a learning model and learning data that can be used for learning. The learning model comprises a variety of attention branches (also simply called "branches"). The interface 103 includes an input unit 104 as a user interface through which user operations are performed, and a display unit 105, such as a liquid crystal display, that presents information. The hardware configuration in Fig. 2 is only an illustration and therefore modification is possible.

[0014] In Fig. 1 and Fig. 2. The fixed branch selection unit 16 of the model learning facility 1 performs transfer learning with respect to the learning model stored in memory 102. Fixed branches, defined as branches to be excluded from learning objects (i.e., branches in which a weighting parameter is defined), are selected from the multitude of branches contained in the learning model. The input of identification information for the fixed branches is performed, for example, via input unit 104, where operations for inputting operations are carried out by a user.

[0015] The computation graph modification unit 13 in the model learning unit 10 modifies a computation graph to be used into one that utilizes the multitude of branches contained in the learning model (i.e., a computation graph corresponding to a configuration at the time of forward propagation, as in Fig. 5 shown, which will be explained later) and a second computation graph that uses learning object branches obtained by excluding the fixed branches from the multitude of branches included in the learning model (i.e., a computation graph corresponding to a configuration at the time of error backpropagation, as in Fig. 6 shown, which will be explained later).

[0016] The inter-branch distance computation unit 11 in the model learning unit 10 computes distances between branches, including distances between features generated by each of the plurality of branches in the learning model, in a state where the computation graph used for learning has been modified to the initial computation graph (i.e., the computation graph corresponding to the configuration at the time of forward propagation). The inter-branch distance computation unit 11 may, in addition to the distances between the features generated by each of the plurality of branches in the learning model, also include the distances between each of the features and the feature of a predetermined target branch.

[0017] The loss function calculation unit 12 in the model learning unit 10 calculates a total sum of losses based on a predetermined loss function and the distances between branches.

[0018] The branch update unit 14 in the model learning unit 10 updates the weight parameter in each learning object branch based on the total sum of losses obtained by the loss function computation unit 12 in a state where the computation graph to be used during learning has been modified to the second computation graph (i.e. the computation graph which corresponds to the configuration at the time of fault backpropagation).

[0019] Branch visualization unit 15 visualizes the features generated by each of the many branches included in the learning model. Specifically, branch visualization unit 15 transmits the feature to display unit 105 and causes display unit 105 to display the feature.

[0020] Fig. Figure 3 is a schematic representation showing the operation of the model learning device 1 according to the first embodiment. The model learning device 1 receives previously learned features in units of branches A1, A2, ..., ..., A n(n: positive integer), memorizes the received branches and learns new features at the time of transfer learning by learning the inter-branch distance. By repeating inter-branch distance learning (e.g., distance learning between each received branch and the target branch, and distance learning between received branches) each time a new feature is obtained by learning the inter-branch distance at the time of transfer learning as described above, the number of times transfer learning is required until a matching feature (i.e., a branch A) is acquired can be reduced. n , which is connected to the target branch B0 in a Fig. (3 overlaps the feature space shown). In this case, the model learning facility 1 memorizes the obtained branches with respect to undesired features obtained in the past and takes into account the distances between branches, thus enabling it to perform the learning process while the results of previous transfer learning processes are fed back and utilized to the best of their ability.

[0021] Fig. Figure 4 is a schematic representation illustrating the functionality of a model learning setup as a comparative example. The model learning setup, serving as the comparative example, receives learned features in units of branches C1, C2, ..., ..., C. n (n: positive integer), while the loss function changes with respect to each data element at the time of transfer learning and repeats the transfer learning, and thereby repeats the transfer learning until a matching feature (i.e., a branch C) is found. n, which is connected to the target branch B0 in a Fig. (4 overlaps the feature space shown). In this case, it is not possible to use the branches obtained in the past with respect to unwanted features, because the branches obtained in the past with respect to unwanted features were not memorized. In this case, there is a possibility that a branch with respect to an unwanted feature will be relearned, and inefficient model learning will be performed.

[0022] Fig. Figure 5 is an explanatory illustration showing the functioning of model learning unit 10 at the time of forward propagation. Fig. Figure 5 shows a case where the computational graph modification unit 13 in the model learning unit 10 handles branch #1 and branch #2 as the learning object branches, and branch #3 as the fixed branch (i.e., a branch that is excluded from the learning objects by the fixed branch selection unit 16). At the time of forward propagation, while features #1, #2, and #3 are being generated from branches #1, #2, and #3 respectively, the computational graph modification unit 13 inserts features #1 and #2 into a header without inserting feature #3 into the header. This is because a command to exclude feature #3 from the learning objects, since feature #3 is an inaccurate feature that is undesirable for a human, has been entered into the fixed branch selection unit 16.Since, on the other hand, feature #1 and feature #2 are to be learned as features with large distances to feature #3, the computation graph modification unit 13 inputs all features, including features #1, #2 and #3, into the inter-branch distance computation unit 11.

[0023] Fig. Figure 6 is an explanatory illustration showing the functionality of the model learning unit 10 at the time of error backpropagation. Fig. Figure 6 shows a case where the computation graph modification unit 13 in the model learning unit 10 handles branch #1 and branch #2 as the learning object branches, and branch #3 as the fixed branch. At the time of the error backpropagation, a command to handle branch #3 as the fixed branch was entered into the fixed branch selection unit 16, and therefore the computation graph modification unit 13 removes the input and output edges of branch #3 from the computation graph to exclude branch #3 from the learning objects. Therefore, the computation graph modification unit 13 enters features #1 and #2 into the inter-branch distance computation unit 11, but does not enter feature #3, which was generated by the fixed branch, into the inter-branch distance computation unit 11.

[0024] As described above, in the first embodiment, the calculation graph at the time of forward propagation and the calculation graph at the time of error backpropagation differ. Features #1–#3 of all branches, including feature #3 of the fixed branch, are output to the inter-branch distance calculation unit 11 when the total loss is obtained based on the distances between branches, whereas features #1–#2 of branches, excluding feature #3 of the fixed branch, are output to the inter-branch distance calculation unit 11 when the branch update is performed. (3-2) Functionality

[0025] Fig. Figure 7 is a flowchart illustrating the operation of the model learning device 1 according to the first embodiment at the time of model learning. In the first embodiment, the model learning unit 10 learns the model using learning data (e.g., the learning data in memory 102 in Fig. 2) (Step S1).

[0026] Subsequently, the branch visualization unit 15 visualizes the feature received from each branch using XAI and causes the display unit (e.g., display unit 105 in) to Fig. 2) The visualization result is presented in a way that allows it to be interpreted by a human (step S2). In this case, it is also possible to display the visualization result using a BI (Business Intelligence) tool or a dedicated GUI (Graphical User Interface). As with XAI, there are local explanations (e.g., explanations relating to individual data elements) and global explanations (e.g., explanations of a model's behavior). The explanation (attention) relating to each local part is used as XAI in conventional technology, while in the first implementation, either the local explanation or the global explanation can be used as XAI, and it is also possible to use both.The user views the visualization result and, if the exclusion of a branch is required, enters the identification information about the fixed branch as the branch to be excluded (i.e., the branch ID) into the fixed branch selection unit 16, for example by entering input unit 104 into . Fig. 2 uses.

[0027] If the feature learned by each branch falls into a predetermined condition, namely a first case or a second case, which are described below, based on the result of the human interpretation of the feature learned by each branch, the Fester branch selection unit 16 sets the weighting parameter of the branch that received the feature that falls into the first case or second case and excludes the feature that falls into the first case or second case from the learning objects.

[0028] The first case is when the feature generated by the branch through learning is undesirable for a human. Since the feature is not used for inference after learning in the first case, the branch that learns the feature in the first case is designated as a fixed branch and excluded from the learning objects to prevent the feature from being relearned.

[0029] The second case is when the feature generated by the branch through learning is a feature that is desirable for a human. Although the feature in the second case is used for inference after learning, the branch that learns the feature in the second case is designated as a fixed branch and excluded from the learning objects to allow the feature in the second case to be retained even when transfer learning is performed.

[0030] Model learning unit 10 assesses whether relearning is required or not, and returns the process to step S1 if relearning is required (YES in step S3), or terminates the process if relearning is not required (NO in step S3).

[0031] Fig. Figure 8 is a flowchart showing the operation of the model learning device 1 according to the first embodiment at the time of model learning (i.e., details of step S1 in Fig. 7) First, the model learning unit 10 assesses whether the learning to be performed is the first learning or not. If it is the first learning (YES in step S101), the process continues to step S106, where the loss function calculation unit 12 calculates the loss function. If the second or subsequent model learning is being performed (NO in step S101), the model learning unit 10 continues the process to step S102.

[0032] In step S102, the model learning unit 10 assesses whether a fixed branch exists whose weighting parameter is defined as a branch to be used for preserving a feature of the data. If a fixed branch exists (YES in step S102), the model learning unit 10 continues the process from step S102 to step S103 and selects the fixed branches. If no fixed branch exists (NO in step S102), the model learning unit 10 continues the process from step S102 to step S104.

[0033] In step S104, the computation graph modification unit 13 in the model learning unit 10 modifies the computation graph for the purpose of forward propagation. As in Fig. As shown in Figure 5, at the time of forward propagation, an edge from the input into Branch #3 is set to valid as the fixed branch, an edge from Branch #3 to the inter-branch distance calculation unit 11 is set to valid, and an edge from Branch #3 to the header is set to invalid.

[0034] In step S105, the Inter-Branch Distance Calculation Unit 11 in the Model Learning Unit 10 calculates the distances between branches. In this case, Inter-Branch Distance Calculation Unit 11 calculates the distances between branches in order to learn branches that differ from those learned in the past. Inter-Branch Distance Calculation Unit 11 calculates the following two types of distances: a first distance and a second distance, as the distances between branches:

[0035] The first distance is the distance between a feature generated by a learning object branch and a feature generated by a fixed branch. The first distance should be large to learn features that differ from those learned in the past.

[0036] The second distance is the distance between features generated by each learning object branch. This second distance should be large so that features generated simultaneously by a large number of learning object branches (i.e., newly acquired branches) are not similar to each other. The second distance is absent if the number of learning object branches is 1.

[0037] Here, the distance can be freely defined by the user. In ArcFace, a deep-distance learning method (deep-metric learning), for example, the cosine between the features mapped on a hypersphere is defined as the distance.

[0038] In the next step S106, the loss function calculation unit 12 in the model learning unit 10 calculates the total sum of losses using a previously determined loss function. The loss function is defined as the sum of a task-dependent loss and a distance loss, which depends on the inter-branch distance calculation unit 11 (i.e., the total sum of losses), and is represented by the following expression (1): (Total losses) = (task-dependent loss) + (β × (distance loss))

[0039] Since the number of terms that contribute to the distance loss varies depending on the result of the selection by the Fester-Branch selection unit 16, the hyperparameter β is adjusted (or normalized) according to equilibrium with the task-dependent loss t.

[0040] If the number of fixed branches is “a” and the number of learning object branches is “b”, there are as many terms as the number represented by the following expression (2): a*b+bC2

[0041] In expression (2) the first term represents the number of combinations of a fixed branch and a learning object branch, and the second term represents the number of combinations between learning object branches.

[0042] In step S107, the computation graph modification unit 13 in the model learning unit 10 modifies the computation graph for the purpose of error backpropagation. As in Fig. As shown in Figure 6, at the time of error backpropagation, the edge from the input into branch #3 is set to valid as the fixed branch, the edge from branch #3 to the inter-branch distance calculation unit 11 is set to valid, and the edge from branch #3 to the header is set to invalid.

[0043] In step S108, the branch update unit 14 in the model learning unit 10 updates the weighting parameters in the learning object branches. <1-3> Effect

[0044] With the model learning device 1 according to the first embodiment, learning can be started with a small number of branches, thereby preventing overtraining and accelerating the learning process. <2> Second embodiment

[0045] Fig. Figure 9 is a block diagram which schematically shows the configuration of a model learning device 2 according to a second embodiment. Fig. 9 is assigned to each component that is part of a larger set of parts. Fig. Component 1 shown is the same as or corresponds to the same component, and is assigned the same reference symbol as in . Fig. 1. Fig. Figure 10 is a representation showing an example of the hardware configuration of the model learning device 2 according to the second embodiment. Fig. 10 was assigned to each component that was connected to a Fig. The component shown in section 2 is identical or corresponds to it, using the same reference symbol as in the previous section. Fig. 2 assigned. The model learning device 2 according to the second embodiment is a device that is capable of performing a model learning procedure, such as a computer that, for example, executes a model learning program according to the second embodiment.

[0046] The model learning device 2 according to the second embodiment differs from the model learning device 1 according to the first embodiment in that it includes a branch addition unit 21 and that a computation graph modification unit 13a modifies the computation graph on the basis of the branches plus branches provided by the branch addition unit 21.

[0047] In general, learning performed in a state where many branches have been prepared in the learning model requires the use of many weighting parameters, resulting in a high level of difficulty, a likelihood of overtraining, or a long processing time. Therefore, there are cases where it is desirable to perform the initial learning phase with a small number of learnable learning object branches and subsequently increase the number of learning object branches by adding branches to the learning model via the branch addition unit 21. Since the number of learning object branches used by the Fester branch selection unit 16 decreases with each iteration of transfer learning, learning object branches are subsequently added to the model learning device 2 as needed by the branch addition unit 21, according to the second embodiment.

[0048] Fig. Figure 11 is a flowchart illustrating the operation of the model learning device 2 according to the second embodiment at the time of model learning. Fig. 11 is every step that involves one in Fig. Step 8 shown is identical or corresponds to it, using the same reference symbol as in Fig. 8 assigned. The operation of the model learning unit 2 at the time of model learning differs from the operation of the model learning unit 1 according to the first embodiment at the time of model learning in that it still includes step S201 of assessing whether the branch addition should be performed by the branch addition unit 21 or not, and step S202 in which the branch addition unit 21 adds branches to a model learning unit 20 if the branch addition is performed, and that the model learning unit 20 performs the processing in steps S104 to S107 using learning object branches including the added branches.

[0049] With the model learning device 2 according to the second embodiment, learning can be started with a small number of branches, thereby preventing overtraining and accelerating the learning process.

[0050] Furthermore, the branch visualization unit 15 with the model learning device 2 according to the second embodiment visualizes the feature obtained from each branch using XAI, and the user is able to add branches whose weighting parameters have been appropriately initialized to the learning model via the branch addition unit 21, thereby increasing the accuracy of the learning.

[0051] With the exception of the features described above, the second embodiment is identical to the first embodiment. <3> Third embodiment

[0052] Fig. Figure 12 is a block diagram which schematically shows the configuration of a model learning device 3 according to a third embodiment. Fig. 12 is each component that is identical to one in Fig. The component shown in point 1, or which corresponds to it, has the same reference symbol as in point 1. Fig. 1 assigned. Fig. Figure 13 is a representation showing an example of the hardware configuration of the model learning device 3 according to the third embodiment. Fig. 13 was assigned to each component that was connected to a Fig. The component shown in section 2 is identical or corresponds to it, using the same reference symbol as in the previous section. Fig. 2 assigned. The model learning device 3 according to the third embodiment is a device that is capable of executing a model learning procedure, such as a computer that, for example, executes a model learning program according to the third embodiment.

[0053] The model learning device 3 according to the third embodiment differs from the model learning device 1 according to the first embodiment in that it includes a branch removal unit 31 and that a computation graph modification unit 13b modifies the computation graph on the basis of the branches from which designated branches were removed by the branch removal unit 31.

[0054] Generally, during a maintenance / operational phase, after model learning is complete, branches that have learned incorrect features remain in memory. In cases where the number of branches is large, the time required to perform inference using the model becomes lengthy, and the memory consumption of the branches increases. Therefore, according to the third embodiment, the model learning unit 3 includes the branch removal unit 31 and is configured to remove user-selected branches based on the model definition and weighting parameters. Furthermore, a backup of the removed branches can be created at the time of removal, as there may be a situation where the branches need to be relearned.

[0055] Fig. Figure 14 is a flowchart illustrating the operation of the model learning device 3 according to the third embodiment at the time of model learning. Fig. 14 is every step that involves one in Fig. Step 8 shown is identical or corresponds to it, using the same reference symbol as in Fig. 8 assigned. The operation of the model learning device 3 at the time of model learning differs from the operation of the model learning device 1 according to the first embodiment at the time of model learning in that it still includes step S301 of assessing whether or not branch removal should be performed by the branch removal unit 31, and step S302 in which the branch removal unit 31 removes branches from a model learning unit 30, in a case of performing branch removal, and that the model learning unit 30 performs the processing in steps S104 to S107 using the learning object branches exclusively of the removed branches.

[0056] Furthermore, the branch visualization unit 15 with the model learning device 3 according to the third embodiment visualizes the feature obtained from each branch using XAI, and the user is able to remove branches via the branch removal unit 31, so that the accuracy of the learning can be increased, thereby reducing the amount of working memory consumed and accelerating the inference.

[0057] With the exception of the features described above, the third embodiment is identical to the first embodiment. Furthermore, it is also possible to apply the branch removal unit 31 in the third embodiment to the model learning device 2 in the second embodiment. <4> Fourth embodiment

[0058] Fig. Figure 15 is a block diagram which schematically shows the configuration of a model learning device 4 according to a fourth embodiment. Fig. 15 was assigned to each component that was associated with a Fig. The component shown in point 1 is identical to or corresponds to it, using the same reference symbol as in point 1. Fig. 1 assigned. Fig. Figure 16 is a representation showing an example of the hardware configuration of the model learning device 4 according to the fourth embodiment. Fig. 16 was assigned to each component that was associated with a Fig. The component shown in section 2 is identical or corresponds to it, using the same reference symbol as in the previous section. Fig. 2 assigned. The model learning device 4 according to the fourth embodiment is a device that is capable of performing a model learning procedure, such as a computer that, for example, executes a model learning program according to the fourth embodiment.

[0059] The model learning device 4 according to the fourth embodiment differs from the model learning device 1 according to the first embodiment in that it comprises a learning object branch selection unit 41, correct attention response data 42 and an attention loss calculation unit 43, and that a calculation graph modification unit 13c modifies the calculation graph on the basis of learning object branches designated by the learning object branch selection unit 41, and a loss function calculation unit 12c modifies the calculation of the loss function on the basis of the attention loss calculation unit 43.

[0060] In the fourth embodiment, the learning of the features to be retained is controlled by modifying the loss function with respect to each individual data element by directly correcting the attention obtained through learning. For example, model learning device 4 selects a specific learning object branch and causes the branch to perform the learning, generating an attention that closely approximates that corrected by a human. In such cases, where a feature with a probability of confusion is already known, the required number of transfer learning cycles can be reduced by selectively preparing data with such attention and causing the data to be learned.Furthermore, in cases where such a feature, where there is a probability of confusion, is specifically learned, the reliability of the learning model can be increased by inferring that this feature is not used.

[0061] In Fig. 15 and Fig. 16 The learning object branch selection unit 41 selects the learning object branches to be used for the learning process by utilizing the correct attention response data. In this case, the type(s) of attention(s) stored in the correct attention response data 42 and the branch(es) selected by the learning object branch selection unit 41 can have a one-to-one correspondence, a one-to-many correspondence, or a many-to-many correspondence. In the case of person recognition data, examples of correct attention response data 42 include heatmap data where the heatmap is applied to the upper body, heatmap data where the heatmap is applied to the lower body, heatmap data where the heatmap is applied to the entire body, and the like.In cases of person recognition, the learning object branch selection unit 41 can, for example, select branches that recognize the head, or branches that recognize a part other than the head (e.g., the upper body or the lower body).

[0062] Fig. Figure 17 is a flowchart illustrating the operation of the model learning device 4 according to the fourth embodiment at the time of model learning. Fig. 17 is every step that involves one in Fig. Step 8 shown is identical or corresponds to it, using the same reference symbol as in Fig. 8 assigned.

[0063] If the correct attention response data 42 is available in the second or subsequent learning (YES in step S401), the model learning device 4 allows the learning object branch selection unit 41 to select the learning object branches (step S402), allows the attention loss calculation unit 43 to calculate losses with respect to the selected learning object branches (step S403), and then proceeds with the process to step S106, which differs from the operation of the model learning device 1 according to the first embodiment at the time of model learning.

[0064] In the fourth embodiment, the loss due to attention is used only for learning the specific branch, and therefore the error backpropagation must be performed multiple times as described below, and the computation graph to be used at that time is also memorized with respect to each of the multiple error backpropagation operations. In the fourth embodiment, the task-dependent loss and the distance loss dependent on the inter-branch distance computation unit 11 can be error-backpropagated to all learning object branches, and it is also possible to error-backpropagate the loss due to attention only to the selected specific branch.

[0065] With the model learning device 4 according to the fourth embodiment, the branch visualization unit 15 visualizes the feature obtained from each branch using XAI, and the user can remove branches via the branch removal unit 31, thereby increasing the accuracy of the learning, thereby reducing the amount of memory consumed and accelerating the inference.

[0066] Furthermore, the required number of transfer learning trials can be reduced by preparing attention data with respect to data in which a feature likely to be confused is already known, and by inducing the learning process to engage this data. Additionally, in cases where learning is initiated for such a feature with a probability of confusion, the reliability of the learning model can be increased by inferring that this feature will not be used.

[0067] With the exception of the features described above, the fourth embodiment is identical to the first embodiment. Furthermore, it is also possible to apply the learning object branch selection unit 41, the correct attention response data 42, and the attention loss calculation unit 43 from the fourth embodiment to the model learning device 2 from the second or third embodiment. REFERENCE MARK LIST

[0068] 1-4: Model learning unit, 10, 20, 30, 40: Model learning unit, 11: Inter-branch distance calculation unit, 12, 12c: Loss function calculation unit, 13, 13a, 13b, 13c: Computation graph modification unit, 14: Branch update unit, 15: Branch visualization unit, 16: Fixed branch selection unit, 21: Branch addition unit, 31: Branch removal unit, 41: Learning object branch selection unit, 42: Correct attention response data, 43: Attention loss calculation unit, 101, 101a, 101b, 101c: Processor, 102: Memory, 103: Interface. QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] JP 2022-79331

[0003]

Claims

[1] Model learning device which performs transfer learning with respect to a learning model stored in memory, wherein the model learning device comprises: a fixed branch selection unit to select fixed branches from a large number of branches included in the learning model, which are to be excluded from learning objects; a computation graph modification unit to modify a computation graph to be used into one consisting of a first computation graph that uses the multitude of branches, and a second computation graph that uses learning object branches obtained by excluding the fixed branches from the multitude of branches; an inter-branch distance computation unit to compute distances between branches, including distances between features generated by each of the plurality of branches, in a state where the computation graph to be used has been modified into the first computation graph; a loss function calculation unit to calculate a total sum of losses based on a predetermined loss function and the distances between branches; and a branch update unit to update weighting parameters in the learning object branches based on the total sum of losses in a state where the computation graph to be used has been modified into the second computation graph. [2] Model learning device according to claim 1, further comprising a branch visualization unit to visualize the feature generated by each of the plurality of branches. [3] Model learning device according to claim 1 or 2, wherein the distances between branches each comprise distances between each of the features and the feature of a predetermined target branch in addition to the distances between the features that are generated by each of the plurality of branches. [4] Model learning device according to one of claims 1 to 3, further comprising an input unit via which an operation for inputting identification information via the fixed branches is performed. [5] Model learning device according to any one of claims 1 to 4, further comprising a branch addition unit to add a new branch to the learning model. [6] Model learning device according to any one of claims 1 to 4, further comprising a branch removal unit to remove a fixed branch from the learning model. [7] Model learning device according to any one of claims 1 to 5, further comprising a branch selection unit to select the learning object branches from the plurality of branches on the basis of previously generated correct response data. [8] Model learning procedure to be carried out by a model learning device which performs transfer learning with respect to a learning model stored in a memory, wherein the model learning procedure comprises: a step of selecting fixed branches as branches to be excluded from learning objects, from a multitude of branches included in the learning model; a step of modifying a computation graph to be used into a first computation graph that uses the multitude of branches, and a second computation graph that uses learning object branches obtained by excluding the fixed branches from the multitude of branches; a step of calculating distances between branches, including distances between features generated by each of the multitude of branches, in a state where the computation graph to be used has been modified into the first computation graph; a step of calculating a total sum of losses based on a predetermined loss function and the distances between branches; and a step of updating weighting parameters in the learning object branches based on the total sum of losses in a state where the computation graph to be used has been modified into the second computation graph. [9] Model learning program that causes a computer to perform transfer learning with respect to a learning model stored in memory, wherein the model learning program causes the computer to execute: a step of selecting fixed branches as branches to be excluded from learning objects, from a multitude of branches included in the learning model; a step of modifying a computation graph to be used into a first computation graph that uses the multitude of branches, and a second computation graph that uses learning object branches obtained by excluding the fixed branches from the multitude of branches; a step of calculating distances between branches, including distances between features generated by each of the multitude of branches, in a state where the computation graph to be used has been modified into the first computation graph; a step of calculating a total sum of losses based on a predetermined loss function and the distances between branches; and a step of updating weighting parameters in the learning object branches based on the total sum of losses in a state where the computation graph to be used has been modified into the second computation graph.

Images