Information processing device, information processing method, program, storage medium, and model

Abstract

To integrate weights corresponding to a common section of a plurality of recognition models so as to increase, as mulch as possible, recognition accuracy of recognition tasks to be integrated in distributed learning.SOLUTION: An information processing apparatus is configured to: update, for each of a plurality of recognition models, weights using corresponding training data; evaluate recognition accuracy for each of the recognition models; set, based on the evaluation of the recognition accuracy, a parameter for integrating the weights corresponding to a common layer between the recognition models; and integrate, based on the set parameter, the weights corresponding to the common layer.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, a program, and a storage medium. [Background technology]

[0002] There are technologies that allow machines such as computers to learn and recognize the content of data such as images or sounds. One such technology for learning and executing recognition processing is the neural network (NN), and multi-layer NNs with a large number of layers are also called deep NNs (DNNs).

[0003] One efficient method for training multi-layer neural networks is distributed learning (as described in Non-Patent Document 1, for example), which involves performing deep learning in a distributed manner across multiple computing nodes connected via a communication network. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Jeffrey Dean,et al.,“Large Scale Distributed Deep Networks”,NIPS 2012 (4687),1223-1231 [Non-Patent Document 2] V Mnih, et al., “Human-level control through deep reinforcement learning”, Nature 518 (7540), 529-533 [Overview of the project] [Problems that the invention aims to solve]

[0005] However, in a learning model having weights shared among a plurality of recognition models such as distributed learning, it is difficult to appropriately set how to integrate the weights of the shared part. Even in the case of performing distributed learning for a single task to integrate the weights of the shared layer (common layer) of the recognition models, generally, since the progress of learning differs for each calculation node, if the weights of a sufficiently learned model and an insufficient model are averaged and integrated, the recognition accuracy may deteriorate.

[0006] An object of the present invention is to integrate weights corresponding to a shared part of a plurality of recognition models so that the discrimination accuracy of recognition tasks to be integrated in distributed learning is improved as much as possible.

Means for Solving the Problem

[0007] In order to achieve the object of the present invention, for example, an information processing apparatus according to an embodiment includes the following configuration. That is, for each of a plurality of recognition models, a first update means for updating weights using corresponding learning data, an evaluation means for evaluating recognition accuracy for each of the recognition models, and based on the evaluation of the recognition accuracy, a first setting means for setting parameters for integrating weights corresponding to a common layer between the recognition models, and an integration means for integrating weights corresponding to the common layer based on the parameters.

Effect of the Invention

[0008] Integrate the weights corresponding to the common layer of a plurality of recognition models so that the discrimination accuracy of recognition tasks to be integrated in distributed learning is improved as much as possible.

Brief Description of the Drawings

[0009] [Figure 1] A block diagram showing an example of the functional configuration of an information processing apparatus according to Embodiment 1. [Figure 2] A block diagram showing an example of the detailed functional configuration of an evaluation unit according to Embodiment 1. [Figure 3] A block diagram showing an example of the detailed functional configuration of a setting unit according to Embodiment 1. [Figure 4] A diagram illustrating the process of generating a set of recognition models according to Embodiment 1. [Figure 5] A flowchart showing an example of the process for setting integrated parameters according to Embodiment 1. [Figure 6] A table showing an example of parameters determined according to the evaluation of recognition accuracy in Embodiment 1. [Figure 7] A block diagram showing an example of the detailed functional configuration of the evaluation unit related to an application example of Embodiment 1. [Figure 8] A table showing an example of parameters determined according to the evaluation of recognition accuracy in Embodiment 1. [Figure 9] A block diagram showing an example of the detailed functional configuration of the decision unit related to an application example of Embodiment 1. [Figure 10] A block diagram showing an example of the detailed functional configuration of the decision unit related to an application example of Embodiment 1. [Figure 11] A schematic diagram showing an example of a GUI presented to the user according to Embodiment 1. [Figure 12] A block diagram showing an example of the functional configuration of the information processing device according to Embodiment 2. [Figure 13] A block diagram showing an example of the hardware configuration according to Embodiment 3. [Figure 14] A diagram illustrating a multi-task recognition model that uses distributed learning. [Modes for carrying out the invention]

[0010] The embodiments will be described in detail below with reference to the attached drawings. Note that the following embodiments do not limit the invention as defined in the claims. While the embodiments describe multiple features, not all of these features are essential to the invention, and the features may be combined in any way. Furthermore, in the attached drawings, identical or similar configurations are given the same reference numerals, and redundant descriptions are omitted.

[0011] [Embodiment 1] In Embodiment 1, multiple mathematical models (recognition models) that learn and execute the purpose of recognition processing (recognition task) have a shared portion that shares some of their weights with each other, and distributed learning is performed in which each model learns its weights separately while integrating through the sharing of the shared portion. Since each recognition model has a common layer (hereinafter also referred to as the "common layer"; "shared portion" and "shared layer" are synonymous with "common layer"), such distributed learning is possible. Hereinafter, the multiple recognition models that are learned by the information processing device according to this embodiment will be referred to as recognition models A to C, and the corresponding recognition tasks will be referred to as recognition tasks A to C, but the number and types of recognition models and recognition tasks are not limited in this way. Also, the objects to be recognized by each recognition task may be different, or the same task may be included.

[0012] The information processing device 1 according to this embodiment updates the weights of each of the multiple recognition models using the corresponding training data and evaluates the recognition accuracy of each of those recognition models. Next, based on the evaluation of the recognition accuracy, the information processing device 1 sets parameters for integrating the weights corresponding to the common parts between the recognition models, and integrates the multiple recognition models so that they share the common parts by integrating the weights corresponding to the common parts based on the set parameters. Hereinafter, when simply referred to as "recognition accuracy," it refers to the recognition accuracy of the recognition target by the recognition model.

[0013] Figure 1 is a block diagram showing an example of the functional configuration of the information processing device 1 according to this embodiment. The information processing device 1 performs distributed learning using recognition model A corresponding to recognition task A and recognition model B corresponding to recognition task B. The information processing device 1 includes a learning unit 102, a recognition model 103, and an evaluation unit 104 corresponding to recognition model A, and a learning unit 107, a recognition model 108, and an evaluation unit 109 corresponding to recognition model B. The information processing device 1 also includes a setting unit 111, an integration unit 112, and an update unit 113.

[0014] Figure 14 is a schematic diagram illustrating the processing in the information processing device 1 according to this embodiment, which is a multi-task recognition model that performs distributed learning using multiple recognition models. Here, a multi-task recognition model is shown that includes recognition models A to C, which perform processing on the non-shared layers A to C corresponding to recognition tasks A to C, after the image input to the information processing device 1 has been processed by a shared layer. An example of the distributed learning process that integrates the weights corresponding to the shared layer among the recognition models will be described later with reference to Figures 3 to 5.

[0015] In this embodiment, the objects to be recognized by the recognition task are not particularly limited. For the sake of explanation, recognition task A is described as a person detection task that detects people, and recognition task B is described as a road area division task that divides and recognizes a road area, but as mentioned above, these recognition tasks may be the same task. Such a multi-task recognition model, which learns different tasks, can be used, for example, in an autonomous driving system for automobiles to reduce collisions with people. Although two models, recognition model A and recognition model B, are shown in this illustration for the sake of explanation, the information processing device 1 may learn three or more recognition tasks and integrate them.

[0016] The learning unit 102 trains the recognition model 103 using the learning model 101 for learning recognition task A (human detection). Hereinafter, the recognition models, including recognition models A and B, are composed of a multilayer neural network (NN) and output recognition results of the object to be recognized in response to inputs such as images or features. Training a multilayer NN to recognize the object to be recognized can be done using known techniques, and a detailed explanation will be omitted.

[0017] The evaluation unit 104 evaluates the recognition accuracy of the recognition model 103 using the evaluation data 105. The evaluation data 105 includes image data for performing the recognition task and ground truth data corresponding to that image data. Here, the evaluation unit 104 can evaluate the recognition accuracy by comparing the recognition result of the recognition model 103 with the ground truth data included in the evaluation data 105. In this embodiment, the evaluation unit 104 evaluates the recognition accuracy at predetermined intervals and transmits the evaluation result to the setting unit 111. The predetermined interval at which the evaluation unit 104 performs the evaluation can be set arbitrarily, and here it is set using the number of learning iterations or the learning time.

[0018] The learning unit 107, the recognition model 108, and the evaluation unit 109 can perform the same processing as the learning unit 102, the recognition model 103, and the evaluation unit 109, except that the recognition task is different; therefore, redundant explanations will be omitted.

[0019] The evaluation process performed by the evaluation unit 104 (or evaluation unit 109) will be described below with reference to Figure 2. Figure 2 is a block diagram showing an example of the detailed functional configuration of the evaluation unit 104 according to this embodiment. The evaluation unit 104 includes an accuracy calculation unit 201, a target acquisition unit 202, and an accuracy comparison unit 203.

[0020] The accuracy calculation unit 201 compares the output result of a recognition task executed by inputting image data from the evaluation data 105 into the recognition model 103 with the ground truth data from the evaluation data 105, and calculates the recognition accuracy. In this embodiment, the recognition accuracy can be any comparable measure that represents the accuracy in the recognition process, such as the value of the loss function for the evaluation data of the detected region, the accuracy rate, the recall rate, or the precision rate. Hereinafter, the accuracy rate of the detected region will be used as the recognition accuracy.

[0021] The target acquisition unit 202 acquires the target accuracy for the accuracy calculated by the accuracy calculation unit 201. Here, the target accuracy is the target value of the recognition accuracy of the recognition model required to perform the recognition task, and it is assumed that the user can set it as desired in advance.

[0022] The accuracy comparison unit 203 evaluates the recognition accuracy of the recognition model 103 by comparing the recognition accuracy calculated by the accuracy calculation unit 201 with the target accuracy acquired by the target acquisition unit 202. In this embodiment, the accuracy comparison unit 203 outputs the difference between the accuracy values ​​of the recognition accuracy and the target accuracy as the recognition accuracy evaluation (evaluation value, Δ). However, the evaluation value may be output in any way as long as it is possible to evaluate the value of the recognition accuracy relative to the target accuracy. For example, the accuracy comparison unit 203 may output the ratio of the recognition accuracy to the target accuracy (recognition accuracy / target accuracy) as the evaluation value.

[0023] The evaluation data 110 in recognition task B is basically structured the same as evaluation data 105, except for the difference in the object to be recognized. As described above, recognition task B in this embodiment is a road area division task, which divides an image (background) into sub-regions and recognizes whether each divided region is a road or not. In recognition task B, the image is divided into grid-like sub-regions at regular intervals (for example, 1-pixel intervals or 3-pixel intervals), and recognition is performed on whether each sub-region is a road or not. That is, the evaluation data 110 includes information indicating the correct label for each sub-region as ground truth data.

[0024] The integration unit 112 sets parameters (integration parameters) for integrating the weights corresponding to the shared parts between the recognition models, based on the recognition accuracy evaluation of the recognition model 103 output by the evaluation unit 104 and the recognition accuracy evaluation of the recognition model 108 output by the evaluation unit 109. Next, the integration unit 112 integrates the recognition models based on the set parameters. A detailed explanation of the integration parameters will be given later, but for example, the integration ratio (weights of the weighted average) when integrating each model is used as an integration parameter.

[0025] Next, the integration process performed by the integration unit 112 will be explained with reference to Figure 3. Figure 3 is a block diagram showing an example of the detailed functional configuration of the integration unit 112 according to this embodiment. The integration unit 112 includes a set generation unit 301 and a determination unit 302.

[0026] The set generation unit 301 generates a set of multiple elements, each set containing a recognition model to be integrated. In this embodiment, when the elements of a set are a, b, and c, this set is denoted as {a, b, c}. A set of such a set (list) is denoted as, for example, [{a}, {a, b}, {c}]. Here, when integrating N recognition models, the set generation unit 301 may generate a list of sets of all combinations of recognition models with 2, 3, ..., N elements. Hereafter, such a set of recognition models may simply be referred to as a "set".

[0027] Figure 4 is a schematic diagram of the generation of a set of recognition model combinations by the set generation unit 301 (404) according to this embodiment. The set generation unit 404 first outputs a combination of recognition models with two elements for the three recognition models A401, B402, and C403. In the example in Figure 4, the set generation unit 404 generates {recognition model A, recognition model B} as set 405, {recognition model B, recognition model B} as set 406, and {recognition model A, recognition model C} as set 407. Next, the set generation unit 404 outputs a combination of recognition models with three elements. In the example in Figure 4, the set generation unit 404 generates {recognition model A, recognition model B, recognition model C} as set 408.

[0028] Next, with reference to Figure 5, the process of setting integrated parameters by the determination unit 302 will be explained. Figure 5 is a flowchart showing an example of the process of setting integrated parameters of the recognition model performed by the determination unit 302.

[0029] In S501, the setting unit 111 obtains the comparison results of the recognition accuracy of each recognition model by the evaluation unit 104. In S502, the determination unit 302 obtains a list of recognition model sets generated by the set generation unit 301. In the example shown in Figure 1, the list of recognition model sets obtained is only {recognition model A, recognition model B}.

[0030] S503 to S507 are loop processes that calculate integration parameters for each set of recognition models included in the list. In S503, the decision unit 302 selects one set of recognition models from the list of recognition model sets to be processed. In S504, the decision unit 302 determines whether the set selected in S503 is a set for which integration parameters are to be set. If it is a set for which integration parameters are to be set, the process proceeds to S505; otherwise, the process proceeds to S507.

[0031] In this embodiment, it is assumed that whether or not integration parameters are set for each set is predetermined, but the sets for which integration parameters are set may be determined based on predetermined conditions. For example, the determination unit 302 may determine the sets for which integration parameters are set according to the integration schedule, such as whether the integration is an odd-numbered or even-numbered integration. For example, if the integration is an odd-numbered integration, integration parameters may be set for {recognition model A, recognition model B} and {recognition model A, recognition model C}. Also, for example, if the integration is an even-numbered integration, integration parameters may be set for {recognition model B, recognition model C} and {recognition model A, recognition model B, recognition model C}.

[0032] Here, the integration parameter is a parameter for integrating the weights corresponding to the shared parts of each recognition model. That is, the integration unit 112 in this embodiment integrates the weights corresponding to the shared parts of recognition model A and the weights corresponding to the shared parts of recognition model B using the integration ratio determined by the determination unit 302, thereby sharing the shared parts of each recognition model. Assuming that the integration parameter in this embodiment is, for example, the integration ratio of the weights of the shared parts between recognition models, the integration ratios of recognition task A and recognition task B are set to αA and α B This is how it is written. In this case, for example, α A =α B Therefore, the weight of the shared portion after integrating recognition task A and recognition task B will be given by the average of the weights corresponding to the shared portion of each recognition model. α A >α B By increasing the integration ratio of one of the models, the weight of the recognition model with the higher ratio will contribute more significantly to the weight of the shared portion of the integration.

[0033] In S505, the decision unit 302 sets the integration parameters using the recognition accuracy comparison results obtained in S501. In this embodiment, the decision unit 302 may set the integration parameters for each recognition model by correcting the integration ratio used in the previous integration according to the value of the comparison results. For example, the decision unit 302 can refer to the comparison results and make a correction to decrease the previous integration ratio if the recognition accuracy has achieved the target accuracy, and make a correction to maintain or increase the previous integration ratio if the target accuracy has not been achieved. Here, the decision unit 302 may record the recognition accuracy at the time of the previous integration, and if the recognition accuracy has not achieved the target accuracy, it may keep the integration ratio the same if the recognition accuracy has improved compared to the previous time (before integration), and increase the integration ratio if the recognition accuracy is lower than or equal to the previous time. If the integration ratio falls below a predetermined threshold (for example, 1.0), it may be rounded to that threshold.

[0034] The method for correcting the integration ratio in this way is not particularly limited as long as it reflects the results of the recognition accuracy comparison. For example, a predetermined amount of correction may be made according to the difference between the recognition accuracy and the target accuracy.

[0035] Figure 6 shows an example of the correspondence of the multiplier used by the determination unit 302 according to this embodiment when correcting the integration ratio of each recognition model according to the evaluation value (Δ), which is the difference between the recognition accuracy and the target accuracy. In Figure 6, the multiplier multiplied by the integration ratio at the time of the previous integration according to the value of Δ is shown in the table. If the recognition accuracy exceeds the target (Δ>0), the multiplier is set to less than 1 so that it decreases compared to the previous integration ratio. Also, if the recognition accuracy is less than the target (Δ<0), the multiplier is set so that the integration ratio increases according to that value. In this way, by increasing the integration ratio of recognition accuracy with low recognition accuracy, it becomes possible to gradually increase the reflection ratio of the learning results of new tasks that have progressed to a certain extent.

[0036] In this example, if the value of Δ is lower than the threshold (Δ < -50), the recognition model has not learned much, and increasing the integration ratio at this stage would have a negative impact on the integration, so the ratio is set to less than 1. In this way, the integration ratio may be reduced when the evaluation value is smaller than a predetermined threshold.

[0037] Furthermore, the decision unit 302 may learn the integration ratio using reinforcement learning and estimate the integration ratio; this example will be described later in Application Example 3. Alternatively, the decision unit 302 may present the user with the results of the recognition accuracy comparison and then use the integration ratio entered by the user; this example will be described later in Application Example 4.

[0038] In S506, the decision unit 302 sets the integration ratio calculated in S505 for the set acquired as the processing target in S503. In S507, the decision unit 302 determines whether or not there are any sets that are not being processed. If there are sets that are not being processed, the process returns to S503; otherwise, the process ends. Through this process, the integration parameters necessary for integration are set for each set of recognition models.

[0039] The integration unit 112 integrates and outputs the weights corresponding to the shared parts between the respective recognition models using the integration parameters set by the setting unit 111. In this embodiment, the weights corresponding to the shared parts of the recognition task A and the recognition task B are w A , w B . Then, the weight w reduce of the shared part after integration is calculated, for example, by the following formula (1). w reduce = (α A × w A + α B × w B ) / (α A + α B ) Formula (1)

[0040] Also, the integration unit 112 may record the weights corresponding to the shared parts of each recognition model that have been integrated in the past, and correct the weights to be integrated using the recorded weights. For example, as the weight of the recognition model A to be integrated, the weight (w A best ) corresponding to the shared part at the time when the recognition accuracy was the highest in the past (predetermined) period is used to correct w A , and the corrected weight w A ’ may be integrated. According to such processing, it is possible to perform the integration of the recognition models, and improve the recognition accuracy of the recognition tasks that have not reached the target accuracy while maintaining the recognition accuracy reflected from the recognition models that have already reached the target accuracy.

[0041] For example, using the above-mentioned w A best , w reduce may be calculated by the following formula (2). Here, β is a parameter set with 0 ≤ β ≤ 1, and can be arbitrarily set by the user according to the desired reflection rate. w A ’ = (1 - β) × w A + β × w A best w reduce = (α A × w A ’ + α B × w B ) / (αA +α B ) Formula (2)

[0042] Furthermore, each recognition model may integrate the weights corresponding to the shared portion and then correct the weights corresponding to the shared portion for each recognition model. For example, the weights of the shared portion shared by each recognition model w may be calculated according to equation (3) below. reduce The weights w corresponding to the corrected common part, which are corrected in recognition model A and recognition model B respectively, are applied. A,reduce and w B,reduce You may calculate this. You may also integrate the corrected weights as the weights corresponding to the shared parts. Here, for recognition model B, the weight corresponding to the shared part at the time when the recognition accuracy was highest in the past period is w B best This is how it is written. Furthermore, γ and δ are parameters defined within the range of 0 to 1, and can be arbitrarily set by the user according to the desired reflection rate. w A,reduce = (1-γ) × w reduce +γ×w A best w B,reduce = (1-δ) × w reduce +δ×w B best Formula (3)

[0043] The update unit 113 updates the weights corresponding to the shared layer (common layer) of each recognition model using the weights of the shared layer (common layer) after integration of each recognition model calculated by the integration unit 112. Here, the update unit 113 sets the weights corresponding to the shared part of each recognition model to w reduce Update as follows.

[0044] Furthermore, before the update unit 113 updates each recognition model, the evaluation unit 104 may determine whether updating the weights corresponding to the shared part will degrade the recognition accuracy of the recognition model (compared to before integration), and may refrain from performing the update if it determines that the recognition accuracy will degrade. For example, if the update causes the recognition accuracy of recognition task A, which has already reached the target accuracy, to fall below the target accuracy (or its recognition accuracy to degrade), and the recognition accuracy of recognition task B, which has not reached the target accuracy, to degrade, then the update may not be performed. This decision can be made for each recognition model whose recognition accuracy has degraded, or if the recognition accuracy of all recognition models has degraded, then no update may be performed on the entire recognition model. With such a configuration, it is possible to avoid updates that degrade recognition accuracy.

[0045] With this configuration, it becomes possible to integrate weights corresponding to the common parts of multiple recognition models in a distributed learning environment, thereby maximizing the accuracy of the recognition tasks being integrated.

[0046] Furthermore, as described above, this embodiment makes it possible to perform distributed learning of a multi-task discrimination model that includes multiple different recognition tasks. When performing distributed learning of multiple tasks, if recognition tasks that have been sufficiently trained and recognition models that you want to add are mixed together, simply averaging the weights and integrating them may degrade the discrimination accuracy for the trained tasks. As shown in this embodiment, even when integrating multi-task discrimination models, by evaluating the discrimination accuracy of each discrimination model and performing the integration using parameters based on the evaluation results, it is possible to integrate them while maintaining the discrimination accuracy of the original models as much as possible.

[0047] In this embodiment, any recognition task that is generally performed using a recognition model may be used. For example, the recognition task may be a classification task that identifies objects in an image, an object detection task that detects the position of an object, or a region segmentation task that extracts a specific area. Alternatively, for example, a tracking task that tracks a subject, a motion recognition task that identifies the movements of a subject, an organ detection task that detects organs such as the face of a subject, a posture estimation task that estimates the posture of a subject, or a depth estimation task that estimates the depth of an object may be used as the recognition task.

[0048] [Application Example 1] In Embodiment 1, the recognition accuracy after the weight update was evaluated for each recognition model performing distributed learning using the target accuracy of the recognition model. However, the method for evaluating the recognition accuracy after the weight update is not limited to the example described above. For example, the evaluation unit 104 may store the past recognition accuracy before the current weight update and evaluate the recognition accuracy after the weight update relative to the previous recognition accuracy. With such a configuration, it is possible to learn for additional recognition tasks while maintaining the accuracy of a recognition model that has been sufficiently trained in the past. Here, the previous recognition accuracy may be, for example, the most recent recognition accuracy, the recognition accuracy selected from past history (for example, the accuracy at the time with the highest accuracy), or a value calculated from the recognition accuracy over a predetermined period. For example, the evaluation unit 104 may calculate the average value of the recognition accuracy over a predetermined period for each recognition model and use this average value to evaluate the recognition accuracy. In the following explanation, recognition task A is a sufficiently trained recognition model, and recognition task B is a pre-prepared recognition model that has not been sufficiently trained (especially a new one).

[0049] Figure 7 is a block diagram showing an example of the functional configuration of the evaluation unit 104 in this application example. The evaluation unit 104 is basically capable of performing the same processing as described with reference to Figure 2. In the following, in each application example, including this application example, the information processing device 1 has basically the same configuration as described with reference to Figures 1 to 6 and is capable of performing the same processing, so redundant explanations will be omitted.

[0050] The accuracy calculation unit 701 calculates the recognition accuracy in the same way as the accuracy calculation unit 201. The accuracy storage unit 702 stores the recognition accuracy previously calculated by the accuracy calculation unit 701. The accuracy comparison unit 703 uses the past recognition accuracy stored in the accuracy storage unit 702 to evaluate the recognition accuracy calculated by the accuracy calculation unit 701. The accuracy comparison unit 703 can, for example, use the past recognition accuracy stored in the accuracy storage unit instead of the target accuracy acquired by the target acquisition unit 202, and perform the same processing as the accuracy comparison unit 203. Here, the accuracy comparison unit 703 may use the recognition accuracy at the time when the recognition accuracy was highest, or it may use the recognition accuracy at the most recent integration point as the past recognition accuracy.

[0051] The determination unit 302 performs the same processing as described with reference to Figure 5, except for the processing in S501 and S505, and can set the integrated parameters. In this application example, in S501, the accuracy comparison unit 703 obtains the recognition accuracy evaluation result.

[0052] In this application example, in S505, the decision unit 302 calculates the integration parameters using the recognition accuracy evaluation results obtained in S501. For recognition model A, if the decision unit 302 determines, for example, that the recognition accuracy is equal to or better than that of the comparison target (in this case, the evaluation value is 95% or higher (105% or lower)), it may maintain or decrease the integration ratio of that recognition model. Alternatively, if the decision unit 302 determines, for example, that the recognition accuracy of recognition model A is inferior to that of the comparison target (in this case, the evaluation value is less than 95%), it may increase the integration ratio of that recognition model. The same applies to recognition model B.

[0053] The process of increasing or decreasing the integration ratio can be performed in the same way as described with reference to Figure 6, so the explanation is omitted here. In many cases, when adding a new recognition task, even if the integration ratio is low, the recognition accuracy for that task will improve in the initial stages of learning. Therefore, the integration ratio for the new task may be adjusted to be small so as not to lower the accuracy of other previously trained recognition tasks. This process allows for the training of a multi-task recognition model that learns about the added recognition task while maintaining as much as possible the recognition accuracy reflected from the recognition model that has been sufficiently trained in the past.

[0054] [Application Example 2] In Embodiment 1, the integration ratio was determined as the integration parameter when integrating the weights corresponding to the common parts of each recognition model. However, as an integration parameter indicating the degree of integration, it is also conceivable to use, for example, the number of training iterations when training the recognition model before the integration is performed once.

[0055] In this application example, the decision unit 302 determines the number of training iterations instead of the integration ratio of the recognition model based on the comparison results output by the accuracy comparison unit 203. The decision unit 302 may set the integration parameters for each recognition model by correcting the number of training iterations used in the previous integration according to the value of the comparison result. For example, the decision unit 302 can refer to the comparison result and correct the number of iterations to decrease from the previous number if the recognition accuracy has achieved the target accuracy, and correct the number of iterations to maintain or increase the previous number if the target accuracy has not been achieved. Here, the decision unit 302 may record the recognition accuracy at the time of the previous integration, and if the recognition accuracy has not achieved the target accuracy, it may keep the number of iterations the same if the recognition accuracy has improved since the previous time, or increase the number of iterations if the recognition accuracy is lower than or equal to the previous time. The process of determining the integration parameters (number of iterations) based on the comparison result can be performed in the same way as when using the integration ratio, and Figure 8 discloses a table showing the number of iterations corresponding to the evaluation value.

[0056] In this application example, the integration unit 112 integrates each recognition model by taking the average of the weights corresponding to the shared parts, for example, as shown in equation (4) below. However, it is also possible to perform calculations as exemplified in equation (1), change the number of training iterations, and then perform a weighted average of each weight. Alternatively, after calculating the weights of the shared parts by averaging as in equation (4), corrections to the weights corresponding to the shared parts may be performed in each recognition model, as in equation (3). w reduce =(w A + w B ) / 2 formula (4)

[0057] This process makes it possible to differentiate the number of training iterations when integrating the shared parts of a distributed learning recognition model, based on an evaluation of its recognition accuracy. Therefore, for example, if there are differences in learning difficulty for each recognition task, and the number of training iterations required to converge sufficiently differs significantly, it becomes possible to set the number of training iterations more appropriately.

[0058] [Difference 3] Furthermore, the decision unit 302 may learn the integration ratio using reinforcement learning and estimate the integration ratio. Figure 9 is a block diagram showing an example of the functional configuration of the decision unit 302 in this application example. In this application example, the decision unit 302 includes a learning unit 901 and an estimation unit 902.

[0059] The learning unit 901 is equipped with an agent, which is a reinforcement learning model. Here, the agent observes the recognition accuracy of each recognition model at the current time as a state and estimates an integration ratio (integration parameter) according to the state. The agent also sets an integration ratio, integrates the weights corresponding to the common parts of multiple recognition models according to the set integration ratio, and learns by repeatedly and randomly trying the accuracy obtained by evaluating the integrated recognition model as a reward. Specific reinforcement learning methods for problems with many states are exemplified, for example, in Non-Patent Literature 2. Furthermore, the agent's learning can be stabilized by providing the agent with prior knowledge of processing such as S505 by the decision unit 302.

[0060] The estimation unit 902 estimates the integration ratio by inputting the recognition accuracy of each recognition model to the agent trained by the learning unit 901. With this configuration, the integration ratio can be appropriately set through reinforcement learning, and the weights of the shared layers (common layers) of multiple recognition models can be efficiently integrated.

[0061] [Differentiation Example 4] Alternatively, the determination unit 302 may present the user with the results of the recognition accuracy comparison and then use the integrated ratio entered by the user.

[0062] Figure 10 is a block diagram showing an example of the functional configuration of the determination unit 302 in this modified example. In this application example, the determination unit 302 has a precision display unit 1001 and a parameter input unit 1002. Figure 11 also shows an example of a graphical user interface (GUI) displayed by the display unit (not shown) of the information processing device 1 in this modified example.

[0063] In Figure 11, the accuracy display unit 1101 displays the target accuracy 1102 of recognition task A and the accuracy 1103 of recognition task A relative to the number of training iterations on the display frame 1101 of the display screen 11. In the example in Figure 11, the progression of recognition accuracy according to the number of training iterations is displayed as a graph, but the method is not limited to this form as long as similar information can be presented to the user, for example, in a table. In this modified example, when the user enters an integration ratio in the integration ratio input frame 1100 of recognition task A and presses the setting button 1104, the set integration ratio will be reflected when the weights corresponding to the next sharing are integrated. In addition, the recognition accuracy 1107 of recognition task B relative to the number of training iterations is displayed on the display frame 1105, and when the user presses the setting button 1109, the integration ratio entered in the integration ratio input frame 1108 will be reflected when the next sharing occurs.

[0064] In Figure 11, when the user places the mouse pointer 1110 over the display frame 1101 corresponding to recognition task A on the display screen 11, a pop-up 1111 is displayed that shows the accuracy and integration ratio for the corresponding number of training iterations. This allows the user to adjust the current integration ratio by referring to the integration ratio set in the past.

[0065] With this configuration, users can determine the integration ratio as needed while checking the recognition accuracy. Therefore, users can heuristically set the integration parameters while checking the learning progress and perform the integration according to those settings.

[0066] [Embodiment 2] In Embodiment 1, a single device, the information processing device 1, evaluated the updated recognition accuracy for each of the multiple recognition models and integrated the weights corresponding to the shared parts. However, the processing for each of these recognition models does not need to be performed on a single device; for example, learning may be performed on separate devices for each recognition model to be integrated, and the final integration may be performed on the information processing device 1. The information processing device 12 in this embodiment is connected to terminals A to Z, each of which learns a recognition model, and evaluates the recognition accuracy according to the learning of those recognition models. Hereinafter, when referring to one or more of these terminals A to Z without distinction, they will simply be referred to as "terminals".

[0067] Figure 12 is a block diagram showing an example of a system configuration including the information processing device 12 according to this embodiment. The information processing device 12 comprises a communication unit 1213, a collection unit 1214, an evaluation unit 1215, a setting unit 1217, an integration unit 1218, and an update unit 1219, and is connected to terminals A to Z by wired or wireless means. In the example in Figure 12, each terminal is exemplified as terminal A1201, terminal B1205, and terminal Z1209. Terminals A1201, B1205, and Z1209 each comprise a recognition model 1204 and a learning unit 1203, a recognition model 1208 and a learning model 1207, and a recognition model 1212 and a learning unit 1211, respectively. In this embodiment, terminals A to Z correspond to recognition models A to Z, including recognition models A to C of Embodiment 1, respectively, but there are no particular limitations on the number of these terminals or the recognition tasks to be learned.

[0068] The learning unit in each terminal according to this embodiment learns the recognition model using the learning data in the same way as the learning unit 102 in Embodiment 1. These learning units also transmit the weights corresponding to the shared units that have been updated by learning to the communication unit 1213 of the information processing device 1. The communication unit 1213 transmits and receives data between the information processing device 12 and each terminal, such as each recognition model or information indicating the progress of the recognition model on the terminal.

[0069] The collection unit 1214 collects information about the recognition model from each terminal. In this embodiment, the collection unit 1214 obtains the recognition accuracy of the recognition model from each terminal, and a comparison of the obtained recognition accuracy with the respective target accuracy (stored as evaluation data 1216 in Figure 12) is performed within the information processing device 1. However, the terminal may retain information indicating the target accuracy, and the collection unit 1214 may collect the comparison results after the terminal has performed a comparison between the recognition accuracy and the target accuracy.

[0070] Here, the collection unit 1214 can collect information about the recognition model after confirming that each terminal has completed the execution of learning at predetermined intervals (which may differ for each terminal). In this case, each terminal performs learning at predetermined intervals and transmits the recognition accuracy to the information processing device 1, and then temporarily suspends learning until it receives the shared weights used for updating from the update unit 1219.

[0071] The evaluation unit 1215 evaluates the recognition accuracy of each recognition model collected by the collection unit 1214. In this embodiment, the evaluation unit 1215 can use the evaluation data 1216 to evaluate the recognition accuracy of the recognition models in the same way as in Embodiment 1 and output an evaluation value.

[0072] The setting unit 1217, the integration unit 1218, and the update unit 1219 perform the same processing as the setting unit 111, the integration unit 112, and the update unit 113 of Embodiment 1, so their description is omitted here.

[0073] This configuration allows for the training of multiple recognition models in a distributed manner using multiple terminals, and enables the integration of shared components while suppressing situations where training becomes difficult due to insufficient computer memory on a single device. Furthermore, by configuring the system to evaluate recognition accuracy on each terminal, it becomes possible to process data when it is difficult for the information processing device 12 to communicate or retain some of the data from each terminal (for example, from the standpoint of personal information, confidentiality, or data capacity).

[0074] [Embodiment 3] In the embodiments described above, each processing unit shown in Figure 1, for example, is implemented by dedicated hardware. Some or all of the processing units of the information processing device 1 may be implemented by a computer. In this embodiment, at least some of the processing according to each embodiment described above is executed by a computer.

[0075] Figure 13 shows the basic configuration of a computer. In Figure 13, the processor 1301 is, for example, a CPU, which controls the operation of the entire computer. The memory 1302 is, for example, RAM, which temporarily stores programs and data. The computer-readable storage medium 1303 is, for example, a hard disk or CD-ROM, which stores programs and data long-term. In this embodiment, the programs that realize the functions of each part, which are stored in the storage medium 1303, are read into the memory 1302. Then, the processor 1301 operates according to the programs in the memory 1302, thereby realizing the functions of each part.

[0076] In Figure 13, the input interface 1304 is an interface for acquiring information from an external device. The output interface 1305 is an interface for outputting information to an external device. The bus 1306 connects the above-mentioned parts and enables data exchange.

[0077] (Other examples) The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0078] The invention is not limited to the embodiments described above, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, claims are attached to disclose the scope of the invention. [Explanation of symbols]

[0079] 102: Learning unit, 103: Recognition model, 104: Evaluation unit, 111: Configuration unit, 112: Integration unit, 113: Update unit

Claims

1. a first updating means for updating weights for each of a plurality of recognition models using corresponding training data; evaluation means for evaluating the recognition accuracy of each of the recognition models; a first setting means for setting a parameter for integrating weights corresponding to common layers between the recognition models based on the evaluation of the recognition accuracy; aggregating means for aggregating weights corresponding to the common layer based on the parameters; An information processing device comprising:

2. The information processing apparatus according to claim 1 , wherein each of the plurality of recognition models performs the same recognition task.

3. The information processing apparatus according to claim 1 , wherein any one of the plurality of recognition models performs a recognition task different from that of the other recognition models.

4. a second setting means for setting a target of recognition accuracy by the recognition model; The information processing apparatus according to claim 1 , wherein the evaluation means evaluates the recognition accuracy according to the target.

5. 5. The information processing apparatus according to claim 4, wherein the second setting means sets the target as a predetermined value or sets the target based on the recognition accuracy of the recognition model evaluated by the evaluation means before updating the weights.

6. The information processing apparatus according to claim 1 , wherein the parameters include an integration ratio when integrating the weights into a weighted average.

7. 2. The information processing apparatus according to claim 1, wherein the parameters include the number of times that the first update means repeats updating the weights using the learning data while the integration is performed once by the integration means.

8. 7. The information processing apparatus according to claim 6, wherein the first setting means increases the integration ratio when the recognition accuracy has deteriorated compared to before the integration by the integration means, and maintains or decreases the integration ratio when the recognition accuracy has improved compared to before the integration by the integration means.

9. 8. The information processing device according to claim 7, wherein the first setting means increases the number of iterations when the recognition accuracy has deteriorated compared to before the integration by the integration means, and maintains or decreases the number of iterations when the recognition accuracy has improved compared to before the integration by the integration means.

10. further comprising a presentation means for presenting the recognition accuracy to a user, 2. The information processing apparatus according to claim 1, wherein said first setting means sets said parameters by acquiring input of said parameters from said user.

11. a generation unit for generating a trained model that performs reinforcement learning to estimate the parameters from the recognition accuracy; The information processing device according to claim 1 , wherein the first setting means sets the parameters based on the trained model.

12. a first correction means for correcting a weight corresponding to the common layer for each of the recognition models based on the recognition accuracy; 2. The information processing apparatus according to claim 1, wherein said integrating means integrates said plurality of recognition models so as to share said common layer by integrating the weights corrected by said first correcting means.

13. 13. The information processing apparatus according to claim 12, wherein the first correction means corrects the weight corresponding to the common layer based on the recognition accuracy that was most highly evaluated in a predetermined period for each of the recognition models.

14. 2. The information processing apparatus according to claim 1, further comprising: second updating means for updating weights corresponding to a common layer of each of the plurality of recognition models using the weights integrated by the integration means.

15. a determination unit for determining whether or not updating the weights by the second update unit will degrade the recognition accuracy of the recognition model; 15. The information processing apparatus according to claim 14, wherein said second updating means does not perform said updating when it is determined that said recognition accuracy will deteriorate.

16. 15. The information processing apparatus according to claim 14, further comprising: second correction means for correcting, for each of the recognition models, a weight corresponding to a common layer updated by the second update means based on the recognition accuracy of the recognition model.

17. a third setting means for setting a set to which two or more of the recognition models belong; The information processing apparatus according to claim 1 , wherein said integration means integrates said recognition models belonging to said set.

18. 2. The information processing apparatus according to claim 1, wherein the first updating means updates the weights for each of the plurality of recognition models that operate on separate terminals.

19. 2. The information processing apparatus according to claim 1, wherein a recognition task is performed using an image as an input, using a recognition model obtained by integrating a plurality of recognition models by said integrating means.

20. 2. The information processing device according to claim 1, wherein the recognition model performs one or more of a classification task for identifying an object in an image, an object detection task for detecting the position of an object, or an area segmentation task for extracting a specific area, a tracking task for tracking a subject, an action recognition task for identifying the action of a subject, a period detection task for detecting organs such as the face of a subject, a posture estimation task for estimating the posture of a subject, or a depth estimation task for estimating the depth of an object.

21. updating weights for each of the plurality of recognition models using corresponding training data; evaluating the recognition accuracy for each of the recognition models; setting a parameter for integrating weights corresponding to common layers between the recognition models based on the evaluation of the recognition accuracy; integrating weights corresponding to the common layer based on the parameters; An information processing method comprising:

22. A program for causing a computer to function as each of the means of the information processing device according to any one of claims 1 to 20.

23. A storage medium storing the program according to claim 22.

24. A model in which a recognition model having a common layer and one or more model-specific layers configured to perform recognition in cooperation with said common layer is integrated with at least one other recognition model so as to share said common layer, The weights corresponding to the common layer are obtained by integrating the weights corresponding to the common layers of each of the recognition models based on parameters set according to the recognition accuracy of each of the recognition models.

Images