Information processing device, control method of information processing device, and program

Abstract

To achieve such multitask learning that a plurality of recognition tasks are learned in a more preferred manner than before.SOLUTION: A learning section A102 and a learning section B107 each learn a shared layer of a muti-layer neural network in each of a recognition task A and a recognition task B in order to handle the recognition task A and the recognition task B in the muti-layer neural network. An integration section 112 integrates the weight of the shared layer in each of the recognition task A and the recognition task B on the basis of integration methods different from each other.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, a control method for an information processing device, and a program. [Background technology]

[0002] Various recognition technologies, such as a technology for recognizing an object in an image, have been proposed, and such recognition technologies are applied to various devices (for example, imaging devices such as digital cameras). Note that in this disclosure, for convenience, the purpose of a process for recognizing an object is also referred to as a recognition task, and a mathematical model for learning and executing a recognition task is also referred to as a recognition model. Neural networks (NNs) are known as a technology for achieving such recognition tasks. In particular, NNs, also known as Deep Convolutional Neural Networks (DCNNs), have been attracting attention in recent years due to their high performance in recognition processing.

[0003] In the recognition phase of DCNN, data is input to the trained DCNN, and the data is processed sequentially by the trained recognition model in each layer, and the recognition result is output from the output layer.

[0004] Also, in the DCNN, a method is known in which the value of a filter for convolutional calculation (hereinafter also referred to as a convolution filter) and the weight of a fully connected layer are shared among multiple recognition tasks to process multiple recognition tasks (hereinafter also referred to as multitask recognition). As a learning method for such a neural network (hereinafter also referred to as multitask learning), a technology for efficiently learning to improve the recognition performance of multiple recognition tasks is being actively discussed. Patent Document 1 discloses an example of a technology for realizing multitask learning. Patent Document 2 discloses a technology that enables efficient search for a suitable NN structure in multiple multi-layer NNs that execute multiple tasks.

[0005] Moreover, as a method for efficiently training a multi-layer NN, there is a method using distributed learning. Non-Patent Document 1 discloses a training method for a multi-layer NN using distributed learning. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2019-192009 A [Patent Document 2] Patent No. 6750854 [Non-patent literature]

[0007] [Non-Patent Document 1] Caruana, R. (1997) “Multitask learning, Machine learning” 28(1), 41-75 Summary of the Invention [Problem to be solved by the invention]

[0008] On the other hand, in the multi-task distributed learning method, as distributed learning continues, the weights in the shared layer may fall into a local solution, and further learning may result in small changes in the weights, meaning that the learning of the model (NN) may not progress smoothly.

[0009] In view of the above problems, an object of the present invention is to realize multi-task learning for learning a plurality of recognition tasks in a more suitable manner. [Means for solving the problem]

[0010] An information processing device according to the present invention is characterized in having, in order to handle a plurality of recognition tasks in a multilayer neural network, a learning means for learning a shared layer of the multilayer neural network for each of the plurality of recognition tasks, and an integration means for integrating weights of the shared layer for each of the plurality of recognition tasks based on mutually different integration methods. Effect of the Invention

[0011] According to the present invention, it is possible to realize multi-task learning for learning a plurality of recognition tasks in a more suitable manner. [Brief description of the drawings]

[0012] [Figure 1] FIG. 2 is a diagram illustrating an example of a functional configuration of an information processing device. [Diagram 2] FIG. 2 is a diagram showing an example of a learning image. [Diagram 3] FIG. 11 is a diagram illustrating an example of an operation of an integration unit. [Figure 4] FIG. 2 is a diagram illustrating an example of a functional configuration of an information processing device. [Diagram 5] 11 is a flowchart showing an example of processing by an information processing device. [Figure 6] FIG. 2 is a diagram illustrating an example of a functional configuration of an information processing device. [Figure 7] FIG. 2 is a diagram illustrating an example of a functional configuration of an information processing device. [Figure 8] 11 is a flowchart showing an example of processing by an information processing device. [Figure 9] FIG. 2 is a diagram illustrating an example of a hardware configuration of an information processing device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functional configurations are denoted by the same reference numerals, and redundant description will be omitted.

[0014] <Distributed multitask learning method for neural networks> First, in order to more clearly explain the features of the embodiments of the present disclosure, an overview of a multitask distributed learning method for a neural network will be described.

[0015] Recognition tasks that are the subject of learning include, for example, a face detection task to detect the area of ​​a human face from an image, as well as a variety of other recognition tasks, such as an object category recognition task to distinguish the category (human, animal, vehicle, etc.) of an object (subject) in an image, a tracking task to search for and track a specific subject, and a scene type recognition task to distinguish the type of scene (city, mountain, coast, etc.).

[0016] As mentioned above, neural networks (NNs) are known as an example of a technology for realizing recognition tasks. Among such neural networks, those that have multiple layers connected in multiple stages are also called deep NNs (DNNs). In addition, particularly deep convolutional NNs are also called deep convolutional neural networks (hereinafter, DCNNs). DCNN has a network structure in which, in each layer, convolution processing is performed on the output from the previous layer and then output to the next layer. The final layer is the output layer that represents the recognition results. Each layer is provided with multiple filters (kernels) for convolution operations. Layers close to the output layer generally have a fully connected structure like a normal NN, rather than connections based on convolution. Filters for convolution operations are also referred to as convolution filters in the following description.

[0017] In the learning phase of the DCNN recognition model, the values ​​of the convolutional filters and the connection weights of the fully connected layer are learned from supervised data using methods such as back propagation (BP). In addition, in the recognition phase of DCNN, data is input to the trained DCNN, and the data is processed sequentially by the trained recognition model in each layer, and the recognition result is output from the output layer. Also known are multitask recognition, in which the values ​​of convolutional filters and the weights of a fully connected layer are shared among multiple recognition tasks to process multiple recognition tasks, and multitask learning, which is a learning method for this.

[0018] As an example of a multi-task learning method, there is a method for efficiently learning by setting learning parameters based on the evaluation results of a recognition model corresponding to multiple recognition tasks. Here, the learning parameters indicate the magnitude of the learning rate in the learning of a multi-layer NN and the ratio of losses calculated for each recognition task during learning.

[0019] Another method for efficiently training a multi-layer NN is to use distributed learning. In distributed learning, communication is mainly performed to calculate the average across all computing nodes for the gradients calculated on each computing node and the weights of the recognition model. Distributed learning is generally used to speed up the learning of a single task, but it can also be applied to multi-task learning. For example, one computing node can be assigned to one recognition task, and each node can learn independently and in parallel, and the weights of the shared layer of each recognition model can be integrated between nodes and updated at a specified frequency.

[0020] On the other hand, in a multi-task distributed learning method, as distributed learning continues, the weights of the shared layer may fall into a local solution, and further learning may result in small changes in the weights. For example, consider a case where the integration of the shared layers of each recognition model is achieved by weighted averaging. Normally, when there is a difference in recognition accuracy between recognition tasks, integrating the weighted average weights with a difference in the value of the weighted average will result in shared layer weights that reduce the accuracy difference. However, as described above, even if weighted averaging is performed when the change in the weights of the shared layers of the recognition models is small, there are cases where the shared layer weights do not reduce the accuracy difference between recognition tasks.

[0021] In consideration of the above circumstances, this disclosure proposes an example of a technology that effectively improves model accuracy without falling into a local solution when learning is continued in multi-task learning, in which multiple recognition tasks are learned.

[0022] <First embodiment> A first embodiment of the present disclosure will be described below. In this embodiment, a plurality of mathematical models (recognition models) that learn and execute the purpose (recognition task) of recognition processing have a shared part that shares a part of weights with each other, and distributed learning is performed in which integration is performed by sharing the shared part while learning the weights separately. In this embodiment, such distributed learning is possible because each recognition model has a common layer (hereinafter also referred to as a shared layer). Hereinafter, the multiple recognition models trained by the information processing device according to this embodiment will be referred to as recognition models A and B, respectively, and the corresponding recognition tasks will be referred to as recognition tasks A and B. However, the above definitions are merely for the sake of convenience, and the number and types of recognition models and recognition tasks are not limited to these. Furthermore, the recognition targets for each recognition task may be different, or the same task may be included.

[0023] The information processing device 1 according to this embodiment performs learning for each of a plurality of recognition models using learning data of the corresponding recognition task, and updates the weights. Next, the information processing device 1 integrates the weights of the shared parts at regular intervals, thereby integrating the plurality of recognition models so as to share the shared parts. At this time, the information processing device 1 does not completely share the weights of the shared parts during learning, but performs integration by placing more importance on the weights of the shared parts of the recognition tasks being learned in each recognition task. Due to such characteristics, in this embodiment, the weights of the shared parts in each recognition model are not completely the same. In contrast, at the timing when completing the model, the weights of the shared layer are shared, so that each recognition model has the same shared part.

[0024] (Hardware configuration) An example of a hardware configuration of the information processing device 1 according to the present embodiment will be described with reference to FIG. The information processing device 1 includes a central processing unit (CPU) 1104, a random access memory (RAM) 905, and a read only memory (ROM) 906. The information processing device 900 also includes a hard disk drive (HDD) 907, an input unit 901, a display unit 902, and a network I / F unit 903. The input unit 901, the display unit 902, the network I / F unit 903, the CPU 904, the RAM 905, the ROM 906, and the HDD 907 are each connected to each other via a data bus 908 so as to be able to transmit and receive information to and from each other.

[0025] The CPU 904 performs various controls by reading out a control computer program stored in the ROM 906, expanding the control computer program in the RAM 905, and executing the control computer program. The RAM 905 is used as a work area and a temporary storage area for expanding the programs executed by the CPU 904. The HDD 907 is a storage area for storing image data, setting parameters, various programs, etc. Instead of the HDD 907 or together with the HDD 907, other auxiliary storage devices such as non-volatile memories typified by SSDs (Solid State Drives) may be used.

[0026] The network I / F unit 903 is a network I / F for connecting the information processing device 900 to a network such as the Internet, and transmits and receives various information to and from other devices via the network. Image data and the like received from an external device via the network I / F unit 903 are transmitted and received to the CPU 904, RAM 905, and ROM 906 via a data bus 908.

[0027] The input unit 901 is realized by, for example, input devices such as a keyboard, a mouse, a touch panel, etc. The display unit 902 is realized by, for example, a display device such as a liquid crystal display.

[0028] For example, by CPU 904 executing a program stored in ROM 906 or HDD 907, the functional configuration described below with reference to Figures 1, 3, 4, 6, and 7, and the processing described below with reference to Figures 5 and 8 are realized.

[0029] (Functional configuration) An example of the functional configuration of an information processing device according to this embodiment will be described with reference to Fig. 1. The information processing device 1 performs distributed learning using a recognition model A corresponding to a recognition task A and a recognition model B corresponding to a recognition task B. The information processing device 1 has a learning unit A102 and a recognition model A103 corresponding to the recognition model A, and a learning unit B107 and a recognition model B108 corresponding to the recognition model B. The information processing device 1 also has an integration unit 112 and an update unit 113. In this embodiment, the recognition target recognized by the recognition task is not particularly limited. In this embodiment, for convenience, the description will be given assuming that recognition task A is a car detection task for detecting an automobile, and recognition task B is a person detection task for detecting a person. In addition, in the example shown in FIG. 1, for convenience, two models, recognition model A and recognition model B, are illustrated, but the number of recognition tasks and recognition models is not limited. In other words, as long as there are multiple recognition tasks to be learned, the number is not particularly limited, and for example, the information processing device 1 may learn about three or more recognition tasks and integrate each of them.

[0030] The learning data A101 includes an image for learning the recognition task A (hereinafter also referred to as a learning image) and correct answer data indicating a detection target captured in the image. An example of the learning image and correct answer data is shown in Fig. 2. In the learning image shown in Fig. 2, an automobile is captured as a detection target 201. A bounding box 202 is set so as to surround the detection target 201, and information regarding the bounding box 202 (for example, center coordinates, width, and height) is saved as correct answer data. A learning unit A102 learns a recognition model A103 using learning data A101 for learning a recognition task A (vehicle detection). Hereinafter, a recognition model including recognition models A and B is configured with a multi-layer neural network (NN) and outputs a recognition result of a recognition target in response to an input image. In the recognition model A103, an image is first input to a sharing unit, where it is processed, and a feature value is output as a result of the processing. The feature value is input to a car detection unit that detects cars, where it is processed, and a car recognition result is output as a result of the processing. Note that since publicly known technology can be applied to the learning of the multi-layer NN that recognizes the recognition target, detailed explanations are omitted.

[0031] A learning unit B107 and a recognition model B108 for recognition task B can perform processing similarly to the learning unit A102 and the recognition model A103 for recognition task A, except for the difference in the recognition task, and therefore a duplicated description will be omitted. Similarly, the learning data B106 is basically configured in the same way as the learning data A101, except for the difference in the recognition target.

[0032] The integrating unit 112 integrates the weights corresponding to the shared parts of the recognition model A 103 and the recognition model B 108. Note that the operation of the integrating unit 112 will be described in detail later with reference to FIG. The update unit 113 updates the weights corresponding to the shared layer of each recognition model with the weights of the shared layer after integration of each recognition model calculated by the integration unit 112.

[0033] (Description of the operation of the integration unit) The integration unit 112 executes a process of integrating weights corresponding to the shared parts of the recognition model A 103 and the recognition model B 108 at a timing of a predetermined integration period (hereinafter, also referred to as an integration process). The integration process by the integration unit 112 will be described below with reference to Fig. 3. Note that the operation of the integration process differs between when the model is saved and when learning is continued.

[0034] As shown in Fig. 3, recognition model A103 has a sharing unit 120 and a car detection unit 121. The sharing unit 120 extracts a feature amount indicating a feature of a recognition target in an input image from the input image, and outputs the feature amount. The feature amount output from the sharing unit 120 is input to the car detection unit 121, and the car detection unit 121 outputs a recognition result for a car. Recognition model B109 also has a sharing unit 123 and a person detection unit 124. That is, recognition model B109 outputs a recognition result for a person in the input image.

[0035] First, the common weighted average shown in Fig. 3(A) will be described. This method is a method for calculating a weighted average commonly applied to two or more recognition tasks when integrating the weights of the shared layer targeted at each of the two or more recognition tasks, and is applied, for example, when the model is saved. When finally evaluating the performance of the recognition model, it is desirable to evaluate it using the model saved by this method. In this case, the weights of the shared part are integrated by weighted average according to the relational expression shown below as Equation (1).

[0036] w =(αA ×wA+αB ×wB) / (αA +αB ) Equation (1)

[0037] In formula (1), wA and wB are weights of the shared part before the integration of the recognition models A103 and B108. w is the weight of the shared part after the integration of the recognition models A103 and B108, and is the same for either the recognition models A103 or B108. αA and αB are weighted average weights used when creating the weights. In this embodiment, for convenience, αA is set to 0.4 and αB is set to 0.6.

[0038] Next, we will explain the task-specific average shown in Figure 3(B). This method is a method for calculating a weighted average that is different between two or more recognition tasks and is applied individually to each of the two or more recognition tasks when integrating the weights of the shared layer targeted at each of the two or more recognition tasks, and is applied, for example, when learning is continued. In this case, the weights of the shared part are integrated by weighted averaging according to the relational expressions shown below as equations (2) and (3).

[0039] wA′=(βA ×wA+βB ×wB) / (βA +βB ) Equation (2) wB′=(βA′×wA+βB′×wB) / (βA′+βB′) Equation (3)

[0040] In formula (2) and formula (3), wA' and wB' are the weights of the shared part after the integration of the recognition models A103 and B108. βA and βB are the weighted average weights when creating the weights of the recognition model A103. βA' and βB' are the weighted average weights when creating the weights of the recognition model B109. Each weighted average weight can be set arbitrarily, but it is usually more desirable to set it so that the weight of the recognition task being learned is larger than that at the time of saving the model. In other words, it is preferable that the weighted average weight for the recognition task to be integrated in the shared layer is controlled to be equal to or greater than the weighted average weight of the other recognition task different from the recognition task. As a more specific example, when calculating the weight wA' of the shared part after the integration of the recognition model A103, it is preferable that the weighted average weight βA for the recognition task A to be integrated in the shared layer is controlled to be equal to or greater than the weighted average weight βB for the recognition task B. As another example, when calculating the weight wB' of the shared part after integration of the recognition model B109, the weighted average weight βB' for the recognition task B to be integrated in the shared layer may be controlled to be equal to or greater than the weighted average weight βA' for the recognition task A. In this embodiment, for the sake of convenience, the values ​​used are 0.6 for βA, 0.4 for βB, 0.3 for βA', and 0.7 for βB'.

[0041] (Effects of the first embodiment) In the information processing device 1 according to the present embodiment, as described above, when learning is continued, it is not necessary to unify the weights of the shared parts to be completely the same shared parts, and it is acceptable for the shared parts to have different weights. However, since it is desirable to ultimately match the weights of the shared parts when completing the model, it is desirable to set the weights so that they are shared to some extent with each other, rather than making them completely different weights even during learning. During learning, by prioritizing learning for the recognition task being learned, recognition model A103 is more likely to change to a weight that easily grasps the features of recognition task A, and recognition model B108 is more likely to change to a weight that easily grasps the features of recognition task B. This learning method is particularly useful when distributed learning progresses and the change in the weight of the shared part becomes minute. Specifically, if the shared part is made to have exactly the same weight, a weighted average weight that takes into account both recognition tasks must be used, which results in small weight changes and makes learning difficult to progress. In contrast, by using the method of this embodiment, it is possible to use the weight changes of each recognition model in the next learning without averaging them with a weighted average, making learning more likely to progress. In each embodiment described below, the differences from the first embodiment will be mainly described, and unless otherwise specified below, each embodiment will be considered to be the same as the first embodiment.

[0042] <Second embodiment> A second embodiment of the present disclosure will be described below. In the first embodiment, the weights of the shared parts are always integrated using the same weighted average weight during learning. In contrast, in the present embodiment, an example will be described in which the weighted average weight when integrating the weights of the shared parts is changed according to the learning status of the neural network. Specifically, the information processing device according to the present embodiment evaluates the recognition accuracy during learning to determine the learning effect, and changes the weighted average weight when integrating the weights of the shared parts according to the result.

[0043] (Example of functional configuration) An example of a functional configuration of an information processing device 2 according to this embodiment will be described with reference to Fig. 4. Note that in this embodiment, the description will focus on parts that differ from the information processing device 1 according to the first embodiment described with reference to Fig. 1, and detailed descriptions of parts that are substantially similar to the information processing device 1 will be omitted. The information processing device 2 according to this embodiment differs from the information processing device 1 according to the first embodiment in that it includes an evaluation unit 104A corresponding to recognition model A, an evaluation unit B109 corresponding to recognition model B, and an integrated control unit 111.

[0044] The evaluation unit A104 compares the output result of a recognition task executed by inputting a learning image included in the evaluation data A105 to the recognition model A103 with the correct answer data included in the evaluation data A105, and measures the recognition accuracy. The evaluation data A105 has the same structure as the training data A101 in that it is composed of training images and answer data corresponding to those images. Note that the training images themselves are different from those included in the training data A101. The evaluation unit A104 evaluates the recognition accuracy of the recognition model A103 at a predetermined interval, and transmits the evaluation result to the integrated control unit 111. Note that the predetermined interval at which the evaluation unit A104 performs the evaluation can be set arbitrarily, and here, it is set using the number of learning iterations or the learning time. As the recognition accuracy according to the present embodiment, any comparable measure that expresses the accuracy in the recognition process, such as the accuracy rate, recall, or precision of the evaluation data of the detected area, can be used. Hereinafter, for convenience, the recall of the detected area will be used as the recognition accuracy. The evaluation unit B 109 performs the same processing as the evaluation unit A 104, except that the recognition task is different. The operation of the integrated control unit 111 will be described separately later.

[0045] (Explanation of the operation of the integrated control unit) The integration control unit 111 receives the recall calculated from the evaluation unit A104 and the evaluation unit B109, and controls the method of integrating weights that correspond to the shared part between the recognition model A103 and the recognition model B108, which is implemented by the integration unit 112. An example of the processing of the integration control unit will be described with reference to Fig. 5, focusing in particular on the processing related to the control of the above-mentioned weight integration method implemented by the integration unit 112.

[0046] When the integrated control unit 111 receives the recall rates from the evaluation unit A 104 and the evaluation unit B 109, it starts a series of processes shown in FIG. In S201, the integrated control unit 111 judges whether or not to terminate the learning of the recognition model A103 and the recognition model B108. The condition for terminating the learning of the recognition model A103 and the recognition model B108 can be arbitrarily determined by the person who implements the learning. As a specific example, the condition may be determined so that the learning is terminated when the number of times of learning reaches a predetermined number. Also, even in a situation where the learning is continued, if the shared layer performs evaluation with a common weight between the recognition model A103 and the recognition model B108, it is judged that the learning is terminated once. When the integration control unit 111 determines in S201 that the learning is to be ended, the process proceeds to S203. In S203, the integration control unit 111 controls the integration unit 112 to use the common weighted average (formula (1)) described with reference to FIG. 3(A). On the other hand, if the integrated control unit 111 determines in S201 not to end the learning (to continue the learning), the process proceeds to S202.

[0047] In S202, the integrated control unit 111 determines whether or not the absolute value of the difference between the recall rate of the recognition task A and the recall rate of the recognition task B is equal to or greater than a predetermined threshold value. If the integration control unit 111 determines in S202 that the absolute value of the difference between the recall rate of recognition task A and the recall rate of recognition task B is equal to or greater than the threshold, the process proceeds to S204. In S204, the integration control unit 111 controls the integration method in the integration unit 112 to be the task-specific average (Equations (2) and (3)) described with reference to FIG. 3B. On the other hand, if the integration control unit 111 determines in S202 that the absolute value of the difference between the recall rate of recognition task A and the recall rate of recognition task B is less than the threshold, the process proceeds to S203. In this case, the integration control unit 111 controls the integration unit 112 to use the common weighted average as the integration method. In this embodiment, for convenience, the threshold is set to 10%.

[0048] The integrated control unit 111 may use a predetermined weighted average weight, or may determine the weighted average weight at the time of task-specific averaging based on the recall obtained from the evaluation unit A104 and the evaluation unit B109. By applying such control, it becomes possible to change the weighted average weight depending on the degree of difficulty in the recognition task, and it is expected to have an advantageous effect of making it possible to progress in learning the difficult recognition task more quickly. As a specific example, when the task-specific averaging is applied in S204, it is also possible to obtain the weighted average weight for the recognition task with the lower recall by applying the relational expression shown below as Equation (4).

[0049] βA′′=RecallAll-RecallA / RecallAll βB′′=RecallAll-RecallB / RecallAll RecallAll = RecallA+RecallB Equation (4)

[0050] In the above formula (4), RecallA indicates the recall of recognition task A. Also, RecallB indicates the recall of the most recent integration of recognition task B. In the recognition task with the higher recall, the same weighted average weight as that in the common weighted average is applied.

[0051] (Effects of the second embodiment) When the difference in recall is large, it is assumed that the learning of one of the recognition tasks is not progressing smoothly and the improvement in accuracy is small. In such a case, by applying task-specific averaging, it is possible to foster a recognition model that is strong in the recognition task being learned and promote learning. On the other hand, when the difference in recall is small, it is assumed that the learning of both recognition tasks is progressing smoothly. In such a case, learning is performed so that the shared layer of the recognition model has the same weights, without forcibly promoting the learning of each recognition task.

[0052] (Modification of the second embodiment: Loss-based control) In the above-described embodiment, the integrated control unit 111 changes the control based on the evaluation results from the evaluation unit A 104 and the evaluation unit B 109, but the control may be changed based on the loss during learning instead of the evaluation results. By applying the loss during learning instead of the evaluation results from the evaluation unit A 104 and the evaluation unit B 109 to the control by the integrated control unit 111, it becomes possible to perform well-balanced learning for each of the recognition tasks A and B without preparing evaluation data.

[0053] With reference to Fig. 6, an example of the functional configuration of the information processing device 3 according to this modification will be described, focusing on parts that are particularly different from the information processing device 2 described with reference to Fig. 4. Compared to the information processing device 2 shown in Fig. 4, the information processing device 3 shown in Fig. 6 does not have configurations equivalent to the evaluation unit A104 and evaluation unit B109, and instead has a loss calculation unit A115 and a loss calculation unit B116.

[0054] The loss acquisition unit A115 acquires the loss during learning calculated by the learning unit A102 at a predetermined interval, and transmits the loss to the integrated control unit 111. The learning unit A102 calculates the loss from the difference between the output of the recognition model A103 and the correct answer data of the learning data A101. For example, a square sum error function or a cross entropy error function is used as a loss function for the loss calculation. Similarly, for the recognition task B, the loss acquisition unit B116 acquires the loss during learning calculated by the learning unit B107 at a predetermined interval, and transmits the loss to the integrated control unit 111. 5, a control is applied to switch to the task-specific averaging method when the difference in recall is large. In contrast, in this modification, a control is applied to switch the integrating method in the integrating unit 112 to the task-specific averaging method when the absolute value of the loss difference is equal to or greater than a predetermined threshold, and switch to the common weighted averaging method when the absolute value of the loss difference is less than the threshold.

[0055] <Third embodiment> A third embodiment of the present disclosure will be described below. In the second embodiment, the learning effect is judged based on the recognition accuracy (recall rate). In contrast, in the present embodiment, an example of the case where the learning effect is judged based on the change in the weight of the recognition model will be described. By applying such control, it becomes possible to check whether or not the change in the weight of the shared part, which corresponds to the case where the task-specific averaging is more useful, is small without using evaluation data.

[0056] An example of the functional configuration of the information processing device 4 according to this embodiment will be described with reference to Fig. 7. Note that in this embodiment, the description will focus on the parts that are different from the information processing device 2 according to the second embodiment described with reference to Fig. 4, and detailed description of the parts that are substantially similar to the information processing device 2 will be omitted. Specifically, compared to the information processing device 2 shown in Fig. 4, the information processing device 4 according to this embodiment does not have configurations equivalent to the evaluation unit A104 and evaluation unit B109, and instead has a weight acquisition unit A117 and a weight acquisition unit B118.

[0057] The weight acquisition unit A117 acquires the weights corresponding to the shared parts of the recognition model A103, and transmits the weights to the integrated control unit 111. The acquisition and transmission of the weights is performed, for example, for each learning. Similarly, the weight acquisition unit B117 acquires the weights corresponding to the shared parts of the recognition model B108, and transmits the weights to the integrated control unit 111.

[0058] The integration control unit 111 controls the method of integrating weights corresponding to the shared parts of the recognition model A103 and the recognition model B108, which is implemented by the integration unit 112. An example of the processing by the integration control unit 111 will be described with reference to Fig. 8, focusing on the control of the method of integrating weights corresponding to the shared parts of the recognition model A103 and the recognition model B108. Note that the series of processing shown in Fig. 8 differs from the series of processing shown in Fig. 5 in the processing of S302. Therefore, the series of processing shown in Fig. 8 will be described with a focus on the parts that are different from the series of processing shown in Fig. 5.

[0059] In S302, the integrated control unit 111 calculates the difference between the weight vectors of the shared parts of the recognition models A103 and B108, and obtains the L2 norm of the difference between the weight vectors as the difference between the weights in the recognition models A103 and B108. Then, the integrated control unit 111 determines whether the calculated difference between the weights is less than a predetermined threshold value. If the integration control unit 111 determines that the weight difference calculated in S302 is not less than the threshold (the weight difference is equal to or greater than the threshold), the process proceeds to S303. In this case, it is expected that the learning specialized for the recognition task is smoothly performed for each of the recognition models A103 and B108. Therefore, in S303, the integration control unit 111 controls the integration method in the integration unit 112 to be the common weighted average. On the other hand, if the integration control unit 111 determines that the difference between the weights calculated in S302 is less than the threshold, the process proceeds to S304. In this case, the recognition models A103 and B108 may not have been trained specifically for their respective recognition tasks, and the change in the weight of the shared part may be a smile. Therefore, the integration control unit 111 controls the integration method in the integration unit 112 to be the task-specific average.

[0060] By applying the above-described control, it becomes possible to control so that the task-specific average is applied when the change in the weight of the shared part becomes small, making the task-specific average more useful.

[0061] (Modification 1 of the third embodiment: Difference in weight from the most recent one) In the above-described embodiment, the integrated control unit 111 obtains the difference in weight of the shared part between the recognition models A103 and B108, and changes the control based on the weight difference. On the other hand, the difference between the weight of the shared part in the previous learning (for example, the most recent previous learning) and the current weight of the shared part in each recognition model may be applied to the change in control by the integrated control unit 111. In this case, each of the weight acquisition units A117 and B118 not only acquires the weight of the target recognition model, but also holds the previously acquired weight (the weight during the previous learning). Then, each of the weight acquisition units A117 and B118 transmits both the weight acquired for the current learning and the weight acquired during the previous learning to the integrated control unit 111.

[0062] 8 is executed for each recognition task, and in S302, the integrated control unit 111 calculates the difference between the weights acquired in the previous learning and the weights acquired for the current learning. Then, the integrated control unit 111 determines the method of integrating the weights of the shared parts for each recognition task according to the weight difference calculated for each recognition task. The threshold value of the difference in weights can be set arbitrarily. In this modification, for convenience, it is assumed that the threshold value of the difference between the weights of the initial recognition model and the weights of the recognition model immediately before the first integration is set to 10%. If the integration control unit 111 determines in S302 that the difference in weights is not less than the threshold (is equal to or greater than the threshold), the process proceeds to S303. In S303, the integration control unit 111 controls the integration unit 112 to use a common weighted average as the integration method. On the other hand, if the integration control unit 111 determines in S302 that the difference in the weights is less than the threshold, the process proceeds to S304. In S304, the integration control unit 111 controls the integration unit 112 to use task-specific averaging as the integration method.

[0063] By applying the above-mentioned control, it becomes possible to control so that the task-specific average is applied when the change in the weight of the shared part becomes small, which makes the task-specific average more useful. This is expected to have the effect of promoting learning in tasks where learning is difficult to progress.

[0064] (Modification 2 of the third embodiment: Selection of integration method for each layer of the shared layer) The shared layer of the recognition model is usually a multi-layer NN and is composed of multiple layers. In this context, for example, a weight integration method may be determined for each layer of the shared layer. In such a case, for example, the integration control unit 111 may obtain the L2 norm of the difference between the weight vectors for each layer as the weight difference for the weights of the shared layer received from the weight acquisition units A117 and B118 in S302. Then, the integration control unit 111 may determine the integration method of the integration unit 112 for each layer based on the obtained weight difference.

[0065] In this way, by changing the method of integrating weights for each layer of the shared layer, it is expected that learning can be controlled more finely and that learning will be promoted more effectively. It is generally known that among the layers of DCNN, simple and general features are extracted in shallow layers closer to the input layer, while complex and abstract features are extracted in deep layers closer to the output layer. Given this background, it is expected that learning will progress more easily by controlling the learning so that general features are learned by applying common-specific averaging to shallower layers, and applying task-specific averaging to learn abstract features for each recognition task to deeper layers.

[0066] <Other embodiments> The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0067] The disclosure of the present embodiment also includes the following configurations, methods, and programs. (Configuration 1) An information processing device, characterized in having: a learning means for learning a shared layer of the multilayer neural network for each of the multiple recognition tasks in a multilayer neural network; and an integration means for integrating weights of the shared layer for each of the multiple recognition tasks based on integration methods that are different from each other. (Configuration 2) The information processing device according to configuration 1, characterized in that the integration means integrates the weights of the shared layer based on a weighted average, and in the weighted average, a weighted average weight for a first recognition task to be integrated with the weights of the shared layer is controlled to be equal to or greater than a weighted average weight of a second recognition task different from the first recognition task. (Configuration 3) The information processing apparatus according to configuration 1 or 2, further comprising a control means for changing an integration method by which the integration means integrates weights of the shared layer targeted for each of the plurality of recognition tasks, into either an integration method that differs for each recognition task or an integration method common to the plurality of recognition tasks, depending on a learning status of the multilayer neural network by the learning means. (Configuration 4) The information processing apparatus according to configuration 3, further comprising a measurement means for measuring the recognition accuracy of each of the plurality of recognition tasks while the learning means is training the multilayer neural network, and the control means is configured to change the integration method used by the integration means to integrate the weights of the shared layer targeted for each of the plurality of recognition tasks, based on a measurement result of the recognition accuracy of each of the plurality of recognition tasks by the measurement means, to either an integration method that differs for each of the recognition tasks or an integration method common to the plurality of recognition tasks. (Configuration 5) The information processing device according to configuration 4, wherein the control means changes the integration method used by the integration means to integrate weights of the shared layer targeted for each of the plurality of recognition tasks to an integration method that differs for each of the plurality of recognition tasks when the difference in recognition accuracy between the plurality of recognition tasks is equal to or greater than a threshold, and changes the method to an integration method common to the plurality of recognition tasks when the difference in recognition accuracy is less than the threshold. (Configuration 6) The information processing device according to configuration 3, wherein the control means changes an integration method by which the integration means integrates weights of the shared layer targeted for each of the plurality of recognition tasks based on a loss during learning for each of the plurality of recognition tasks to either an integration method that differs for each recognition task or an integration method that is common to the plurality of recognition tasks. (Configuration 7) The information processing device according to configuration 6, wherein the control means changes the integration method used by the integration means to integrate weights of the shared layer targeted for each of the plurality of recognition tasks to an integration method that differs for each of the plurality of recognition tasks when a difference in loss during training between each of the plurality of recognition tasks is equal to or greater than a threshold, and changes the method to an integration method common to the plurality of recognition tasks when the difference in loss during training is less than a threshold. (Configuration 8) The information processing device described in Configuration 3, wherein the control means changes the integration method by which the integration means integrates the weights of the shared layer targeted for each of the multiple recognition tasks based on the weights of the shared layer for each of the multiple recognition tasks to either an integration method that differs for each recognition task or an integration method that is common to the multiple recognition tasks. (Configuration 9) The information processing device described in configuration 8, wherein the control means changes the integration method used by the integration means to integrate weights of the shared layer targeted for each of the multiple recognition tasks to an integration method that differs for each of the multiple recognition tasks when the difference in weights between each of the multiple recognition tasks is equal to or greater than a threshold, and changes the method to an integration method common to the multiple recognition tasks when the difference in weights is less than the threshold. (Configuration 10) The information processing device described in configuration 8, wherein the control means changes to a common integration method among the multiple recognition tasks when a difference between the shared layer weight in current learning and the shared layer weight in previous learning for each recognition task is equal to or greater than a threshold, and changes to a different integration method for each recognition task when a difference between the shared layer weight in current learning and the shared layer weight in previous learning for each recognition task is less than a threshold. (Configuration 11) The information processing device described in configuration 8, wherein the control means changes the integration method by which the integration means integrates weights of the shared layer targeted at each of the multiple recognition tasks for each layer of the shared layer to either an integration method that differs for each recognition task or an integration method that is common to the multiple recognition tasks. (Method 1) A method for controlling an information processing device, comprising: a learning step of learning a shared layer of a multilayer neural network for each of a plurality of recognition tasks in order to handle the plurality of recognition tasks in the multilayer neural network; and an integration step of integrating weights of the shared layer for each of the plurality of recognition tasks based on integration methods different from each other. (Program 1) A program for causing a computer to function as an information processing device, comprising: a learning means for learning a shared layer of a multilayer neural network for each of a plurality of recognition tasks in a multilayer neural network; and an integration means for integrating weights of the shared layer for each of the plurality of recognition tasks based on integration methods that are different from each other. [Explanation of symbols]

[0068] 1. Information processing device 112 Integration Department 102 Learning Section A 107 Learning Section B

Claims

1. a learning means for learning a shared layer of the multilayer neural network for each of a plurality of recognition tasks in order to handle the plurality of recognition tasks in the multilayer neural network; a synthesis means for synthesizing weights of the shared layer for each of the plurality of recognition tasks based on a synthesis method different from each other; An information processing device comprising:

2. 2. The information processing apparatus according to claim 1, wherein the integrating means integrates the weights of the shared layer based on a weighted average, and is controlled so that in the weighted average, a weighted average weight for a first recognition task to be integrated with the weights of the shared layer is equal to or greater than a weighted average weight of a second recognition task different from the first recognition task.

3. 2. The information processing apparatus according to claim 1, further comprising a control means for changing an integration method by which said integrating means integrates weights of said shared layer targeted for each of said plurality of recognition tasks, into either an integration method that differs for each recognition task or an integration method that is common to said plurality of recognition tasks, depending on a learning status of said multilayer neural network by said learning means.

4. a measuring means for measuring a recognition accuracy of each of the plurality of recognition tasks during training of the multilayer neural network by the training means, The control means changes an integration method used by the integration means to integrate the weights of the shared layer targeted for each of the plurality of recognition tasks, based on a measurement result of the recognition accuracy of each of the plurality of recognition tasks by the measurement means, to either an integration method that differs for each of the recognition tasks or an integration method that is common to the plurality of recognition tasks.

4. The information processing device according to claim 3.

5. The control means a method for integrating weights of the shared layer targeted for each of the plurality of recognition tasks by the integration means, When the difference in recognition accuracy between each of the plurality of recognition tasks is equal to or greater than a threshold, the integration method is changed to a different one for each of the recognition tasks; When the difference in the recognition accuracy is less than a threshold, the integration method is changed to a common integration method among the plurality of recognition tasks.

5. The information processing device according to claim 4.

6. 4. The information processing device according to claim 3, wherein the control means changes an integration method used by the integrating means to integrate the weights of the shared layer targeted for each of the plurality of recognition tasks, to either an integration method that differs for each recognition task or an integration method common to the plurality of recognition tasks, based on a loss during learning for each of the plurality of recognition tasks.

7. The control means a method for integrating weights of the shared layer targeted for each of the plurality of recognition tasks by the integration means, When a difference in the loss during training between each of the plurality of recognition tasks is equal to or greater than a threshold, the integration method is changed to a different one for each of the recognition tasks; When the difference in the loss during the learning is less than a threshold, the integration method is changed to a common integration method among the plurality of recognition tasks.

7. The information processing device according to claim 6.

8. 4. The information processing device according to claim 3, wherein the control means changes an integration method by which the integrating means integrates the weights of the shared layer targeted for each of the plurality of recognition tasks based on the weights of the shared layer for each of the plurality of recognition tasks to either an integration method that differs for each recognition task or an integration method that is common to the plurality of recognition tasks.

9. The control means a method for integrating weights of the shared layer targeted for each of the plurality of recognition tasks by the integration means, When the difference in the weights between the plurality of recognition tasks is equal to or greater than a threshold, the integration method is changed to a different one for each of the plurality of recognition tasks; If the difference between the weights is less than a threshold, the integration method is changed to a common integration method among the plurality of recognition tasks. The information processing device according to claim 8 .

10. The control means When a difference between the weights of the shared layer in the current learning and the weights of the shared layer in the previous learning for each of the recognition tasks is equal to or greater than a threshold, the integration method is changed to a common integration method among the plurality of recognition tasks; When a difference between the weights of the shared layer in the current learning and the weights of the shared layer in the previous learning for each of the recognition tasks is less than a threshold, the integration method is changed to a different one for each of the recognition tasks. The information processing device according to claim 8 .

11. 9. The information processing device according to claim 8, wherein the control means changes, for each layer of the shared layer, a merging method by which the merging means merges the weights of the shared layer targeted for each of the plurality of recognition tasks, to either a different merging method for each recognition task or a common merging method for the plurality of recognition tasks.

12. A method for controlling an information processing device, comprising: A learning step of learning a shared layer of the multilayer neural network for each of a plurality of recognition tasks in order to handle the plurality of recognition tasks in the multilayer neural network; a merging step of merging the weights of the shared layer for each of the plurality of recognition tasks based on a different merging method; 2. A method for controlling an information processing device, comprising:

13. Computer, a learning means for learning a shared layer of the multilayer neural network for each of a plurality of recognition tasks in order to handle the plurality of recognition tasks in the multilayer neural network; a synthesis means for synthesizing weights of the shared layer for each of the plurality of recognition tasks based on a synthesis method different from each other; A program for causing an information processing device to function, comprising:

Images