TRANSFER LEARNING FACILITY, TRANSFER LEARNING PROCESS AND TRANSFER LEARNING PROGRAM

Abstract

A transfer learning device comprises: a knowledge transfer learning unit (11) for performing knowledge transfer learning using learning data relating to a second data type different from a first data type based on a trained model relating to the first data type, and generating an intermediate model from the trained model; and a knowledge transfer progress degree determining unit (12; 12') for determining, in the generated intermediate model, through a multiple comparison test, whether or not a reference probability of learning data assigned a reference label is greater than probabilities of a plurality of individual learning data each assigned non-reference labels and not the reference label.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to a transfer learning technique. BACKGROUND TO THE STATE OF THE ART

[0002] When data of a particular modality (data type) is rehandled and there is a model asset that has been sufficiently trained for a modality other than that particular modality by machine learning, the knowledge related to that model asset can be transferred and utilized. Patent Literature 1 discloses a technique related to such knowledge transfer. More specifically, Patent Literature 1 discloses a learning method comprising: learning data assigned a label using a root mean squared error (RMSE) as a loss function, setting a condition that learning is terminated without waiting for the maximum number of epochs 50 when the loss does not decrease over five epochs, and using a mean absolute error (MAE) and a determination coefficient (R 2) as evaluation indices to assess the accuracy of a modality of a knowledge transfer goal. REFERENCE LISTPATENT LITERATURE

[0003] Patent literature 1: JP 2021 / 135726 A SUMMARY OF THE INVENTIONTECHNICAL PROBLEM

[0004] A method disclosed in Patent Literature 1 uses an evaluation index to evaluate the accuracy of a modality of a knowledge transfer target, so that it is possible to ensure the quality of knowledge transfer by setting an appropriate threshold with respect to the evaluation index.

[0005] However, in the conventional method described in Patent Literature 1 or the like which searches for the minimum loss using a loss function, the learning data is learned regardless of whether the learning data is sufficiently learned or not, and therefore there is a problem that a long learning time is required.

[0006] The present disclosure has been made to solve such a problem, and an object of the present disclosure is to provide a transfer learning technique which can reduce a learning time of transfer learning. SOLUTION TO THE PROBLEM

[0007] One aspect of a transfer learning device according to an embodiment of the present disclosure includes: a knowledge transfer learning unit for performing knowledge transfer learning using learning data relating to a second data type different from a first data type based on a trained model relating to the first data type, and generating an intermediate model from the trained model; a knowledge transfer progress degree determining unit for determining, in the generated intermediate model, through a multiple comparison test, whether or not a reference probability of learning data assigned a reference label is greater than probabilities of a plurality of individual learning data each assigned non-reference labels and not the reference label. ADVANTAGEOUS EFFECTS OF THE INVENTION

[0008] A transfer learning device according to embodiments of the present disclosure can shorten a learning time of transfer learning. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a diagram showing a configuration example of a transfer learning device according to Embodiment 1. Fig. 2A is a diagram showing a configuration example of hardware of the transfer learning device. Fig. 2B is a diagram showing a configuration example of hardware of the transfer learning device. Fig. Figure 3 is a diagram showing an overview of transfer learning. Fig. Figure 4A shows a probability transition graph representing a transition of a probability of a visible image according to an intermediate model generated by transfer learning. Fig. Figure 4B shows a probability transition graph representing a transition of a probability of a depth image according to the intermediate model generated by transfer learning. Fig. Figure 5A is a diagram showing a target region of a p-value table created for a visible image. Fig. Figure 5B is a diagram showing a target region of a p-value table created for a depth image. Fig. Figure 6A shows the p-value tables of the visible image and the depth image in S1. Fig. Figure 6B shows the p-value tables of the visible image and the depth image in S2. Fig. Figure 6C shows the p-value tables of the visible image and the depth image in S3. Fig. 7 is a flowchart illustrating a transfer learning method according to Embodiment 1. Fig. 8 is a diagram showing a configuration example of a transfer learning device according to Embodiment 2. Fig. Figure 9A shows a p-value table of the depth image. Fig. Figure 9B shows a p-value table of the depth image. Fig. Figure 9C shows a p-value table of the depth image. Fig. 10 is a flowchart illustrating a transfer learning method according to Embodiment 2. DESCRIPTION OF THE EMBODIMENTS

[0009] Embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. It should be noted that components designated by identical or similar reference numerals in the drawings have identical or similar components or functions, and redundant description of such components will be omitted. Embodiment 1. <konfiguration>

[0010] A transfer learning device 1 according to Embodiment 1 of the present disclosure will be described with reference to Fig. 1 to 6C. Fig. 1 is a diagram showing a configuration of the transfer learning device 1 according to Embodiment 1. The transfer learning device 1 is a device that performs transfer learning by transferring knowledge related to a transfer source (source) to a transfer target (target).

[0011] With reference to Fig. 3, for example, the transfer learning device 1 transfers the knowledge of a model 1 related to a domain of a visible image that has been sufficiently trained to determine what a target detected in the visible image is, and generates a model 1' that makes it possible to determine what a target detected in a depth image is. In other words, the transfer learning device 1 is a device that performs heterogeneous transfer learning whose feature space is different and whose task is identical.

[0012] In the “pre-knowledge transfer” phase, which Fig. As shown in Figure 3, model 1 is a trained model that has been sufficiently trained for the visible image so that it is possible to adequately determine what a specific target is based on a visible image VI obtained by imaging the specific target. In Fig. 3, the probabilities of correct labels of the visible images are marked in gray. Since the probabilities of correct labels of the visible image are maximal, an appropriate determination is made (in Fig. 3 marked by "o"). However, Model 1 cannot adequately determine what a specific target is from the depth image DI obtained by imaging the specific target. In Fig. 3, the probabilities of correct labels of the depth images are marked by hatching. Since the probabilities of the correct labels marked by hatching are not the maximum, an appropriate determination is not performed (indicated by "x" in Fig. 3).

[0013] In the phase "after the knowledge transfer", which is Fig. As shown in Figure 3, a model 1' is a trained model generated by transfer learning using a depth image based on the model 1. However, the model 1' cannot adequately determine what a specific target is from the depth image DI obtained by imaging the specific target. Since the probabilities of the correct labels indicated by the hatching are the maximum, an adequate determination is not performed (indicated by "o" in Fig. 3). On the other hand, the model 1' is trained for depth images, and therefore cannot adequately determine what a particular target is from the visible image VI obtained by imaging the particular target, as in Fig. 3. The probabilities of the correct labels of the visible images are not the maximum, and therefore an appropriate determination is not performed (indicated by "x" in Fig. 3). It should be noted that the degree of determinability of Model 1' for the domain of the knowledge transfer source changes depending on the degree of similarity between a domain of the knowledge transfer source and a domain of the knowledge transfer target.

[0014] In Fig. Figure 3 shows the phase "during knowledge transfer" between before knowledge transfer and after knowledge transfer, showing how a determination result of an intermediate model changes during knowledge transfer.

[0015] The transfer learning device 1 can generate a single model such as the model 1' based on a plurality of models 1 to 3 related to the domain of the knowledge transfer source.

[0016] In order to carry out the transfer learning described above, the transfer learning device 1 comprises, as shown in Fig. 1, a knowledge transfer learning unit 11, a knowledge transfer progress degree determination unit 12, a knowledge transfer repetition determination unit 13, and a knowledge transfer result output unit 14. The transfer learning device 1 is connected to a storage device 2 that stores data related to the knowledge of the knowledge transfer source, a storage device 3 that stores data related to knowledge of the knowledge transfer destination, and a storage device 4 that stores data related to knowledge of the knowledge transfer destination after the knowledge transfer, to form a transfer learning system.

[0017] The data stored in the storage device 2 is data from a trained model that has already been sufficiently trained. For example, data from the trained model is stored for a visible image to which a label has been assigned, wherein the trained model has been sufficiently trained to determine whether, for example, a target shown in a visible image is a person or a car. An example of such a trained model is ImageNet. A target of the trained model can be an image other than visible images. The target can be, for example, an infrared image or a thermal image. Furthermore, the target of the trained model is not limited to images.For example, the target of the trained model can be a measured value measured with an unmapped sensor and expressed two-dimensionally, such as a reflectance, a reflection intensity, or a frequency, or it can be a frequency distribution.

[0018] The data stored in storage device 3 is data for learning at the knowledge transfer target. For example, data for learning depth images such as people, cars, and buildings is stored. Each depth image is assigned a label that indicates the displayed target. The depth image is also called a distance image and is an image that has a distance value as a pixel value. If the learning data to be stored has a different modality than the modality of the data to be stored in storage device 2, the learning data can be an image other than depth images, a two-dimensionally expressed measurement value, or a different frequency distribution than the images.

[0019] The data of a post-knowledge transfer model is stored as data to be stored in the storage device 4. For example, when a new model is formed by using the knowledge of a trained model for a visible image stored in the storage device 2 and learning data of a depth image stored in the storage device 3, the post-knowledge transfer model is a trained model for the depth image. (Knowledge transfer learning unit)

[0020] A configuration of the transfer learning device 1 is described in more detail below. The following description assumes that the data stored in the storage device 2 is the data of a trained model for a visible image, and the data stored in the storage device 3 is the learning data of a depth image.

[0021] The knowledge transfer learning unit 11 is a function that performs knowledge transfer learning using learning data relating to a second data type different from a first data type based on a trained model relating to the first data type, and generates an intermediate model from the trained model. The knowledge transfer learning unit 11 performs such learning when it first acquires the trained model and the learning data and when it accepts an output from the knowledge transfer repetition determination unit 13 indicating that the knowledge transfer learning should be repeated. Specifically, the knowledge transfer learning unit 11 acquires data such as structure and weight coefficient data of neural network nodes of the trained model with respect to a visible image as data relating to a modality (data type) of the knowledge transfer source from the storage device 2.Furthermore, the knowledge transfer learning unit 11 acquires learning data related to depth images as data related to the modality of the knowledge transfer target from the storage device 3. The knowledge transfer learning unit 11 uses these acquired data to perform knowledge transfer learning using the learning data related to the depth images and based on the trained model related to the visible images. The knowledge transfer learning is performed using a known technique described in Patent Literature 1 or the like. The knowledge transfer learning unit 11 generates an intermediate model during knowledge transfer by performing knowledge transfer learning and outputs the generated intermediate model. The output intermediate model may be temporarily stored in a storage device (not shown).

[0022] The intermediate model to be generated is gradually adapted to the modality of the knowledge transfer target through transfer learning. Fig. 4A and Fig. 4B are diagrams that illustrate how this transformation takes place. Fig. Figure 4A shows a probability transition graph representing a transition of a probability of a visible image according to the intermediate model generated by transfer learning. Fig. Figure 4B shows a probability transition graph representing a transition of a depth image probability according to the intermediate model generated by transfer learning. A correct label is assumed to be label 1 at any one of the knowledge transfer source and the knowledge transfer destination.

[0023] Since in Fig. 4A, a base model is a trained model for visible images, the generated intermediate model determines with high probability that an input visible image is an image with label 1 in a phase immediately after the start of transfer learning, that is, for example, in stages of the number of epochs 1 to 10. However, the intermediate model transforms into a model for depth images as transfer learning progresses, and therefore the generated intermediate model cannot determine with high probability that an input visible image is an image with label 1 in a phase that is a while after the start of transfer learning, that is, for example, in stages of the number of epochs 41 to 50.

[0024] On the other hand, the intermediate model, as in Fig. 4B, cannot determine with a high probability that the input depth image is the image of label 1 in the phase immediately after the start of transfer learning, but can determine with a high probability that the input depth image is the image of label 1 in the phase a while after the start of transfer learning.

[0025] Although in Fig. 4A and Fig. 4B assumes a single trained model for visible images as the base model, multiple trained models for visible images can also be used as base models. The multiple base models are usually learned for a visible image assigned the same label using different data sets, so that by integrating and transferring knowledge from the multiple models, the determination accuracy of a learning model to be generated can be increased. The base models include Model 1, Model 2, and Model 3. In this case, it is in the Fig. 4A and Fig. 4B, by assigning the number of epochs 1, 4, 7, 10 and ... to the epochs of model 1, by assigning the number of epochs 2, 5, 8, 11 and ... to the epochs of model 2, by assigning the number of epochs 3, 6, 9, 12 and ... to the epochs of model 3, and processing each of three consecutive epochs (the number of epochs 1 to 3, 4 to 6, 7 to 9, 10 to 12, and ...) as a single unit, it is possible to integrally transfer the knowledge from the plurality of trained models. (Knowledge transfer progress determination unit)

[0026] The knowledge transfer progress degree determination unit 12 is a functional unit that determines the progress degree of knowledge transfer learning. Specifically, the knowledge transfer progress degree determination unit 12 determines the progress degree of knowledge transfer learning by determining, through a multiple comparison test in the intermediate model generated by the knowledge transfer learning unit 11, whether a reference probability of learning data assigned a reference label is greater than the probabilities of a plurality of individual learning data each assigned non-reference labels and not the reference label.Since it is not clear at what degree of difference between the probabilities it can be determined that a size relationship exists, the determination of whether a size relationship exists or not is made by a multiple comparison test based on a common index, namely the confidence level or a confidence interval.

[0027] The multiple comparison test refers to a method for testing which groups have a difference when measuring values ​​from a plurality of three or more groups. As a method for conducting the multiple comparison test, for example, the Dunnett multiple comparison test described in the following Non-Patent Literature 1 can be used. Note that a multiple comparison test scheme is not limited to the Dunnett method but can also include other schemes such as the Williams test or the Tukey test.

[0028] Non-Patent Literature 1: "Theory of Multiple Comparison Procedures and Its Computation" by Takaaki Shirashi and Hiroshi Sugiura, Kyoritsu Shuppan Co., Ltd., pp. 76 to 81 (2018).

[0029] According to Dunnett's multiple comparison test, when comparing mean values ​​of groups of measured values ​​from the groups of measured values ​​of several experiments, a value called "tolerance" is calculated depending on a confidence level p. If an average value of a sample serving as a reference and an average value of each sample have a size ratio that exceeds this tolerance value, it is determined that the size ratio holds with the confidence level p. A value of p that serves as a cutoff to determine whether the size ratio holds or not is called a p-value.

[0030] The knowledge transfer progress determination unit 12 calculates the p-value from a probability of image data input to the intermediate model to determine, through the multiple comparison test, whether the reference probability of learning data assigned the reference label is greater than the probabilities of a plurality of individual learning data, each of which has been assigned non-reference labels and not the reference label, in the intermediate model output from the knowledge transfer learning unit 11. The p-value varies in an epoch unit and is therefore calculated from the probabilities of the image data in a plurality of epochs.

[0031] The Fig. 6A to 6C show tables showing examples of the calculated p-values. Fig. Figure 6A illustrates a p-value table of section S1 in Fig. 5A, which illustrates a probability transition of a visible image, and illustrates a p-value table of Section S1 in Fig. 5B, which illustrates a probability transition of a depth image. Fig. Figure 6B further illustrates a p-value table of section S2 in Fig. 5B, which illustrates a probability transition of a visible image, and illustrates a p-value table of section S2 in Fig. 5B, which illustrates a probability transition of a depth image. Fig. Figure 6C illustrates a p-value table of section S3 in Fig. 5C, which illustrates a probability transition of a visible image, and illustrates a p-value table of section S3 in Fig. 5B, which illustrates a probability transition of a depth image. Note that the probability transition in Fig. 5A is identical to that in Fig. 4A, and the probability transition in Fig. 5B is identical to that in Fig. 4B. In addition, section S1 is a section of the number of epochs 1 to 10, section S2 is a section of the number of epochs 21 to 30, and section S3 is a section of the number of epochs 41 to 50. By performing the multiple comparison test once, it is possible to evaluate a size relationship between a plurality of other labels (e.g., Label 2 to Label 6) for which a certain label (e.g., Label 1) serves as a reference, and to calculate the p-value of each cell of a column (e.g., a column of Label 1) of the p-value table. By performing the multiple comparison test using another label (e.g., Label 2) as the reference label, it is possible to evaluate a size relationship between a plurality of other labels (e.g., Label 1 and Label 3 to Label 6) for which this other label (e.g., Label 2) serves as a reference, and to calculate the p-value of each cell of a column (e.g., a column of Label 2) of the p-value table.

[0032] According to the multiple comparison test, it is concluded that if the p-value of a cell where a column with a reference label number and a row with a non-reference label that is not the reference label intersect is 0.05 or less, a score of the non-reference label is smaller than that of the reference label at a confidence level of 95%. Furthermore, it can be determined that if the p-values ​​of all cells in the columns with the reference label number are 0.05 or less, the reference label is maximal.

[0033] By setting a correct label as the reference label using such a multiple comparison test method, the knowledge transfer progress degree determination unit 12 determines in the intermediate model output from the knowledge transfer learning unit 11 whether the reference probability of learning data assigned the reference label (i.e., the correct label) is greater than the probabilities of a plurality of individual learning data assigned the respective non-reference labels other than the reference label, and outputs a determination result. Note that when the reference label is label 1, the non-reference labels correspond to labels 2 to 6. (Knowledge transfer-repetition-determination unit)

[0034] The knowledge transfer progress degree determination unit 13 determines whether or not to repeat the knowledge transfer learning based on the determination result output by the knowledge transfer progress degree determination unit 12. Specifically, the knowledge transfer repetition determination unit 13 determines that the knowledge transfer learning will not be repeated if it is indicated that the reference probability of the reference label is greater than the probabilities of all non-reference labels other than the reference label, and determines that the knowledge transfer learning will be repeated if this is not the case (if it is not indicated that the reference probability is greater than the probabilities of all non-reference labels). The knowledge transfer repetition determination unit 13 outputs a result of the determination of whether or not to repeat the knowledge transfer learning. (Knowledge transfer result output unit)

[0035] When the result of the determination by the knowledge transfer repetition determination unit 13 that knowledge transfer learning is not repeated is accepted, the knowledge transfer result output unit 14 recognizes that the intermediate model is a post-knowledge transfer trained model for which knowledge transfer has been completed, and outputs data of the post-knowledge transfer trained model, which is a result of the knowledge transfer, to the storage device 4.

[0036] Next, a hardware configuration example of the transfer learning device 1 will be described with reference to Fig. 2A and Fig. 2B. The functional units of the transfer learning device 1 are implemented by a processing circuit. That is, the transfer learning device 1 comprises a processing circuit for performing various arithmetic operations, such as knowledge transfer learning, knowledge transfer progress determination, knowledge transfer repetition determination, and knowledge transfer result output. The processing circuit may be a Fig. 2A, or a computer that includes a processor 100b that executes a Fig. 2B executes programs stored in the memory 100c shown.

[0037] In a case where the processing circuit is the dedicated processing circuit 100a, the dedicated processing circuit 100a corresponds, for example, to a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a combination thereof. The functions of the respective units of the knowledge transfer learning unit 11, the knowledge transfer progress degree determination unit 12, the knowledge transfer repetition determination unit 13, and the knowledge transfer result output unit 14 may be implemented by a plurality of different processing circuits, or the functions of the units may be implemented collectively by a single processing circuit.

[0038] In a case where the processing circuit is the processor 100b, each function of the knowledge transfer learning unit 11, the knowledge transfer progress determination unit 12, the knowledge transfer repetition determination unit 13, and the knowledge transfer result output unit 14 can be implemented by software, firmware, or a combination of software and firmware. The software and firmware are described as programs and stored in the memory 100c. The processor 100b implements the function of each unit by reading and executing the programs stored in the memory.That is, the working memory 100c for storing a program (transfer learning program) that ultimately executes steps such as a step of performing knowledge transfer learning, a step of performing knowledge transfer progress determination, a step of performing knowledge transfer repetition determination, and a step of performing knowledge transfer result output is provided. Examples of the working memory 100c include a non-volatile or volatile semiconductor memory such as a random access memory (RAM), a read-only memory (ROM), a flash memory, an erasable programmable read-only memory (EPROM), or an electrically erasable programmable read-only memory (EEPROM), a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, and a DVD.

[0039] Note that the functions of the knowledge transfer learning unit 11, the knowledge transfer progress determination unit 12, the knowledge transfer repetition determination unit 13, and the knowledge transfer result output unit 14 may be implemented by dedicated hardware, and some of the functions may be implemented by software or firmware. Thus, the processing circuit may implement each of the above functions by dedicated hardware, software, firmware, or a combination thereof. <funktionsweise>

[0040] Next, the functionality of the Fig. 7 is described. In step S1, the knowledge transfer learning unit 1 performs knowledge transfer learning using learning data relating to a second data type different from a first data type based on a trained model relating to the first data type, and generates an intermediate model from the trained model.

[0041] In step S2, the knowledge transfer progress degree determination unit 12 determines, through the multiple comparison test, in the intermediate model generated by the knowledge transfer learning unit 11, whether the reference probability of learning data assigned the reference label is greater than the probabilities of a plurality of individual learning data each assigned non-reference labels and not the reference label. The knowledge transfer progress degree determination unit 12 outputs a determination result.

[0042] In step S3, the knowledge transfer repetition determination unit 13 determines whether or not to repeat knowledge transfer learning based on the determination result output by the knowledge transfer progress degree determination unit 12. Specifically, the knowledge transfer repetition determination unit 13 determines that knowledge transfer learning will not be repeated if it is indicated that the reference probability of the reference label is greater than the probabilities of all non-reference labels other than the reference label, and determines that knowledge transfer learning will be repeated if this is not the case (if it is not indicated that the reference probability is greater than the probabilities of all non-reference labels).The processing returns to step S1 when it is determined that knowledge transfer learning should be repeated, and the processing goes to step S4 when it is determined that knowledge transfer learning should not be repeated.

[0043] In step S4, when the result of the determination by the knowledge transfer repetition determination unit 13 that the knowledge transfer learning is not repeated is accepted, the knowledge transfer result output unit 14 recognizes that the intermediate model is a post-knowledge transfer trained model for which knowledge transfer has been completed, and outputs data of the post-knowledge transfer trained model that is a result of knowledge transfer.

[0044] The transfer learning device 1 described above can evaluate, for each non-reference label, the degree of learning progress of individual learning data assigned to a plurality of non-reference labels with respect to learning data assigned to the reference label. Consequently, it is possible to identify learning data for which the progress of transfer learning is delayed and to establish a basis for intensive learning of the learning data for which the progress of transfer learning is delayed. Consequently, it is possible to shorten the learning time in transfer learning. Embodiment 2. <konfiguration>

[0045] Next, a transfer learning device 1' according to Embodiment 2 of the present disclosure will be described with reference to Fig. 8 to 9C. The transfer learning device 1' according to Embodiment 2 differs from the transfer learning device 1 according to Embodiment 1 in that it additionally includes a learning data selection unit 15. Since the learning data selection unit 15 has been added, the function of the knowledge transfer progress degree determination unit 12' included in the transfer learning device 1' differs slightly from that of the knowledge transfer progress degree determination unit 12 of the transfer learning device 1.

[0046] In the transfer learning device 1' according to Embodiment 2, the knowledge transfer progress degree determination unit 12' determines that the progress degree of knowledge transfer learning is delayed by determining whether or not there is learning data whose progress of knowledge transfer is delayed. The determination of whether or not there is such learning data is made by specifying a non-reference label from a multiple comparison test result whose p-value is smaller than a predetermined value. Thus, if it is not determined that the reference probability is greater than the probabilities of a plurality of individual learning data assigned to respective non-reference labels, the knowledge transfer progress degree determination unit 12' specifies the non-reference label from the multiple comparison test whose p-value is smaller than the predetermined value.

[0047] The learning data selection unit 15 is a function that, when it is not determined that the reference probability is greater than the probabilities of the plurality of individual learning data to which the respective non-reference labels are assigned, selects learning data that contributes to not determining that the reference probability is greater than the probabilities of the plurality of individual learning data. Specifically, the learning data selection unit 15 selects (i.e., acquires) from the storage device 3 learning data to which a label specified by the knowledge transfer progress degree determination unit 12' has been assigned, and delivers the selected learning data to the knowledge transfer learning unit 11. The knowledge transfer learning unit 11 performs transfer learning using the learning data selected by the learning data selection unit 15.

[0048] Referring to, for example, Fig. 9A, the knowledge transfer progress determination unit 12' specifies the label 6 from a column C1, whose reference label number is 1 in a p-value table in Fig. 9A, which is calculated from the probabilities of depth images at the number of epochs 26 to 35. Label 6 is specified by determining whether the value of the p-value is a predetermined value such as 0.05 or less. The learning data selection unit 15 selects learning data assigned with label 6 from the storage device 3 and delivers the selected learning data to the knowledge transfer learning unit 11.

[0049] The p-values ​​from Label 1 to Label 5 that are equal to or less than the specified value in Fig. 9A, indicate that the learning of Label 1 to Label 5 is sufficiently advanced. On the other hand, the p-value of Label 6, which is larger than the predetermined value, indicates that the learning of Label 6 is not sufficiently advanced. Consequently, Label 6, whose learning progress degree is delayed compared to the other labels, is intensively learned.

[0050] By intensively learning depth images of label 6, the p-value of label 6 is significantly reduced, as shown in Fig. 9B and Fig. 9C. Fig. Figure 9B shows a p-value table calculated from the probabilities of the depth images at the number of epochs 27 to 36, and Fig. Figure 9C shows a p-value table calculated from the probabilities of the depth images at the number of epochs 28 to 37. By intensively learning the depth images of label 6, the p-value of label 6 is reduced to 0.33 in a column C2 in Fig. 9B and to 0.00 in a column C3 in Fig. 9C reduced.

[0051] As described above, by intensively learning data whose learning progress degree is delayed, it is possible to reduce the number of times of learning data whose learning progress degree is delayed, so that it is possible to reduce the learning time required for the entire transfer learning.

[0052] The hardware of the transfer learning device 1' can be described using the configuration example in Fig. 2A or Fig. 2B similar to the configuration example of the hardware of the transfer learning device 1 according to Embodiment 1. <funktionsweise>

[0053] Next, the functionality of the Fig. 10 described transfer learning device 1. Since the steps S1, S3 and S4 are similar to the steps in Fig. 7, their description is omitted.

[0054] In step S2, the knowledge transfer progress degree determination unit 12 determines whether there is any learning data whose progress is delayed by knowledge transfer. Processing proceeds to step S5 if there is any learning data whose progress is delayed, and processing proceeds to step S3 if there is no learning data whose progress is delayed.

[0055] In step S5, the learning data selection unit 15 selects learning data from the storage device 3 for which the knowledge transfer progress degree determination unit 12 has determined that the progress of transfer learning is delayed, and supplies the selected learning data to the knowledge transfer learning unit 11. Then, the processing step returns to step S1.

[0056] It should be noted that the embodiments may be combined, and that each embodiment may be modified or omitted as needed. INDUSTRIAL APPLICABILITY

[0057] The transfer learning device according to the present disclosure can be used as a device that transfers and utilizes knowledge related to a model asset that has already been sufficiently learned by machine learning, and can be applied to a technical field such as video image analysis. LIST OF REFERENCE SYMBOLS

[0058] 1 (1'): transfer learning device, 2: storage device, 3: storage device, 4: storage device, 11: knowledge transfer learning unit, 12 (12'): knowledge transfer progress degree determination unit, 13: knowledge transfer repetition determination unit, 14: knowledge transfer result output unit, 15: learning data selection unit, 100a: processing circuit, 100b: processor, 100c: working memory. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] JP 2021 / 135726 A

[0003] Cited non-patent literature

[0000] "Theory of Multiple Comparison Procedures and Its Computation" by Takaaki SHIRAISHI and Hiroshi SUGIURA, KYORITSU SHUPPAN CO., LTD. S. 76 to 81 (2018

[0028] < / funktionsweise> < / konfiguration> < / funktionsweise> < / konfiguration>

Claims

[1] Transfer learning facility, comprising: a knowledge transfer learning unit for performing knowledge transfer learning using learning data relating to a second data type different from a first data type based on a trained model relating to the first data type, and generating an intermediate model from the trained model; and a knowledge transfer progress degree determination unit for determining in the generated intermediate model through a multiple comparison test whether a reference probability of learning data assigned a reference label is greater than probabilities of a plurality of individual learning data each assigned non-reference labels and not the reference label. [2] The transfer learning device according to claim 1, further comprising a learning data selecting unit for, when it is not determined that the reference probability is larger than the probabilities of the plurality of individual learning data, selecting learning data that contributes to not determining that the reference probability is larger than the probabilities of the plurality of individual learning data. [3] The transfer learning device according to claim 1 or 2, further comprising a knowledge transfer repetition determining unit for determining, when it is determined that the reference probability is larger than the probability of the plurality of individual learning data, to terminate the knowledge transfer learning. [4] A transfer learning method performed by a transfer learning device having a knowledge transfer learning unit and a knowledge transfer progress determination unit, the transfer learning method comprising: a step of performing knowledge transfer learning, by the knowledge transfer learning unit, performing knowledge transfer learning using learning data relating to a second data type different from a first data type based on a trained model relating to the first data type, and generating an intermediate model from the trained model; and a step of determining a knowledge transfer progress degree, by the knowledge transfer progress degree determining unit, determining in the generated intermediate model, by a multiple comparison test, whether or not a reference probability of learning data assigned a reference label is greater than probabilities of a plurality of individual learning data each assigned non-reference labels and not the reference label. [5] Transfer learning program that causes a computer to execute: a function to perform knowledge transfer learning, including performing knowledge transfer learning using learning data relating to a second data type that is different from a first data type, based on a trained model relating to the first data type, and generating an intermediate model from the trained model; and a function to determine a degree of knowledge transfer progress, including determining in the generated intermediate model, by a multiple comparison test, whether or not a reference probability of learning data assigned a reference label is greater than probabilities of a plurality of individual learning data each assigned non-reference labels and not the reference label.

Images