Training device, training method, and training program

Abstract

A classification device (1) comprises: a training data input unit (11) that acquires, as a related dataset, data composed of training data that is associated with a target dataset and is assigned an accurate label; a pseudo noisy training data generation unit (13) that, on the basis of the related dataset, generates, in a pseudo manner, noisy training data in which data labeling is incorrect; and a classifier generation model training unit (14) that evaluates the performance of a first classifier, which is generated on the basis of the pseudo noisy training data generated by the pseudo noisy training data generation unit (13), by using the labeled data of the related dataset, and executes training of a classifier generation model for generating the classifier such that the evaluation result increases.

Description

Learning device, learning method, and learning program

[0001] The present invention relates to a learning device, a learning method, and a learning program.

[0002] Supervised machine learning techniques require training data to train a classifier, which is a set of data-label pairs.

[0003] In general, training data is often created by multiple annotators. For example, in crowdsourcing, training data is created by asking multiple workers to label data (for example, in the case of image data, they would assign labels to objects in the image). In the medical and security domains, multiple experts label the data. In general, annotators' labeling abilities vary widely, so the created training data is often noisy (data is incorrectly labeled).

[0004] In order to train a highly accurate classifier even in such a situation, supervised learning techniques have been proposed that take into account that training data is obtained from multiple annotators (Non-Patent Documents 1 and 2).

[0005] Non-Patent Document 1 describes a method of modeling the ability of each annotator using a probabilistic model based on the training data provided by each annotator, estimating the ability of the annotator, and training a classifier from the training data weighted by the estimated ability value of the annotator.

[0006] Non-Patent Document 2 describes a method for pre-training a data feature extractor (neural network) using high-quality training data from a related domain. The method described in Non-Patent Document 2 transforms data using a pre-trained neural network, and trains a classifier by applying a probabilistic model (for example, the probabilistic model described in Non-Patent Document 1) to the transformed data. The method described in Non-Patent Document 2 can improve the performance of classifier training by utilizing information from related datasets.

[0007] Raykar, Vikas C., et al., “Learning From Crowds”, Journal of machine learning research 11.4 (2010). S. Xu and J. Zhang, “Crowdsourcing with Meta-Knowledge Transfer (Student Abstract)”, 2022.

[0008] In order to achieve high performance in the method described in Non-Patent Document 1, a large amount of data obtained from a large number of annotators is required for the task for which a classifier is to be constructed.

[0009] However, when it is not possible to collect a large amount of data, as in the case of real-world problems, it is difficult to train a highly accurate classifier. For example, in crowdsourcing, budgetary constraints may prevent large-scale labeling requests. Furthermore, in domains that require advanced expertise, such as medicine or security, and where data is difficult to gather, it is difficult to gather a large amount of data to be labeled and annotators (experts) who can label the data.

[0010] The method described in Non-Patent Document 2 can improve the performance of classifier training by utilizing information from related datasets. However, the method described in Non-Patent Document 2 does not perform pre-training that takes into account the fact that training data is provided from multiple annotators, and does not perform pre-training corresponding to the solved task, so its performance is limited.

[0011] The present invention has been made in consideration of the above, and aims to provide a learning device, a learning method, and a learning program that can accurately train a classifier even when using a small amount of noisy training data provided by multiple annotators.

[0012] In order to solve the above-mentioned problems and achieve the object, the learning device of the present invention is characterized by having an acquisition unit that acquires data that is related to a target dataset and consists of accurately labeled training data as a related dataset; a generation unit that generates pseudo-noisy training data in which data is incorrectly labeled based on the related dataset; and a learning unit that evaluates the performance of a first classifier generated based on the pseudo-noisy training data generated by the generation unit using labeled data in the related dataset, and executes learning of a classifier generation model to generate a classifier so as to improve the evaluation result.

[0013] According to the present invention, a classifier can be trained with high accuracy even using a small amount of noisy training data provided by multiple annotators.

[0014] FIG. 1 is a diagram illustrating processing by a classification device according to an embodiment. FIG. 2 is a diagram schematically illustrating an example of the configuration of a classification device according to an embodiment. FIG. 3 is a flowchart illustrating a processing procedure of a learning process. FIG. 4 is a flowchart illustrating a processing procedure of a classification process. FIG. 5 is a diagram illustrating processing by a learning unit. FIG. 6 is a diagram illustrating an example of a computer that realizes the classification device by executing a program.

[0015] An embodiment of the present invention will be described in detail below with reference to the drawings. However, the present invention is not limited to this embodiment. In addition, in the description of the drawings, the same parts are denoted by the same reference numerals. In the following, when "A" is written for A, which is a vector, matrix, or scalar, it is considered to be equivalent to "a symbol with "A" with "A" written immediately above it."

[0016] [Embodiment] In the embodiment, attention is paid to the fact that high-quality training data can often be obtained if the task domain is related.

[0017] For example, if you want to build a medical image diagnostic system, it is difficult to obtain a large amount of high-quality medical images for the reasons mentioned above, but for general images, data with accurate labels can be easily obtained from the Web, etc.

[0018] In the security field, building a cutting-edge attack and malware classification system requires scarce data and advanced, up-to-date expertise, making it difficult to gather a large amount of high-quality data.In contrast, for known attacks and malware, high-quality data that has been analyzed in detail is already being accumulated.

[0019] In an embodiment, a machine learning-based classification system trains a highly accurate classifier by leveraging information from multiple related datasets, each of which consists of correctly labeled training data, in addition to a target dataset (a small number of noisy training data from multiple annotators).

[0020] For example, if the target dataset is a small number of noisy training data sets for cutting-edge attacks and malware, the related datasets are training data sets with accurate labels for known attacks and malware. If the target dataset is a small number of noisy medical images, the related datasets are general images with accurate labels. Thus, in the embodiments, the related datasets are datasets that are composed of the same type of data as the target dataset or datasets for a different task in the same field as the target task and that have already been accurately labeled.

[0021] [Overview of Classification Apparatus] Fig. 1 is a diagram illustrating the processing of a classification apparatus according to an embodiment. As shown in Fig. 1, in the training phase, training is performed using only data from multiple related datasets. Each related dataset consists of training data with accurate labels.

[0022] In the training phase, we first generate a group of pseudo annotators with various labeling abilities (Fig. 1 (1)), and then generate pseudo noisy training data with incorrectly labeled data from data randomly extracted from related datasets (Fig. 1 (1)). Hereafter, the noisy training data generated from data randomly extracted from related datasets is referred to as pseudo noisy training data.

[0023] Next, in the training phase, a classifier generation model is trained by machine learning to directly maximize the generalization performance (classification performance for untrained data) of the first classifier generated based on pseudo-noisy training data ((2) in Figure 1). The classifier generation model is a machine learning model for generating classifiers. Furthermore, the generalization performance can be calculated directly from labeled data in the related dataset.

[0024] In other words, the performance of the first classifier generated based on pseudo-noisy data is evaluated using labeled data randomly extracted from the related dataset, and the classifier generation model is trained to improve the evaluation results ((3) in Figure 1).

[0025] In the learning phase, a classifier is generated from noisy training data of multiple pseudo annotators (pseudo annotators), using, for example, the method described in Non-Patent Document 1.

[0026] In the test phase, a dataset for a target task (hereinafter referred to as the target dataset) is input to the trained classifier generation model to generate a second classifier suitable for the target dataset. The target dataset is a small amount of noisy training data for the target task obtained from multiple annotators. In the test phase, the generated second classifier is used to classify test data for the target task.

[0027] [Classification Device] FIG. 2 is a diagram schematically illustrating an example of the configuration of a classification device according to an embodiment. The classification device 1 according to the embodiment is realized, for example, by loading a predetermined program into a computer or the like including a ROM (Read Only Memory), a RAM (Random Access Memory), a CPU (Central Processing Unit), etc., and having the CPU execute the predetermined program. The classification device 1 also has a communication interface for transmitting and receiving various information to and from other devices connected via a network, etc. The classification device 1 (learning device) is realized by a general-purpose computer such as a workstation or a personal computer. As shown in FIG. 2, the classification device 1 has a learning unit 10 that performs learning processing and a classification unit 20 that performs classification processing.

[0028] The learning unit 10 uses a plurality of related datasets to learn a classifier generation model 151 (described below). The related datasets are training data to which true (accurate) labels are assigned. The learning unit 10 generates pseudo-noisy training data based on the related datasets. The learning unit 10 evaluates the performance of a classifier (first classifier) ​​generated using the pseudo-noisy training data with labeled data from the related datasets, and trains the classifier generation model 151 so as to increase the evaluation value.

[0029] When a target dataset is given, the classifier 20 generates a second classifier corresponding to the target task using the classifier generation model 151 trained by the learning unit 10. The classifier 20 classifies test data for the target task using the generated second classifier. The classifier 20 may be implemented on the same hardware as the learning unit 10, or on different hardware.

[0030] [Learning Unit] The learning unit 10 includes a learning data input unit 11 (acquisition unit), a feature extraction unit 12, a pseudo-noisy learning data generation unit 13 (generation unit), a classifier generation model learning unit 14 (learning unit), and a storage unit 15.

[0031] The training data input unit 11 is implemented using input devices such as a keyboard and a mouse, and accepts input of various instruction information in response to input operations by an operator. The training data input unit 11 functions as an acquisition unit and receives multiple related datasets as input. As described above, the related datasets are composed of training data that are related to the target dataset and have been accurately labeled. The related datasets may be input to the training unit 10 from an external server device or the like via a communication control unit (not shown) implemented by a NIC (Network Interface Card) or the like.

[0032] The feature extraction unit 12 converts each data item in the input related dataset into a feature vector. Here, a feature vector is a representation of the features of the required data as an n-dimensional numeric vector. The conversion to a feature vector uses a method commonly used in machine learning. For example, if the data is text, methods such as morphological analysis, n-gram, and delimiter analysis can be applied.

[0033] The pseudo-noisy training data generator 13 generates multiple pseudo annotators. Specifically, the pseudo-noisy training data generator 13 receives as input a pair of data randomly extracted from a related dataset and the true label of the data, and regards a “function” that outputs a noisy label for the data as one annotator.

[0034] The pseudo-noisy training data generator 13 generates pseudo-annotators with different labeling methods and / or error characteristics by varying the parameters of the function. The pseudo-noisy training data generator 13 generates pseudo-noisy training data by inputting labeled data randomly extracted from a related dataset into the pseudo-annotators (functions).

[0035] That is, the pseudo-noisy training data generation unit 13 inputs labeled data randomly extracted from a related dataset into multiple functions with different parameters that have different labeling methods and / or error characteristics, thereby generating pseudo-noisy training data.

[0036] The classifier generation model training unit 14 uses the related data set and the pseudo-noisy training data to train the classifier generation model 151 for generating a classifier. As described above, the classifier generation method uses, for example, the method described in Non-Patent Document 1.

[0037] The classifier generation model training unit 14 evaluates the performance of a classifier (first classifier) ​​generated using pseudo-noisy data with labeled data of a related dataset, and performs training of the classifier generation model 151 so as to increase the evaluation value. The classifier generation model training unit 14 trains the classifier generation model 151 so as to directly maximize the generalization performance of the obtained classifier.

[0038] In the test phase, when a target dataset is given, the classifier generation model 151 outputs a classifier (second classifier) ​​suitable for the target task of the target dataset. When test data for the target task is input, the second classifier classifies the classification result of the input test data (e.g., the label of the test data).

[0039] The storage unit 15 is realized by a semiconductor memory element such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 15 stores a trained classifier generation model 151.

[0040] [Classification Unit] The classification unit 20 includes a data input unit 21, a feature extraction unit 22, a learning and classification unit 23, and a classification result output unit 24.

[0041] The data input unit 21 is realized using input devices such as a keyboard and a mouse, and accepts input of various instruction information in response to input operations by an operator. The data input unit 21 accepts a target dataset and test data. The data input unit 21 outputs the accepted input target dataset and test data to the feature extraction unit 22. Once the classification unit 20 receives the target dataset and generates a classifier, it can perform classification by simply inputting the test data for the target task.

[0042] The target data set and the test data may be input to the classification unit 20 from an external server device or the like via a communication control unit (not shown) realized by a NIC or the like. The data input unit 21 may be the same hardware as the training data input unit 11.

[0043] Similar to the feature extraction unit 12 of the learning unit 10, the feature extraction unit 22 converts the acquired target data set, test data, and each piece of test data into a feature vector in preparation for processing in the learning / classification unit 23.

[0044] The learning and classification unit 23 generates a second classifier from the target data set using the classifier generation model 151. The obtained second classifier is saved, and thereafter, when test data for the target task is input, the learning and classification unit 23 performs classification using the obtained second classifier.

[0045] The classification result output unit 24 is realized by a display device such as a liquid crystal display, a printing device such as a printer, an information communication device, etc., and outputs the results of the classification process to the operator. For example, the classification result output unit 24 outputs identification information of the labels into which the input test data has been classified.

[0046] 3 and 4, a processing procedure of the processing by the classification device 1 will be described. The classification processing by the classification device 1 includes a learning process by the learning unit 10 and a classification process by the classification unit 20.

[0047] [Learning Process] Fig. 3 is a flowchart showing the processing procedure of the learning process, which is started, for example, when a user performs an operation input to instruct the start of the learning process.

[0048] 3, the training data input unit 11 receives a plurality of related data sets (training data to which true labels have been assigned) as input (step S1). The feature extraction unit 12 converts each data item of the received related data sets into a feature vector (step S2).

[0049] The pseudo-noisy training data generating unit 13 generates a group of pseudo annotators, and generates pseudo-noisy training data based on the group of pseudo annotators (step S3).

[0050] The classifier generation model training unit 14 uses the related dataset and pseudo-noisy training data to train the classifier generation model 151 for generating a classifier (step S4). The classifier generation model training unit 14 evaluates the performance of the classifier (first classifier) ​​generated using the pseudo-noisy data with labeled data in the related dataset, and trains the classifier generation model 151 so as to increase the evaluation value.

[0051] The classifier generation model learning unit 14 stores the learned classifier generation model 151 in the storage unit 15 .

[0052] [Classification Processing] Fig. 4 is a flowchart showing the processing procedure of the classification processing, which is started, for example, when a user performs an operation input to instruct the start of the classification processing.

[0053] The data input unit 21 receives a target data set and test data to be classified (step S11). The feature extraction unit 22 converts the target data set, the test data, and each data item in the test data into a feature vector (step S12).

[0054] The learning and classification unit 23 uses the classifier generation model 151 to generate a second classifier from the target data set and classifies each test data set using the generated second classifier (step S13). By generating a second classifier from the target data set of the target task, the classification unit 20 can output the classification results by inputting only the test data of the target task. The classification result output unit 24 then outputs the classification results generated by the second classifier (step S14).

[0055] [Effects of the embodiment] As described above, in the embodiment, focusing on the fact that high-quality training data can often be obtained from related task domains, in addition to the target dataset (a small amount of noisy training data), a high-performance classifier is trained from the target dataset by utilizing multiple related datasets that are related to the target dataset and comprise training data that are accurately labeled.

[0056] Specifically, the classification device 1 according to the embodiment acquires a related dataset, generates pseudo-noisy training data with incorrectly labeled data based on the acquired related dataset, evaluates the performance of a classifier (first classifier) ​​generated based on the pseudo-noisy training data using labeled data in the related dataset, and performs training of a classifier generation model 151 for generating a classifier so as to improve the evaluation results.

[0057] In this way, the classification device 1 performs training of the classifier generation model 151 using multiple related datasets (training data with accurate labels) rather than a target dataset (a small amount of noisy training data).

[0058] The classification device 1 then inputs labeled data randomly extracted from the related dataset into a plurality of functions with different labeling methods and / or error characteristics to generate pseudo-noisy training data. The classification device 1 evaluates the performance of the first classifier generated based on the pseudo-noisy training data using the labeled data of the related dataset, and performs training of the classifier generation model 151 so as to improve the evaluation results.

[0059] In this way, by using this classifier generation model 151, the classification device 1 can generate a high-performance classifier (second classifier) ​​from a target data set even if the target data set is a small amount of noisy training data.

[0060] [Example] An example of a specific method for realizing the embodiment will be described.

[0061] First, we will explain the problem setting.

[0062] During training, D related datasets D (Equation (1)) are given, where N d is the number of data in the dth dataset, and x dn is the feature vector of the nth data in the dth dataset, and t dn is x dn are the true class labels corresponding to

[0063]

[0064] At test time, training data S (equation (2)) is obtained from R annotators.

[0065]

[0066] y n r is the label given by the rth annotator to the nth data. n (Equation (3)) represents the index set of the annotator who labeled the n-th data.

[0067]

[0068] For any n, I n We assume that S is not an empty set. That is, we assume that at least one person labels all data. We also assume that the class labels in the training data S do not appear in the related dataset D. The goal is to generate (train) a highly accurate classifier from the training data S.

[0069] First, we explain the classifier generation model for generating and training a classifier from training data S (test phase), and then explain how to train the classifier generation model (training phase).

[0070] First, as shown in equation (4), each data x is transformed using a neural network f.

[0071]

[0072] Here, Θ is a parameter of the neural network f. This parameter Θ is a parameter common to all datasets (tasks), and learning this parameter Θ corresponds to learning a classifier generation model. This learning method will be described later.

[0073] In the test phase, a probabilistic model is constructed to generate a classifier (second classifier) ​​from the training data S. Specifically, consider the probabilistic model of Equation (5).

[0074]

[0075] Here, U (equation (6)) is a data set obtained by transforming x in the training data S with f.

[0076]

[0077] Y is the set of labels assigned by the annotators in the training data S. n represents the true class label (unobserved) of the nth data in the training data S.

[0078] p (t n |π) is a categorical distribution with parameter π. p(u n |t n = k, M) (Equation (7)) is the average u k , which represents a Gaussian distribution with variance I.

[0079]

[0080] M = (u k ) k＝1 K is a parameter. K is the number of classes in the target dataset. P(y n r =l|t m = k, A) = α lk r is the probability that the rth annotator labels the true class label k as l. A (Equation (8)) is a parameter.

[0081]

[0082] In this model, M, π, and A are parameters specific to the dataset (task), and the goal is to learn a classifier specific to the dataset (task) by estimating these from the training data S.

[0083] These parameters M, π, and A can be learned by maximizing the posterior distribution when the probability model (Equation (5)) described above is used as the likelihood function. Specifically, this is Equation (9).

[0084]

[0085] Here, the two terms on the rightmost side of equation (9) represent a prior distribution. Specifically, the conjugate prior distribution given by the following equation (10) is used.

[0086]

[0087] This local maximization of the objective function can be achieved by the EM algorithm. Specifically, the parameters M, π, and A can be found by alternately and sequentially updating the following E step (Equation (11)) and M step (Equation (12)).

[0088]

[0089]

[0090] The parameters estimated by the EM algorithm are M * , π * , A * Then, using Bayes' theorem, the classifier (probability that the class label of the test data x is k) (second classifier) ​​is given by equation (13).

[0091]

[0092] Next, we will describe a training method for a classifier generation model using a related dataset D. Here, the noisy training data of pseudo annotators generated from the related dataset D is represented by the symbol S. The training parameter of the proposed classifier generation model is the parameter θ of the neural network f. The objective function is given by Equation (14).

[0093]

[0094] In Equation (14), E represents the expected value. Q (Equation (15)) is labeled data for generalization performance calculation randomly sampled from the d-th dataset (task).

[0095]

[0096] S (Equation (16)) is labeled data for learning randomly sampled from the same dataset, and there is no overlap between S and Q.

[0097]

[0098] Since the related dataset is assumed to be labeled data with true labels, the labels of ￣S are also true.

[0099] To create pseudo-noisy training data, we transform the labels of ￣S. Specifically, B r =(y lk r ) is a matrix whose elements are the probability that the pseudo rth annotator will label the true class label k as l, and this matrix is ​​created by sampling from a predefined annotator distribution p(B).

[0100] This {B r}, let S (Equation (17)) be the noisy training data (pseudo noisy training data) of R pseudo annotators created by probabilistically converting the true labels of ̂S.

[0101]

[0102] The objective function of the classifier generation model shown in equation (14) is the loss when the test data Q is classified by a classifier (first classifier) ​​trained with pseudo-noisy training data S. The classifier (first classifier) ​​obtained by estimating M, π, and A (equations (9) to (12)) is differentiable with respect to the parameter Θ of the neural network f, so this objective function can be optimized by the usual stochastic gradient descent method.

[0103] [Pseudo Code of Learning Method] Fig. 5 is a diagram for explaining the processing of the learning unit 10. Fig. 5 illustrates pseudo code of the processing of the learning unit 10.

[0104] First, the learning unit 10 sets D as a related data set and creates a data set S (number of data N S ), generalization performance calculation data set Q (number of data N Q ) (Algorithm1 Require).

[0105] In the learning phase, the learning unit 10 randomly selects a sample task d, a labeled dataset S for learning, and a labeled dataset Q for generalization performance calculation (lines 2-4 of Algorithm 1).

[0106] As described above, the learning unit 10 samples the annotator distribution p(B) and generates {B r}, we create noisy training data shown in equation (17) (lines 5-6 of Algorithm 1).

[0107] Then, by alternately updating the E step (Equation (11)) and the M step (Equation (12)), the parameter M * , π * , A * By alternately repeating the update formulas shown in formula (11) and formula (12), the local maximization of the objective function (formula (9)) is obtained, and the parameter M * , π * , A * Based on this, we generate the first classifier (lines 9-10 of Aolorithm1).

[0108] The learning unit 10 calculates the objective function of equation (14) (line 12 of Aolorithm1), and updates the parameters of the classifier generation model based on the calculation result (line 13 of Aolorithm1).

[0109] The classification device according to the present embodiment provides a specific improvement over conventional training methods such as those described in Non-Patent Documents 1 and 2, and represents an advancement in the technical field of classification detection performance evaluation through a pre-training method that takes into account multiple annotators.

[0110] [System Configuration of the Embodiment] The components of the classification device 1 are conceptual and functionally conceptual, and do not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of the functions of the classification device 1 is not limited to that shown in the figure, and all or part of the functions can be functionally or physically distributed or integrated in any unit depending on various loads, usage conditions, etc.

[0111] Furthermore, all or any part of the processes performed in the classification device 1 may be realized by a CPU, a GPU (Graphics Processing Unit), and a program analyzed and executed by the CPU and the GPU. Furthermore, each process performed in the classification device 1 may be realized as hardware using wired logic.

[0112] Furthermore, among the processes described in the embodiments, all or part of the processes described as being performed automatically can be performed manually. Alternatively, all or part of the processes described as being performed manually can be performed automatically using a known method. In addition, the processing procedures, control procedures, specific names, and information including various data and parameters described above and illustrated can be changed as appropriate unless otherwise specified.

[0113] 6 is a diagram showing an example of a computer in which a program is executed to realize the classification device 1. The computer 1000 has, for example, a memory 1010 and a CPU 1020. The computer 1000 also has a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0114] The memory 1010 includes a ROM 1011 and a RAM 1012. The ROM 1011 stores a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to a hard disk drive 1090. The disk drive interface 1040 is connected to a disk drive 1100. A removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to a mouse 1110 and a keyboard 1120, for example. The video adapter 1060 is connected to a display 1130, for example.

[0115] The hard disk drive 1090 stores, for example, an OS (Operating System) 1091, application programs 1092, program modules 1093, and program data 1094. That is, the programs that define each process of the classification device 1 are implemented as program modules 1093 in which code executable by the computer 1000 is written. The program modules 1093 are stored, for example, in the hard disk drive 1090. For example, the program modules 1093 for executing processes similar to those of the functional configuration of the classification device 1 are stored in the hard disk drive 1090. The hard disk drive 1090 may be replaced with an SSD (Solid State Drive).

[0116] Furthermore, setting data used in the processing of the above-described embodiment is stored as program data 1094, for example, in memory 1010 or hard disk drive 1090. Then, CPU 1020 reads out program module 1093 or program data 1094 stored in memory 1010 or hard disk drive 1090 into RAM 1012 as necessary and executes them.

[0117] The program module 1093 and program data 1094 may not necessarily be stored in the hard disk drive 1090, but may also be stored in, for example, a removable storage medium and read by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and program data 1094 may be stored in another computer connected via a network (such as a local area network (LAN) or a wide area network (WAN)). The program module 1093 and program data 1094 may then be read by the CPU 1020 from the other computer via the network interface 1070.

[0118] Although the present invention has been described above as an embodiment, the present invention is not limited to the descriptions and drawings that form part of the disclosure of the present invention. In other words, other embodiments, examples, and operational techniques that can be made by those skilled in the art based on the present invention are all included in the scope of the present invention.

[0119] REFERENCE SIGNS LIST 1 Classification device 10 Learning unit 11 Learning data input unit 12, 22 Feature extraction unit 13 Pseudo-noisy learning data generation unit 14 Classifier generation model learning unit 15 Storage unit 20 Classification unit 21 Data input unit 23 Learning and classification unit 24 Classification result output unit 151 Classifier generation model

Claims

1. A learning device comprising: an acquisition unit that acquires data that is related to a target dataset and consists of accurately labeled training data as a related dataset; a generation unit that generates pseudo-noisy training data in which data is incorrectly labeled based on the related dataset; and a learning unit that evaluates the performance of a first classifier generated based on the pseudo-noisy training data generated by the generation unit using labeled data from the related dataset, and performs learning of a classifier generation model to generate a classifier so as to improve the evaluation result.

2. The learning device described in claim 1, further comprising a classification unit that, when given a small amount of noisy learning data for a target task, generates a second classifier corresponding to the target task using the classifier generation model trained by the learning unit, and classifies test data for the target task using the generated second classifier.

3. The learning device described in claim 1, characterized in that the generation unit inputs labeled data randomly extracted from the related dataset into a plurality of functions having different labeling methods and / or error characteristics, and generates the pseudo-noisy learning data.

4. A learning method executed by a learning device, comprising: a step of acquiring data that is related to a target dataset and is composed of accurately labeled learning data as a related dataset; a step of pseudo-generating noisy learning data in which data is incorrectly labeled based on the related dataset; and a step of evaluating the performance of a first classifier generated based on the pseudo noisy learning data generated in the generating step using labeled data from the related dataset, and performing learning of a classifier generation model to generate a classifier so as to improve the evaluation results.

5. A learning program for causing a computer to execute the steps of: acquiring data associated with a target dataset and consisting of accurately labeled training data as an associated dataset; generating pseudo-noisy training data in which data is incorrectly labeled based on the associated dataset; and evaluating the performance of a first classifier generated based on the pseudo-noisy training data generated in the generating step using labeled data from the associated dataset, and performing training of a classifier generation model to generate a classifier so as to improve the evaluation results.

Images