Classification device, training method, and training program

Abstract

A classification device (10) creates a loss function that decreases the better the classification performance for imbalanced data of a classification model for a test data distribution using unlabeled data that is data based on the test data distribution and does not have binary labels that represent a positive class and a negative class, unlabeled data that is data based on a training data distribution and does not have the binary labels, and labeled data (for example, positive data) that is data based on the training data distribution and has the binary label for the positive class. The classification device (10) updates the parameters of the classification model so as to reduce the loss function.

Description

Classification device, learning method, and learning program

[0001] The present invention relates to a classification device, a learning method, and a learning program.

[0002] A binary classification problem in machine learning is the problem of classifying given data into one of two class labels. For example, in intrusion detection in network security, given communication data is classified into either a malicious or benign class label. In medical imaging diagnosis, given patient image data is classified as whether the patient has a certain disease or not. Hereinafter, by convention, one of the two class labels will be referred to as the "positive" class and the other as the "negative" class.

[0003] Here, a method for learning a classification model for a binary classification problem, which is capable of adapting to changes in data distribution, has been proposed (see, for example, Non-Patent Document 1).

[0004] Hammoudeh, Zayd, and Daniel Lowd. "Learning from positive and unlabeled data with arbitrary positive shift."Advances in Neural Information Processing Systems 33 (2020): 13088-13099.

[0005] However, in the conventional techniques, there are cases where it is not possible to train a classification model with high accuracy.

[0006] In real-world binary classification problems, the number of positive data is often extremely small compared to the number of negative data. In the example above, the amount of malicious communication data is much smaller than the amount of non-malicious data. The number of people suffering from a certain disease is much smaller than the number of people who do not. This type of data is called "imbalanced data." In such cases, when using common machine learning techniques, the small amount of positive data is ignored during learning, resulting in a classification model with poor performance.

[0007] Additionally, in the real world, data distribution (properties) often differs between training and testing (operation). For example, in network security, the presence of attackers causes trends in malicious data (positive data) to change over time. As a result, the distribution of data collected during training may differ from that of data collected during testing. In this case, too, if a typical machine learning technique is used that does not take changes in data distribution into account, the performance of the resulting classification model will decline due to the changes in data.

[0008] In contrast, the technology described in Non-Patent Document 1 is intended for non-imbalanced data, and when applied to imbalanced data, performance deteriorates and a classification model cannot be obtained with high accuracy.

[0009] The present invention has been made in view of the above, and has an object to perform learning of a classification model with high accuracy.

[0010] In order to solve the above-mentioned problems and achieve the object, a classification device according to the present invention is characterized by having: a creation unit that creates a loss function, which is an index based on scores output by a classification model for: first data based on a first distribution that is not assigned a binary label representing a positive class or a negative class; second data based on a second distribution that is not assigned the binary label; and third data based on the second distribution that is assigned the positive class label from the binary labels, the loss function becoming smaller as the classification performance of the classification model for imbalanced data for the first data improves; and an update unit that updates parameters of the classification model so that the loss function becomes smaller.

[0011] In order to solve the above-mentioned problems and achieve the object, a classification device according to the present invention comprises: a calculation unit that inputs data based on the first distribution into the classification model and calculates a score, the score being an index based on scores output by the classification model for: first data based on a first distribution that is not assigned a binary label representing a positive class or a negative class; second data based on a second distribution that is not assigned the binary label; and third data based on the second distribution that is assigned the positive class label from the binary labels; the loss function being smaller as the classification performance of the classification model for imbalanced data for the first data improves; and a classification result output unit that outputs a result of classifying the first data based on the score calculated by the calculation unit.

[0012] According to the present invention, it is possible to learn a highly accurate classification model.

[0013] Fig. 1 is a diagram illustrating an example of the configuration of a classification device according to a first embodiment. Fig. 2 is a diagram illustrating an overview of processing by the classification device. Fig. 3 is a flowchart illustrating the flow of learning processing by the classification device according to the first embodiment. Fig. 4 is a flowchart illustrating the flow of inference processing by the classification device according to the first embodiment. Fig. 5 is a diagram illustrating an example of a computer that executes a program.

[0014] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to this embodiment. In addition, in the description of the drawings, the same parts are designated by the same reference numerals.

[0015] 1 is a diagram showing an example of the configuration of a classification device according to Embodiment 1. As shown in FIG. 1, a classification device 10 includes a learning unit 11 and an inference unit 12.

[0016] In the learning phase, the learning unit 11 learns a classification model for binary classification. In the inference phase, the inference unit 12 performs inference using the trained classification model. Binary classification is a task of classifying given data into either a positive class or a negative class. Note that the names of classes in binary classification are not limited to positive and negative, and may be TRUE and FALSE, 1 and 0, normal and abnormal, etc.

[0017] An overview of the processing of the classification device 10 will be described using FIG. 2. FIG. 2 is a diagram for explaining an overview of the processing of the classification device. Labels represent positive classes and negative classes. Data labeled with a positive class is called positive data. Data labeled with a negative class is called negative data. Data that is not labeled with any label is called unlabeled data. For example, unlabeled data is data whose class is unknown, whether it belongs to the positive class or the negative class.

[0018] Here, the training data is a training data distribution P ｔｒ and includes data that has been assigned a positive label. The test data is also classified into the test data distribution P ｔｅ The data conforms to the above, and all of the data may be unlabeled. In the inference process, classification of the test data, i.e., estimation of the label of the test data, is performed.

[0019] Both the training data and the test data may be imbalanced data, which is data in which the number of positive data is significantly smaller than the number of negative data.

[0020] The training unit 11 of the classification device 10 trains the classification model using positive data sampled from the training data distribution, unlabeled data sampled from the training data distribution, and unlabeled data sampled from the test data distribution. These data are collectively referred to as training data.

[0021] Specifically, the learning unit 11 uses the above-mentioned learning data to approximate the evaluation index for imbalanced data of the test data distribution, and then performs learning of the classification model so that the evaluation index is optimized.

[0022] The evaluation index in this embodiment is, for example, an index based on the area under the ROC curve (AUC). Conventional AUC is an index that can be calculated using both positive data and negative data.

[0023] In this embodiment, the test data in the training data is unlabeled data only, so the conventional AUC cannot be calculated for the test data distribution.

[0024] In contrast, the classification device 10 of this embodiment makes assumptions about changes in data distribution between training data and test data, and is therefore able to calculate evaluation indices for the test data distribution from the training data.

[0025] As a result, the classification device 10 can optimize the output of the binary classification model for imbalanced data. Note that the classification device 10 may use an evaluation index based on class-balanced loss or the like in addition to the conventional AUC.

[0026] The inference unit 12 of the classification device 10 uses the trained classification model to output a classification result (positive or negative) of test data, which is data for inference.

[0027] For example, in a binary classification problem related to network security, the data in the training data distribution is data collected in the past, and the data in the test data distribution is data collected most recently. The test data in the inference data is also obtained from the most recently collected data.

[0028] 1 , the learning unit 11 includes a learning data input unit 111, a feature extraction unit 112, a model learning unit 113, and a storage unit 114. The learning data input unit 111 receives learning data as input. The learning data includes positive data sampled from the learning data distribution, unlabeled data sampled from the learning data distribution, and unlabeled data sampled from the test data distribution.

[0029] The feature extraction unit 112 converts each input data into a feature vector. Here, a feature vector is a representation of the features of the required data as an n-dimensional numerical vector. The feature extraction unit 112 uses a method commonly used in machine learning. For example, if the data is text, the feature extraction unit 112 performs feature extraction using morphological analysis, n-grams, or delimiters.

[0030] The model training unit 113 uses the data converted into feature vectors to train a classification model so as to optimize (maximize or optimize) the value of the evaluation index for imbalanced data in the test data distribution. The model training unit 113 can calculate the evaluation index without using positive data and negative data in the test data distribution.

[0031] The storage unit 114 stores the trained classification model. For example, the storage unit 114 stores parameters such as weights of the classification model, which is a neural network. Also, for example, the storage unit 114 stores parameters such as regression coefficients of the classification model, which is a regression model.

[0032] The inference unit 12 includes a data input unit 121, a feature extraction unit 122, a classification unit 123, and a classification result output unit 124. The data input unit 121 receives data for inference as input. The data for inference is unlabeled data sampled from a test data distribution.

[0033] The feature extraction unit 122 converts each input data into a feature vector in the same manner as the feature extraction unit 112 .

[0034] The classification unit 123 inputs the feature vector into a classification model and calculates a score. The classification result output unit 124 outputs the result of classifying the data for inference based on the score.

[0035] The classification device 10 of this embodiment can perform learning that accommodates both imbalanced data and changes in data distribution. For example, at least one of the following three assumptions can be made regarding changes in data distribution:

[0036] (Assumption 1: Positive Data Distribution Shift) Positive data distribution shift is the assumption that the negative data distribution is constant between training data and test data, but the positive data distribution can change. For example, in the case of network security, attackers will rapidly change the properties of malicious data in an attempt to evade defense systems, but benign data is thought to change less than malicious data, so assumption 1 is (approximately) satisfied.

[0037] (Assumption 2: Covariate Shift) Covariate shift is the assumption that the distribution p(x) of data x changes between training and testing, but the distribution p(x|y) of class label y given x remains unchanged between training and testing.

[0038] (Assumption 3: Class prior probability shift) The class prior probability shift is the assumption that the distribution p(y) of the class label y changes between training and testing, but the data distribution p(x|y) for each class remains unchanged between training and testing. The class prior probability shift means that the proportion of positive data contained in the unlabeled data changes.

[0039] Examples based on the above assumptions are described below. Examples 1 and 2 are based on a positive data distribution shift. Example 3 is based on a covariate shift. Example 4 is based on a class prior probability shift.

[0040] [Example 1] The problem setting of Example 1 will be described. From the training data distribution, positive data X ｔｒ ｐ (Equation (1)) and unlabeled data X ｔｒ (Equation (2)) is given.

[0041]

[0042]

[0043] where x is the feature vector, y is the class label (y = +1 is positive, y = -1 is negative), and p ｔｒ (x) is the marginal density function of the training data distribution, p ｔｒ ｐ :=p ｔｒ (x | y = +1) is the density function of the positive class of the training data distribution, p ｔｒ ｎ :=p ｔｒ (x | y = -1) is the density function of the negative class of the training data distribution, π ｔｒ :=p ｔｒ (y = +1) is the prior distribution of the positive class of the training data distribution. ｔｒ The value of x may be known, ｔｒ ｐ and x ｔｒ It may be estimated from

[0044] Unlabeled data X from the test data distribution ｔｅ is given by equation (3).

[0045]

[0046] Here, p ｔｅ (x) is the marginal density function of the test data distribution, p ｔｅ ｐ :=p ｔｅ (x | y = +1) is the density function of the positive class of the test data distribution, p ｔｅ ｎ :=p ｔｅ (x | y = -1) is the density function of the negative class of the test data distribution, π ｔｅ :=p ｔｅ (y=+1) is the prior distribution of the positive class of the test data distribution.

[0047] It is also assumed that the relationship of equation (4) holds between the training data distribution and the test data distribution.

[0048]

[0049] That is, equation (4) assumes that the negative data distribution is invariant between training and testing. Under this assumption, the classification device 10 learns a score function (a function that outputs a positive score for x when x is given) (equation (5)) that maximizes the AUC in the test data distribution.

[0050]

[0051] If the score function can be learned, the classification device 10 can classify x's with scores greater than a threshold into a positive class and the rest into a negative class. The score function may be any model (e.g., a linear model or a neural network). The score function is an example of a classification model.

[0052] The AUC for the test data distribution is expressed by equation (6).

[0053]

[0054] E is the expected value, f(x, x'):=σ(s(x)-s(x')), and σ is the sigmoid function. AUC takes values ​​between 0 and 1, and when data in the positive class are scored higher than data in the negative class, the AUC takes a large value. This nature of ranking makes AUC an appropriate metric for imbalanced data. On the other hand, widely used metrics such as accuracy are not suitable for imbalanced data, because when the ratio of positive data to negative data is 1:99, for example, simply classifying all data as negative will result in a high accuracy of 99%.

[0055] Since AUC depends on the positive and negative data distributions of the test data distribution, it cannot be calculated by conventional techniques when only unlabeled data is provided from the test data distribution. On the other hand, according to Example 1, AUC can be approximately calculated even when only unlabeled data is provided from the test data distribution.

[0056] The following describes a procedure for calculating an evaluation index based on AUC, that is, an index that approximates AUC, in Example 1. First, equation (7) holds from equation (3).

[0057]

[0058] Substituting equation (7) into the expected value part of equation (6) gives equation (8).

[0059]

[0060] Here, due to the symmetry of the sigmoid function (σ(z) + σ(-z) = 1, for all z), the second term on the right side of equation (8) is a constant and is therefore irrelevant to the optimization of the score function s. Furthermore, equation (9) holds true from equations (2) and (4).

[0061]

[0062] Substituting equation (9) into equation (8), equation (10) is obtained.

[0063]

[0064] Here, C is a constant that is unrelated to the optimization of the score function s. Also, the coefficient 1 / π ｔｅ (1-π ｔｒ ) only changes the overall scale and is therefore irrelevant to the optimization of the score function s. As a result, the loss to be optimized in equation (10) is given by equation (11) with the sign inverted.

[0065]

[0066] Each expected value in equation (11) can be approximated by equation (12) using the positive data and unlabeled data in the training data distribution and the unlabeled data in the test data distribution. That is, the classification device 10 performs training using equation (12) as a loss function, thereby obtaining a score function (classification model) that maximizes the AUC in the test data distribution.

[0067]

[0068] In the first embodiment, the only necessary teaching information is the positive label of the training data distribution, which has the advantage of reducing the labeling cost and being adaptable to changes in the data distribution of imbalanced data.

[0069] The operation of the classification device 10 of the first embodiment can be restated as follows: That is, the classification device 10 creates a loss function that decreases as the score output by the classification model for unlabeled data based on the test data distribution (unlabeled data in the test data distribution) increases compared to the score output by the classification model for unlabeled data based on the training data distribution (unlabeled data in the training data distribution), and that increases as the score output by the classification model for unlabeled data in the test data distribution increases as the score output by the classification model for positively labeled data based on the training data distribution (positive data in the training data distribution) increases.

[0070] Here, equation (10) is the index in Example 1. The index is calculated from unlabeled data in the test data distribution, unlabeled data in the training data distribution, and positive data in the training data distribution. In other words, labeled data in the test data distribution is not required to calculate the index. Furthermore, while the index is different from the conventional AUC, it is an approximation of the conventional AUC and is suitable for evaluating classification models for imbalanced data. The loss function in equation (11) becomes smaller as the index in equation (10) improves (in this case, the larger it is).

[0071] [Example 2] In Example 1, it was assumed that positive data and unlabeled data were obtained from the training data distribution. However, in real problems, negative data may be obtained from the training data distribution. If the same assumption as in Example 1 can be made about the distribution shift (the negative data distribution is unchanged between training and testing), the negative data obtained from the training data distribution can be naturally used to optimize indicators for imbalanced data in the test data distribution.

[0072] In the second embodiment, the equation (10) is rewritten as shown in equation (13) by removing the constant terms and coefficients that are not required for optimizing the score function s.

[0073]

[0074] Equation (13) utilizes the distribution shift assumption (the negative data distribution is unchanged between training and testing). The loss in equation (13) can be approximated by unlabeled data in the test data distribution and negative data in the training data distribution. Therefore, the classification device 10 can perform training using equation (14), which is the weighted average of the loss derived in Example 1 (equation (12)) and the loss in Example 2 (equation (13)), as the loss function. Note that α is a hyperparameter for weighting that takes a value between 0 and 1.

[0075]

[0076] [Example 3] In Example 1, a positive data distribution shift was assumed as a change in data distribution. In Example 3, a covariate shift (p ｔｅ (x)≠p ｔｒ (x), p ｔｅ (y|x) = p ｔｒ It is assumed that (y|x) = : p(y|x)).

[0077] In Example 3, the AUC in the test data distribution is as shown in equations (15-1) to (15-5). The conversion from equation (15-1) to equation (15-2) and from equation (15-3) to equation (15-4) utilizes the covariate shift assumption and Bayes' formula.

[0078]

[0079] Here, w(x):=p ｔｅ (x) / p ｔｒ (x). w(x) is estimated from unlabeled data of the training data distribution and the test data distribution using any density ratio estimation method. The classification device 10 can train a classification model using a loss function obtained by approximating the equation obtained by substituting the estimated w(x) into equation (15-5) with the positive data and unlabeled data of the training data distribution.

[0080] [Example 4] In Example 1, a positive data distribution shift was assumed as a change in data distribution. In Example 4, a class prior distribution shift (p ｔｅ (y)≠p ｔｒ (y), pｔｅ (x|y)=p ｔｒ It is assumed that (x|y) = : p(x|y)).

[0081] In Example 4, the AUC in the test data distribution is as shown in equations (16-1) to (16-4). The conversion from equation (16-1) to equation (16-2) utilizes the assumption of the class prior distribution. The conversion from equation (16-2) to equation (16-3) utilizes equation (2). The conversion from equation (16-3) to equation (16-4) is achieved by setting the second term, which is a constant, to C.

[0082]

[0083] Here, 1 / (1-π) in equation (16-4) ｔｒ ) is irrelevant to the optimization of the score function s. The classification device 10 can train a classification model using a loss function obtained by approximating equation (16-4) with positive data and unlabeled data of the training data distribution.

[0084] [Processing of First Embodiment] The flow of the learning process of the classification device 10 will be described with reference to Fig. 3. Fig. 3 is a flowchart showing the flow of the learning process of the classification device according to the first embodiment.

[0085] 3, first, the training data input unit 111 receives a dataset as input (step S101). The dataset includes positive data and unlabeled data from the training data distribution, and unlabeled test data. However, in Example 2, negative data from the training data distribution is included in the dataset. The feature extraction unit 112 converts each piece of data into a feature vector (step S102).

[0086] The model learning unit 113 learns the classification model (step S103). The model learning unit 113 may include a calculation unit that calculates a score using the classification model, a creation unit that creates a loss function, and an update unit that updates parameters of the classification model (score function) based on the loss function. For example, the loss function is an approximation of Equation (12), Equation (14), Equation (15-5), or Equation (16-4).

[0087] The flow of the inference process of the classification device 10 will be described with reference to Fig. 4. Fig. 4 is a flowchart showing the flow of the inference process of the classification device according to the first embodiment.

[0088] The data input unit 121 receives test data as input (step S201), and the feature extraction unit 122 converts the test data into a feature vector (step S202).

[0089] The classification unit 123 classifies the test data using the trained classification model (step S203). The classification unit 123 may include a calculation unit that calculates a score using the classification model. The classification result output unit 124 outputs the classification result (step S204).

[0090] [Effects of the First Embodiment] As described above, the classification device 10 creates a loss function, which is an index based on scores output by a classification model for first data based on a first distribution that is not assigned a binary label representing a positive class or a negative class, second data based on a second distribution that is not assigned a binary label, and third data based on the second distribution that is assigned a binary positive class label, and which decreases as the classification performance of the classification model for imbalanced data for the first data improves. The classification device 10 updates the parameters of the classification model so as to decrease the loss function.

[0091] For example, the first data based on the first distribution is unlabeled data based on the test data distribution, the second data based on the second distribution is unlabeled data based on the training data distribution, and the third data based on the second distribution is positive data based on the training data distribution.

[0092] According to the first embodiment, learning can be performed that accommodates both imbalanced data and changes in data distribution, enabling learning of a highly accurate classification model.

[0093] Although Reference 1 describes a method for optimizing AUC, this method does not take into account changes in data distribution.

[0094] Reference 1: Brefeld, Ulf, and Tobias Scheffer. "AUC maximizing support vector learning." Proceedings of the ICML 2005 workshop on ROC Analysis in Machine Learning. 2005.

[0095] Furthermore, Reference 2 describes a learning method that can accommodate changes in data distribution, but this method is intended for non-imbalanced data.

[0096] Reference 2: Ganin, Yaroslav, and Victor Lempitsky. "Unsupervised domain adaptation by backpropagation." International conference on machine learning. PMLR, 2015.

[0097] [Program] In one embodiment, the classification device 10 can be implemented by installing a program that executes the above-described learning process or inference process as package software or online software on a desired computer. For example, by executing the above-described program on an information processing device, the information processing device can function as the classification device 10. The information processing device referred to here includes desktop and notebook personal computers. Other examples of information processing devices include smartphones, tablet terminals, and the like.

[0098] 5 is a diagram showing an example of a computer that executes a program. The computer 1000 includes, for example, a memory 1010 and a CPU (Central Processing Unit) 1020. The computer 1000 also includes a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0099] The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM (Random Access Memory) 1012. The ROM 1011 stores, for example, a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to a hard disk drive 1090. The disk drive interface 1040 is connected to a disk drive 1100. A removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to, for example, a mouse 1110 and a keyboard 1120. The video adapter 1060 is connected to, for example, a display 1130.

[0100] The hard disk drive 1090 stores, for example, an OS (Operating System) 1091, application programs 1092, program modules 1093, and program data 1094. That is, the programs that define each process of the classification device 10 are implemented as program modules 1093 in which computer-executable code is written. The program modules 1093 are stored, for example, in the hard disk drive 1090. For example, the program modules 1093 for executing processes similar to those of the functional configuration of the classification device 10 are stored in the hard disk drive 1090. The hard disk drive 1090 may be replaced by an SSD (Solid State Drive).

[0101] Furthermore, setting data used in the processing of the above-described embodiment is stored as program data 1094, for example, in the memory 1010 or the hard disk drive 1090. The CPU 1020 then reads the program module 1093 or the program data 1094 stored in the memory 1010 or the hard disk drive 1090 into the RAM 1012 as necessary, and executes the processing of the above-described embodiment.

[0102] The program module 1093 and program data 1094 may not necessarily be stored in the hard disk drive 1090, but may also be stored in, for example, a removable storage medium and read by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and program data 1094 may be stored in another computer connected via a network (such as a local area network (LAN) or a wide area network (WAN)). The program module 1093 and program data 1094 may then be read by the CPU 1020 from the other computer via the network interface 1070.

[0103] Although the present invention has been described above as an embodiment, the present invention is not limited to the description and drawings that form part of the disclosure of the present invention. In other words, other embodiments, examples, and operational techniques that can be made by those skilled in the art based on the present invention are all included in the scope of the present invention.

[0104] The following additional notes are provided regarding the above-described embodiments.

[0105] (Supplementary Item 1) A classification device including: a memory; and at least one processor connected to the memory, wherein the processor creates a loss function, which is an index based on scores output by a classification model for first data based on a first distribution that is not assigned a binary label representing a positive class or a negative class, second data based on a second distribution that is not assigned the binary label, and third data based on the second distribution that is assigned the positive class label from the binary labels, the loss function becoming smaller as the index indicating the classification performance of the classification model for imbalanced data for the first data improves; and updates parameters of the classification model so that the loss function becomes smaller. (Supplementary Item 2) A classification device comprising: a memory; and at least one processor connected to the memory, wherein the processor inputs data based on a first distribution into the classification model whose parameters have been updated so as to decrease a loss function, the loss function becoming smaller as the classification performance of the classification model for imbalanced data for the first data improves; the loss function being an index based on scores output by a classification model for first data based on a first distribution and not assigned a binary label representing a positive class or a negative class, second data based on a second distribution and not assigned the binary label, and third data based on the second distribution and assigned a positive class label among the binary labels; and calculates scores; and outputs a result of classifying the first data based on the calculated scores.(Supplementary Item 3) A non-transitory storage medium storing a program executable by a computer, the program creating a loss function, the loss function being an index based on scores output by a classification model for first data based on a first distribution that is not assigned a binary label representing a positive class or a negative class, second data based on a second distribution that is not assigned the binary label, and third data based on the second distribution that is assigned the positive class label from the binary labels, the loss function becoming smaller as the index indicating the classification performance of the classification model for imbalanced data for the first data improves, and causing the computer to execute a process of updating parameters of the classification model so that the loss function becomes smaller. (Supplementary Item 4) A non-transitory storage medium storing a program executable by a computer, the program causing a computer to execute the following processes: inputting data based on the first distribution into the classification model whose parameters have been updated so as to decrease a loss function, which decreases as the classification performance of the classification model for imbalanced data for the first data improves; calculating a score; and outputting a result of classifying the first data based on the calculated score. The loss function is an index based on scores output by a classification model for: first data based on a first distribution that is not assigned a binary label representing a positive class and a negative class; second data based on a second distribution that is not assigned the binary label; and third data based on the second distribution that is assigned a positive class label from the binary labels.

[0106] REFERENCE SIGNS LIST 10 Classification device 11 Learning unit 12 Inference unit 111 Learning data input unit 112, 122 Feature extraction unit 113 Model learning unit 114 Storage unit 121 Data input unit 123 Classification unit 124 Classification result output unit

Claims

1. A classification device comprising: a creation unit that creates a loss function, which is an index based on scores output by a classification model for first data based on a first distribution that is not assigned a binary label representing a positive class or a negative class, second data based on a second distribution that is not assigned the binary label, and third data based on the second distribution that is assigned the positive class label from the binary labels, the loss function becoming smaller as the index indicating the classification performance of the classification model for imbalanced data for the first data improves; and an update unit that updates parameters of the classification model so that the loss function becomes smaller.

2. A classification device comprising: a calculation unit that inputs data based on the first distribution into the classification model whose parameters have been updated so as to decrease a loss function that decreases as the classification performance of the classification model for imbalanced data for the first data improves; and a classification result output unit that outputs a result of classifying the first data based on the score calculated by the calculation unit, the loss function being an index based on scores output by the classification model for: first data based on a first distribution that is not assigned a binary label representing a positive class or a negative class; second data based on a second distribution that is not assigned the binary label; and third data based on the second distribution that is assigned the positive class label from the binary labels.

3. A learning method executed by a computer, comprising: a creation step of creating a loss function, the loss function being an index based on scores output by a classification model for: first data based on a first distribution that is not assigned a binary label representing a positive class or a negative class; second data based on a second distribution that is not assigned the binary label; and third data based on the second distribution that is assigned the positive class label from the binary labels; the loss function becoming smaller as the index indicating the classification performance of the classification model for imbalanced data for the first data improves; and an update step of updating parameters of the classification model so that the loss function becomes smaller.

4. A learning program that creates an index based on scores output by a classification model for first data based on a first distribution that is not assigned a binary label representing a positive class or a negative class, second data based on a second distribution that is not assigned the binary label, and third data based on the second distribution that is assigned the positive class label from the binary labels, the loss function being smaller as the index indicating the classification performance of the classification model for imbalanced data for the first data improves, and causes a computer to execute a process of updating the parameters of the classification model so that the loss function becomes smaller.

Images