Identification device, identification device generation device, identification device generation method and program

Abstract

To improve identification accuracy. [Solution] In the identification device 100, the classifier 30 acquires a scale indicating the likelihood that the input data belongs to each of several classes, and outputs a predicted value based on a scale corrected by a correction parameter that corrects the scale indicating the likelihood that the data belongs to each of several classes. The identification processing unit 115 uses the classifier 30 to identify the class to which the input data belongs.

Description

Technical Field

[0001] The present invention relates to an identification device, an identifier generation device, an identifier generation method, and a program.

Background Art

[0002] Conventionally, an identifier (also called a classifier) that determines to which class among a plurality of classes input data belongs using a neural network, SVM (Support Vector Machine), etc. is known and is used in various fields. For example, there is an identifier that outputs the type of flower when an image of a flower is input with the type of flower as a class. Also, there is an identifier that outputs the name of a skin disease when an image of the skin is input with the name of the skin disease as a class.

[0003] Such an identifier is generated by learning using a large number of learning data for which the classes are known. However, if the composition ratio of the classes included in the learning data is extremely biased, that is, if the imbalance in the number of samples per class is large, it will be biased towards the class with a high ratio (the class with a large number of samples), and there is a problem that the accuracy of identification decreases. To solve this problem, in Patent Document 1, even when the number of learning data belonging to one class is extremely smaller than the number of the other class, a part of the learning data of the minority class is replicated and increased (oversampling), or a part of the learning data of the majority class is removed and decreased (undersampling) to adjust the balance of the learning data, and a technique for performing highly accurate identification has been proposed.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, achieving an appropriate balance in sample size between classes is not easy, as it is greatly influenced not only by the number of samples in each class but also by the distribution of sample data near the boundaries between classes.

[0006] The present invention aims to solve the above-mentioned problems and to provide an identification device, an identification device, an identification device, an identification device generation method, and a program that can improve identification accuracy. [Means for solving the problem]

[0007] To achieve the above objective, the identification device according to the present invention is characterized by comprising an identification processing unit that identifies the class to which the input data belongs by using a classifier that acquires a scale indicating the likelihood that the input data belongs to each of a plurality of classes and outputs a predicted value based on a scale corrected by a correction parameter that corrects the scale indicating the likelihood that the input data belongs to each of the plurality of classes. [Effects of the Invention]

[0008] According to the present invention, it is possible to improve the accuracy of identification. [Brief explanation of the drawing]

[0009] [Figure 1] This is a block diagram showing the configuration of the identification device according to Embodiment 1. [Figure 2] This is a diagram showing the configuration of the identifier according to Embodiment 1. [Figure 3] This figure shows an example of creating a confusion matrix in Embodiment 1. [Figure 4] This figure shows an example of correcting the correction parameters using the confusion matrix created in Figure 3. [Figure 5] This figure shows an example of recreating the confusion matrix using the correction parameters modified in Figure 4. [Figure 6] This is a flowchart of the classifier generation process according to Embodiment 1. [Figure 7] This is a flowchart of the identification process according to Embodiment 1. [Figure 8] This is a block diagram showing the configuration of the identifier generation device according to Embodiment 2. [Figure 9] This is a block diagram showing the configuration of the identification device according to Embodiment 2. [Modes for carrying out the invention]

[0010] Embodiments of the present invention will be described below with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals. The identification device 100 according to Embodiment 1 is a device that uses a classifier 30 generated by machine learning to identify which of several classes input data belongs to. Examples of the identification device 100 include a device that identifies the name of a disease from an image of the disease, a device that identifies the name of a flower from an image of the flower, and so on.

[0011] As shown in Figure 1, the identification device 100 comprises a control unit 11, a storage unit 12, an operation unit 13, a display unit 14, and a communication unit 15. The control unit 11 includes a CPU (Central Processing Unit), ROM (Read Only Memory), and RAM (Random Access Memory). The CPU is a central processing unit that includes a microprocessor and performs various processes and calculations. In the control unit 11, the CPU reads the control program stored in the ROM and controls the operation of the entire identification device 100 while using the RAM as work memory. The processing of the control unit 11 may be executed by a single CPU or by multiple CPUs.

[0012] The memory unit 12 is a non-volatile memory such as flash memory or a hard disk. The memory unit 12 stores programs and data executed by the control unit 11, and data generated by the control unit 11. Specifically, the memory unit 12 stores the training dataset 121, the test dataset 122, and the classifier 30. Details of these will be described later.

[0013] The operation unit 13 is equipped with input devices such as a keyboard, mouse, and touch panel, and accepts user input. The display unit 14 is equipped with a display device such as a liquid crystal display or an organic EL (Electro-Luminescence) display, and displays various images under the control of the control unit 11. The communication unit 15 is equipped with a communication interface for communicating with external devices of the identification device 100. For example, the communication unit 15 communicates with external devices in accordance with well-known communication standards such as LAN (Local Area Network) and USB (Universal Serial Bus).

[0014] The control unit 11 performs processing in two phases: a first phase, which is the generation phase of the classifier 30, and a second phase, which is the inference phase by the classifier 30. The control unit 11 includes a learning unit 111 and a parameter modification unit 112 as functions in the generation phase. The control unit 11 also includes a data receiving unit 114, a classification processing unit 115, and a result output unit 116 as functions in the inference phase. In the control unit 11, the CPU functions by reading the program stored in ROM into RAM and executing the program to control each of these units. In the control unit 11, one CPU may function as the unit for the generation phase and the unit for the inference phase, or multiple CPUs may jointly function as the unit for the generation phase and the unit for the inference phase.

[0015] First, the generation phase will be described. The generation phase is a phase in which an identifier 30 that can accurately identify input data is generated using a machine learning method. Here, the identifier 30 is a computer program for identifying to which class among a plurality of classes the input data to be identified belongs, and is a learned model learned by machine learning using the learning dataset 121.

[0016] More specifically, as shown in FIG. 2, the identifier 30 includes a calculation unit 31 and a correction unit 32. The calculation unit 31 is a unit that executes the main operations in the identifier 30. Specifically described, the calculation unit 31 calculates a logit value, which is an example of a measure indicating the probability that the given input data belongs to each of a plurality of classes, by a method such as logistic regression or DNN (Deep Neural Network). As an example, the calculation unit 31 is composed of a neural network with a multi-layer structure, and has an input layer to which input data is input, an intermediate layer (hidden layer) that performs operations such as convolution and pooling on the input data, and an output layer (fully connected layer) that outputs the operation result. Note that the present invention is not limited to this, and for example, an SVM can also be used as the calculation unit 31.

[0017] Here, assuming that the number of the plurality of classes identified by the identifier 30 is N (N is an integer of 2 or more), that is, the identifier 30 is an N-class identifier, the calculation unit 31 calculates N logit values x1 to xN indicating the probability that the input data input to the input layer belongs to each of the N classes, and outputs them from the fully connected layer. Each of the N logit values x1 to xN means that the larger the value, the higher the probability that the input data belongs to the corresponding class among the N classes. The calculation unit 31 is an example of a first acquisition unit that acquires logit values x1 to xN indicating the probability that the input data belongs to each of a plurality of classes in response to the input of the input data.

[0018] The correction unit 32 corrects the N logit values x1 to xN calculated by the calculation unit 31 through a linear transformation. Specifically described, the correction unit 32 calculates the following formula (1) for each of i = 1 to N, and calculates a correction value yi (where i is any one of 1 to N) from the N logit values x1 to xN. In the following formula (1), “Σexp(xj)” represents the sum “exp(x1)+exp(x2)+exp(x3)+…+exp(xN)” of the values obtained by raising the base e of the natural logarithm to the power of xj (where j is any one of 1 to N). yi = γi×{xi - logΣexp(xj)} …(1)

[0019] In the above formula (1), γi represents a correction parameter. Corresponding to each of the N logit values x1 to xN, there are N correction parameters γ1 to γN. Each of the correction parameters γ1 to γN is a parameter for suppressing the reduction in discrimination accuracy even when there is a bias in the number of samples among multiple classes. The initial value of each of the correction parameters γ1 to γN is 1, and is corrected by a parameter correction unit 112 described later.

[0020] The correction unit 32 converts the N correction values y1 to yN calculated from the logit values x1 to xN into N prediction values q1 to qN through a softmax function and outputs them. The softmax function is a function for normalizing the values of the input vector into a probability distribution. Through the softmax function, the N correction values y1 to yN are converted into N prediction values q1 to qN, each of which is a value between 0 and 1 and the sum is 1. Thereby, the N prediction values q1 to qN can be treated as probability values indicating the probabilities that the input data belongs to each of the N classes. The class corresponding to the maximum prediction value among the N prediction values q1 to qN output by the correction unit 32 becomes the discrimination result by the discriminator 30. The correction unit 32 is an example of a second acquisition unit that corrects the logit values x1 to xN acquired by the calculation unit 31 with the correction parameters γ1 to γN and then acquires the prediction values q1 to qN.

[0021] In this way, the classifier 30 accepts input data and obtains logit values ​​x1 to xN, which are measures of the likelihood that the input data belongs to each of the multiple classes. The classifier 30 then corrects the logit values ​​x1 to xN using correction parameters γ1 to γN and outputs predicted values ​​(probability values) q1 to qN, which are values ​​based on the corrected logit values ​​x1 to xN.

[0022] Returning to Figure 1, in the generation phase, the learning unit 111 generates a classifier 30 by performing machine learning using the training dataset 121. Here, the training dataset 121 is the dataset (training dataset) used by the learning unit 111 to perform machine learning. The training dataset 121 contains multiple training input data (hereinafter referred to as "training data") used as training data. Each of the training data is data in which the class to which the training data belongs among multiple classes, i.e., the correct class (ground truth class), is known in advance. In the training dataset 121, each training data is labeled with the ground truth class as the ground truth label.

[0023] For example, if the identification device 100 is a device that identifies disease names from disease images, then multiple candidate disease names are predetermined as multiple classes, and each training data is assigned the class corresponding to the correct disease name among them as the correct class. As another example, if the identification device 100 is a device that identifies flower names from flower images, then multiple candidate flower names are predetermined as multiple classes, and each training data is assigned the class corresponding to the correct flower name among them as the correct class.

[0024] The learning unit 111 fixes all the correction parameters γ1 to γN to their initial value of 1.0 and performs supervised learning using multiple training data included in the training dataset 121 as training data. Specifically, the learning unit 111 inputs the training data to the calculation unit 31. The calculation unit 31 calculates logit values ​​x1 to xN for the input training data, and the correction unit 32 converts the logit values ​​x1 to xN into correction values ​​y1 to yN using equation (1) above, and further converts the correction values ​​y1 to yN into predicted values ​​q1 to qN using the softmax function and outputs them.

[0025] The learning unit 111 adjusts the computation parameters of the calculation unit 31 using methods such as backpropagation so that among the N predicted values ​​q1 to qN output from the classifier 30, the predicted value corresponding to the correct class of the input training data approaches 1, and the other predicted values ​​approach 0. The computation parameters of the calculation unit 31 are, for example, the weights of the connections in each layer of the neural network. The learning unit 111 optimizes the parameters of the calculation unit 31 by performing this process for each of the multiple training data included in the training dataset 121. Through this learning process by the learning unit 111, a classifier 30 is generated in the state before the correction parameters γ1 to γN are corrected by the parameter correction unit 112, that is, in a state where all values ​​of the correction parameters γ1 to γN are set to their initial value of 1.0.

[0026] Returning to Figure 1, the parameter correction unit 112 corrects the correction parameters γ1 to γN used by the correction unit 32 of the classifier 30. Here, if there is a bias in the number of samples (number of training data) for each class in the multiple training data included in the training dataset 121, the classifier 30 may be biased towards the class with a larger number of samples, potentially reducing its classification accuracy. The parameter correction unit 112 corrects the correction parameters γ1 to γN to suppress such a decrease in classification accuracy due to a bias in the number of samples.

[0027] The process of modifying the correction parameters γ1 to γN will be explained in detail below with reference to Figures 3 to 5. In the following explanation, we will use the case where the classifier 30 is a three-class classifier that identifies the class to which the input data belongs from among three classes, "A", "B", and "C", i.e., N=3, as an example. The same explanation is possible for cases where N is other than 3.

[0028] The parameter correction unit 112 creates a confusion matrix 50 based on the classifier 30 generated by the learning unit 111 and the test dataset 122 pre-prepared in the storage unit 12. Here, the test dataset 122 is the dataset used by the parameter correction unit 112 to correct the correction parameters γ1 to γ3. The test dataset 122 contains multiple test input data (hereinafter referred to as "test data"). Each of the multiple test data is data in which the correct class is known in advance, similar to the learning data. In the test dataset 122, each test data is pre-assigned with the correct class as the correct label. Note that all or part of the multiple test data in the test dataset 122 may be the same as the learning data.

[0029] The parameter modification unit 112 inputs each test data included in the test dataset 122 to the classifier 30 generated by the learning unit 111. In the classifier 30, the calculation unit 31 calculates logit values ​​x1 to x3 for each input test data, and the correction unit 32 converts the logit values ​​x1 to x3 into correction values ​​y1 to y3 using equation (1) above, and further converts the correction values ​​y1 to y3 into predicted values ​​q1 to q3 using the softmax function and outputs them. The parameter modification unit 112 obtains the class corresponding to the maximum value among the predicted values ​​q1 to q3 output from the classifier 30 as the classification result.

[0030] On the left side of Figure 3, the correct class associated with each test data in the test dataset 122 is shown. To the right of the correct class for each test data is shown the predicted values ​​q1 to q3 output from the classifier 30 when each test data is input to the classifier 30, and the classification result showing the class identified by the classifier 30 (classification class). For example, the correct class for the first input data is "C", and the predicted values ​​q1 to q3 output from the classifier 30 are (0.44, 0.13, 0.43). In this case, the maximum value among the predicted values ​​q1 to q3 is q1 = 0.44, and q1 corresponds to class "A", so the classification result is "A".

[0031] The parameter correction unit 112 creates a confusion matrix 50, as shown on the right side of Figure 3, based on the classification results of the classifier 30 for each of these test data. Here, the confusion matrix 50 is data that summarizes the relationship between the correct class and the classification result in matrix form for each of the multiple test data included in the test dataset 122. In the confusion matrix 50, the correct classes that are correct for the test data are represented vertically (column direction), and the classification results that the classifier 30 has identified for the test data are represented horizontally (row direction). The confusion matrix 50 has the corresponding number of samples (number of test data) at each position as a matrix element.

[0032] Specifically, in the example of confusion matrix 50 shown in Figure 3, among the test data with the correct class "A", 10 were identified as "A", 2 as "B", and 0 as "C". Similarly, among the test data with the correct class "B", 3 were identified as "A", 5 as "B", and 1 as "C". Furthermore, among the test data with the correct class "C", 2 were identified as "A", 0 as "B", and 1 as "C".

[0033] The parameter modification unit 112 modifies the correction parameters γ1 to γ3 to enhance the symmetry of the confusion matrix 50. Here, high symmetry of the confusion matrix 50 means that when the input data is entered with an equal number of samples for each class, there is little bias in the sum of the values ​​for each column, i.e., the sum of the values ​​for each identified class. In other words, the parameter modification unit 112 modifies the correction parameters γ1 to γ3 in the confusion matrix 50 so that the bias in the sum of the number of samples for each class for each correct label of the classifier 30 is small among multiple classes.

[0034] To enhance the symmetry of the confusion matrix 50, the parameter correction unit 112 first normalizes the confusion matrix 50. Specifically, as shown in the upper part of Figure 4, the parameter correction unit 112 calculates the total number of test data points horizontally across the confusion matrix 50, that is, for each correct answer class. Then, as shown in the middle part of Figure 4, the parameter correction unit 112 normalizes the sum of each row of the confusion matrix 50 by dividing the element by the sum of its values. In this way, the parameter correction unit 112 normalizes the confusion matrix 50 row by row so that the number of correct answer data points for each class becomes 1. The value of each element in the normalized confusion matrix 50 is called the "prediction rate".

[0035] When the confusion matrix 50 is normalized, the parameter correction unit 112 calculates the sum of elements (hereinafter referred to as the "sum of prediction rate") for each column of the normalized confusion matrix 50, vertically, that is, for each class identified as the identification result. In the example in Figure 4, the sum of prediction rates for class "A" is calculated to be 1.833, for class "B" it is 0.722, and for class "C" it is 0.444, showing that there is considerable variation in these values. In this example, the proportion of cases where the identification result is "A" is quite high, and the symmetry of the confusion matrix 50 is low.

[0036] Furthermore, the parameter correction unit 112 calculates the average accuracy rate. Here, the average accuracy rate is the average value obtained by averaging the proportion of test data where the correct class is "X" and the classification result was "X" across all classes. In the example in Figure 4, the parameter correction unit 112 calculates the average accuracy rate by averaging the values ​​of the elements located on the diagonal of the normalized confusion matrix 50, resulting in "(0.833 + 0.556 + 0.333) ÷ 3 = 0.574". The calculated average accuracy rate of 0.574 is recorded in the lower right column of the confusion matrix 50.

[0037] The parameter modification unit 112 modifies the correction parameters γ1 to γ3 so that the sum of prediction rates for each class in the normalized confusion matrix 50, i.e., the sum in the column direction, approaches 1 as close to 1 as possible, in order to increase the symmetry of the confusion matrix 50. To this end, the parameter modification unit 112 modifies the correction parameter γi so that the predicted value qi for classes with a large sum of prediction rates becomes small, or the predicted value qi for classes with a small sum of prediction rates becomes large. Due to the properties of the linear transformation equation shown in equation (1) above, when the correction parameter γi is greater than 1, the predicted value qi will be smaller than the original value within the range of 0 to 1, and when the correction parameter γi is less than 1, the predicted value qi will be larger than the original value within the range of 0 to 1. Using this property, to increase the symmetry of the confusion matrix 50, the correction parameter γi corresponding to classes with a large sum of prediction rates should be made greater than 1, or the correction parameter γi corresponding to classes with a small sum of prediction rates should be made less than 1.

[0038] In the example in Figure 4, the total prediction rate for class "A" is the highest at 1.833, and the total prediction rate for class "C" is the lowest at 0.444. Therefore, to reduce the predicted value q1 for class "A", the correction parameter γ1 needs to be changed to a value greater than 1, or to increase the predicted value q3 for class "C", the correction parameter γ3 needs to be changed to a value less than 1. However, if multiple parameters are changed simultaneously, the predicted values ​​for each class may change abruptly, potentially leading to unexpected changes in the classification results. Therefore, the parameter correction unit 112 changes only one correction parameter γi at a time.

[0039] Specifically, the parameter correction unit 112 identifies the column in the normalized confusion matrix 50 whose absolute difference (absolute difference) from the reference value of 1 is the largest among the sum of the prediction rates of each column. In the example in Figure 4, the class with the largest absolute difference from the reference value of 1 is "A", and its absolute difference is |1.833-1|=0.833. Then, if the absolute difference of the identified column is greater than the reference value SD, the parameter correction unit 112 corrects the correction parameter γi of the class corresponding to that column.

[0040] The parameter correction unit 112 increases the correction parameter γi for a specified column class if the sum of the prediction rates for that class is greater than 1, and decreases the correction parameter γi for that class if the sum of the prediction rates for that class class is 1 or less. In the example in Figure 4, the sum of the prediction rates for class "A" is 1.833, which is greater than 1, so the parameter correction unit 112 increases the correction parameter γ1 for class "A".

[0041] The amount by which the correction parameter γi is increased or decreased (the amount of change) may be a constant, such as 0.05, or it may be varied according to the value of the variable MD (the maximum value of the absolute difference). For example, if the value of MD is large, the amount of change of the correction parameter γi can be increased, and if the value of MD is small, the amount of change of the correction parameter γi can be decreased. However, if the amount of change of the correction parameter γi is large, the value of the variable MD is more likely to oscillate or diverge, so if it is acceptable to increase the computation time, it is better to make the amount of change as small as possible. In the following explanation, we will assume that the amount of change of the correction parameter γi is a constant of 0.05.

[0042] When the correction parameter γ1 for class "A" is increased by 0.05, the correction parameter γ1 for class "A" becomes 1.05, while the correction parameters γ2 and γ3 for classes "B" and "C" remain at 1. Therefore, when the correction parameters γ1 to γ3 are represented as a vector, they become (1.05, 1.0, 1.0) as shown in the lower part of Figure 4. The parameter correction unit 112 then recreates the confusion matrix 50 using the classification result of the classifier 30 after the correction parameters γ1 to γ3 have been corrected to (1.05, 1.0, 1.0).

[0043] Specifically, the parameter correction unit 112 inputs each test data from the test dataset 122 again into the classifier 30 after the correction parameters γ1 to γ3 have been corrected. In the classifier 30, the calculation unit 31 calculates logit values ​​x1 to x3 for each input test data. The correction unit 32 uses the correction parameters γ1 to γ3 corrected by the parameter correction unit 112 to convert the logit values ​​x1 to x3 into new correction values ​​y1 to y3 using equation (1) mentioned above. Then, the correction unit 32 converts the new correction values ​​y1 to y3 into new prediction values ​​q1 to q3 using the softmax function and outputs them. The parameter correction unit 112 obtains the class corresponding to the maximum value among the new prediction values ​​q1 to q3 output from the classifier 30 as a new classification result.

[0044] When the classifier 30, to which the corrected correction parameters γ1 to γ3 have been applied, obtains a new classification result, the parameter correction unit 112 creates a new confusion matrix 50 based on the new classification result. The parameter correction unit 112 then performs the same processing on the new confusion matrix 50 as it did on the original confusion matrix 50. Specifically, the parameter correction unit 112 performs normalization processing and a process to calculate the sum of prediction rates for each column on the new confusion matrix 50, and identifies the column whose absolute difference from the reference value 1 is the largest among the sum of prediction rates for each column. The parameter correction unit 112 then corrects the correction parameter γi of the identified column's class by increasing or decreasing it, and recreates the confusion matrix 50 using the classification result of the classifier 30 after the correction parameter γi has been corrected.

[0045] In this way, the parameter correction unit 112 creates a confusion matrix 50 based on the classification result of the classifier 30, and repeatedly modifies the correction parameter γi of the class of the column with the largest absolute difference between the sum of prediction rates and the reference value 1 in the created confusion matrix 50. As a result, the parameter correction unit 112 adjusts the correction parameters γ1 to γ3 to enhance the symmetry of the confusion matrix 50.

[0046] Here, assuming that the classification result of the classifier 30 does not change from the original classification result after the correction parameters γ1 to γ3 are modified to (1.05, 1.0, 1.0), the new confusion matrix 50 will be the same as the original confusion matrix 50. In this case, the parameter modification unit 112 further increases the correction parameter γ1 for class "A" by 0.05. As a result, the correction parameters γ1 to γ3 become (1.1, 1.0, 1.0) as shown in the bottom row of Figure 4. The classification result of the classifier 30 after the correction parameters γ1 to γ3 are modified to (1.1, 1.0, 1.0), and the confusion matrix 50 created from that classification result are shown in the upper row of Figure 5.

[0047] More specifically, the upper left of Figure 5 shows the predicted values ​​q1 to q3 output by the classifier 30 when each test data is input to the classifier 30 after the correction parameters γ1 to γ3 have been modified to (1.1, 1.0, 1.0). These predicted values ​​q1 to q3 have changed slightly from the predicted values ​​q1 to q3 shown in Figure 3 because the correction parameters γ1 to γ3 have been modified. As a result, the first classification result has changed from "A" to "C", and the seventh classification result has changed from "A" to "B". From these classification results, the confusion matrix 50 shown in the upper right of Figure 5 is obtained.

[0048] Normalizing the confusion matrix 50 shown in Figure 5 yields the normalized confusion matrix 50 as shown in the middle of Figure 5. Calculating the sum of prediction rates for each class, we find that the sum of prediction rates for class "A" is 1.167, for class "B" is 1.056, and for class "C" is 0.778, indicating increased symmetry compared to the previous confusion matrix 50 (Figure 4). The average accuracy also increased to 0.796, an improvement from the previous version.

[0049] In this case (Figure 5), class "C" is identified as the class with the largest absolute difference from the reference value 1, so MC = "C", and the maximum absolute difference value MD = |0.778 - 1| = 0.222. In this case, since the sum of the prediction rates for class "C", which is 0.778, is less than 1, the parameter correction unit 112 reduces the correction parameter γ3 for class "C". Then, the correction parameter γ3 for class "C" becomes 0.95, and as shown in the lower part of Figure 5, the correction parameters γ1 to γ3 become (1.1, 1.0, 0.95).

[0050] The parameter correction unit 112 then recreates the confusion matrix 50 using the classification result of the classifier 30 after the correction parameters γ1 to γ3 have been corrected to (1.1, 1.0, 0.95). The subsequent processing is not shown in the diagram, but the parameter correction unit 112 creates the confusion matrix 50 based on the classification result of the classifier 30 and repeatedly corrects the correction parameter γi of the class in the column with the largest absolute difference between the total prediction rate and the reference value 1 in the created confusion matrix 50. The parameter correction unit 112 repeats this process until the maximum value MD of the absolute difference between the total prediction rate and the reference value 1 becomes less than or equal to the reference value SD, or until the number of processing iterations exceeds the upper limit. Finally, the correction parameters γ1 to γ3 obtained when the total prediction rate of each class is closest to 1 are obtained as the final correction parameters γ1 to γ3.

[0051] Once the final correction parameters γ1 to γ3 are obtained, the parameter correction unit 112 generates a classifier 30 to which the final correction parameters γ1 to γ3 are applied and stores it in the storage unit 12. The classifier 30 to which the final correction parameters γ1 to γ3 are applied is then corrected in the correction unit 32, which corrects the logit values ​​x1 to x3 to correction values ​​y1 to y3 using the final correction parameters γ1 to γ3, and further converts them into predicted values ​​q1 to q3 for output.

[0052] Next, with reference to Figure 6, the flow of the classifier generation process performed by the classifier device 100 in the generation phase will be explained. The classifier generation process shown in Figure 6 starts when the operation unit 13 receives a start command from the user, provided that the training dataset 121 and the test dataset 122 are prepared in the storage unit 12. The classifier generation process shown in Figure 6 is an example of a classifier generation method.

[0053] When the classifier generation process is started, the control unit 11 first initializes the correction parameters γ1 to γN for each class to 1.0 and initializes the counter variable CT to 0 (step S101). Next, the control unit 11 functions as a learning unit 111 and performs machine learning using the learning dataset 121 (step S102). As a result, the control unit 11 generates a classifier 30 in its initial state, that is, a classifier 30 in which all correction parameters γ1 to γN are set to their initial value of 1.0.

[0054] Once the classifier 30 is generated in its initial state, the control unit 11 then functions as a parameter correction unit 112 and performs the process of correcting the correction parameters γ1 to γN in the generated classifier 30. First, the control unit 11 inputs each of the multiple test data contained in the test dataset 122 into the classifier 30 and obtains the predicted values ​​q1 to qN, which are the outputs of the classifier 30 (step S103). If the test dataset 122 has M test data, the control unit 11 obtains M sets of predicted values ​​q1 to qN, each of which is a combination of N values ​​(N-dimensional vector).

[0055] Next, the control unit 11 creates a confusion matrix 50 (step S104). When M sets of predicted values ​​q1 to qN are obtained, the control unit 11 creates an N x N confusion matrix 50 where the sum of the matrix elements is M. In the example shown in Figure 3, the control unit 11 creates a 3 x 3 confusion matrix 50 where the sum of the matrix elements is 24. After creating the confusion matrix 50, the control unit 11 normalizes each row of the confusion matrix 50 so that the sum of the elements is 1, as shown in Figure 4 (step S105). Then, the control unit 11 calculates the total prediction rate, which is the sum of the elements in each column of the normalized confusion matrix 50 (step S106).

[0056] After calculating the sum of the prediction rates for each column, the control unit 11 identifies the column whose absolute difference from the reference value of 1 is the largest among the calculated sum of prediction rates for each column. The control unit 11 then sets the class of the identified column to the variable MC, sets the absolute difference of the identified column, i.e., the maximum absolute difference from 1 among the sum of each column, to the variable MD, and increments the counter variable CT (step S107). In the example shown in Figure 4, the class with the largest absolute difference from the reference value of 1 is "A", so MC = "A", and the maximum absolute difference value MD = |1.833 - 1| = 0.833.

[0057] Next, the control unit 11 determines whether the variable MD (the maximum value of the absolute difference) is less than or equal to the reference value SD (e.g., 0.02) (step S108). If the variable MD is greater than the reference value SD (step S108; NO), the control unit 11 determines whether the counter variable CT is greater than or equal to the upper limit number of times LIMIT (e.g., 100) (step S109). The upper limit number of times LIMIT is a value set to terminate the process if the variable MD does not become less than or equal to the reference value SD after a certain number of processing cycles, because the value of the variable MD may oscillate without becoming less than or equal to the reference value SD, etc.

[0058] If the counter variable CT is less than the upper limit number LIMIT (step S109; NO), the control unit 11 determines whether the sum of the prediction rates for the class of variable MC is greater than 1 (step S110). If the sum of the prediction rates for the class of variable MC is greater than 1 (step S110; YES), the control unit 11 increases the correction parameter γi for the class of variable MC (step S111). Conversely, if the sum of the prediction rates for the class of variable MC is 1 or less (step S110; NO), the control unit 11 decreases the correction parameter γi for the class of variable MC (step S112).

[0059] Subsequently, the control unit 11 returns to step S103. The control unit 11 then repeats the process from steps S103 to S112 until the variable MD becomes less than or equal to the reference value SD in step S108, or until the counter variable CT becomes greater than or equal to the upper limit LIMIT in step S109. Finally, when the variable MD becomes less than or equal to the reference value SD (step S108; YES), or when the counter variable CT becomes greater than or equal to the upper limit LIMIT (step S109; Yes), the control unit 11 determines the correction parameters γ1 to γN at that point as the final correction parameters γ1 to γN (step S113). The control unit 11 then stores the classifier 30 to which the determined correction parameters γ1 to γN have been applied in the storage unit 12, and terminates the classifier generation process shown in Figure 6.

[0060] Returning to Figure 1, the second phase will be described. The inference phase is a phase in which the class to which the unknown input data belongs is identified from among multiple classes using the classifier 30 generated in the generation phase. In the inference phase, the data reception unit 114 receives input of the unknown data to be identified. Here, the unknown data is input data to which the class to which it belongs from among multiple classes is unknown. For example, if the identification device 100 is a device that identifies disease names from disease images, the unknown data is image data of an unknown disease. Or, if the identification device 100 is a device that identifies the name of a flower from an image of a flower, the unknown data is image data of an unknown flower. The data reception unit 114 accepts the designation of the unknown data to be identified from among multiple data pre-stored in the storage unit 12, according to the user's instructions from the operation unit 13. Alternatively, the data reception unit 114 may also receive the unknown data to be identified from an external source via the communication unit 15.

[0061] The identification processing unit 115 uses the classifier 30, whose correction parameters γ1 to γN have been corrected by the parameter correction unit 112, to identify the class to which the unknown data received by the data reception unit 114 belongs. Specifically, the identification processing unit 115 inputs the unknown data received by the data reception unit 114 into the classifier 30. In the classifier 30, the calculation unit 31 calculates logit values ​​x1 to xN for the input unknown data. The correction unit 32 uses the correction parameters γ1 to γN corrected in the classifier generation process to calculate correction values ​​y1 to yN from the logit values ​​x1 to xN, and further converts the correction values ​​y1 to yN into predicted values ​​q1 to qN using the softmax function and outputs them. The identification processing unit 115 identifies the class to which the unknown data belongs as the class corresponding to the largest predicted value among the predicted values ​​q1 to qN output from the classifier 30 in this way.

[0062] The result output unit 116 outputs the identification result from the identification processing unit 115. Specifically, the result output unit 116 displays on the display unit 14 output information indicating the class to which the unknown data is predicted to belong, which is the result identified by the identification processing unit 115. Alternatively, the result output unit 116 may output the output information as audio, or output it to an external device via the communication unit 15. This allows the user to confirm the identification result from the identification device 100.

[0063] Next, with reference to Figure 7, the flow of the identification process performed by the identification device 100 in the inference phase will be explained. The identification process shown in Figure 7 starts when the operation unit 13 receives a start command from the user, while the identification device 30 generated by the identification device generation process shown in Figure 6 is stored in the storage unit 12.

[0064] When the identification process begins, the control unit 11 functions as a data reception unit 114 and accepts input of unknown data to be identified (step S301). Upon receiving the input of unknown data, the control unit 11 functions as an identification processing unit 115 and inputs the unknown data to the classifier 30 to obtain the predicted values ​​q1 to qN for each class, which are the output of the classifier 30 (step S302). The control unit 11 then identifies that the class corresponding to the largest predicted value among the obtained predicted values ​​q1 to qN is the class to which the unknown data belongs (step S303). Next, the control unit 11 functions as a result output unit 116 and outputs the identification result by displaying output information indicating the identified class on the display unit 14, etc. (step S304). With this, the identification process shown in Figure 7 is completed. Because the identification process described above uses a classifier 30 to which the correction parameters γ1 to γN, which were modified in the classifier generation process, have been applied, even if the number of samples for each class in the training data is unbalanced, the influence of this can be eliminated as much as possible, and highly accurate identification can be performed.

[0065] As described above, the identification device 10 according to Embodiment 1 generates a classifier 30 that outputs predicted values ​​q1 to qN, which are predicted values ​​q1 to qN indicating the likelihood that the input data belongs to each of several classes, and which are based on corrected values ​​y1 to yN corrected by correction parameters γ1 to γN. At that time, the identification device 10 according to Embodiment 1 creates a confusion matrix 50 based on the predicted values ​​q1 to qN output from the classifier 30 when test data is input to the classifier 30, and modifies the correction parameters γ1 to γN in the classifier 30 in the created confusion matrix 50 so that the bias in the total number of samples for each class for each correct label of the classifier 30 is small among the several classes. Then, the identification device 10 according to Embodiment 1 uses the classifier 30 with the corrected correction parameters γ1 to γN to identify the class to which the unknown input data to be identified belongs. In this way, the identification device 10 according to Embodiment 1 modifies the correction parameters γ1 to γN in the confusion matrix 50 so that the bias in the total number of samples between classes is small, and thus the balance of the number of samples between classes can be made appropriate. As a result, even if the number of samples in the training data is skewed across classes, the impact of this skew can be mitigated, thereby improving the accuracy of classification.

[0066] Furthermore, corrections using these correction parameters γ1 to γN can enhance or intentionally distort the symmetry of the confusion matrix 50, making them useful in specific applications. For example, in disease differentiation models aimed at assisting physicians in diagnosis, the risk of misclassification varies depending on which disease is classified into which class. In such cases, adjusting the sensitivity using correction parameters γ1 to γN can be expected to reduce the risk during product use.

[0067] Next, Embodiment 2 will be described. The same configuration and functions as in Embodiment 1 will not be described. As shown in Figure 1, the identification device 100 according to Embodiment 1 had both generation phase functions and inference phase functions. In contrast, the identification device 100 according to Embodiment 2 does not have a generation phase function, and a separate device, the identification generator 200, which is separate from the identification device 100, has the generation phase function.

[0068] Specifically, as shown in Figure 8, the classifier generation device 200 according to Embodiment 2 comprises a control unit 21, a storage unit 22, an operation unit 23, a display unit 24, and a communication unit 25. The hardware configuration of each of these units is the same as that of the control unit 11, storage unit 12, operation unit 13, display unit 14, and communication unit 15 in the classifier device 100, so a detailed explanation is omitted. The control unit 21 includes a learning unit 111 and a parameter modification unit 112 as functions of the generation phase. In the control unit 21, the CPU functions as each of these units by reading the program stored in ROM into RAM and executing the program to control it. The storage unit 22 stores a learning dataset 121 and a test dataset 122 as data used in the generation phase.

[0069] Each function of the control unit 21 is the same as that of the learning unit 111 and parameter modification unit 112 provided in the identification device 100 in Embodiment 1. Specifically, the control unit 21 executes the classifier generation process shown in Figure 6 using the functions of the learning unit 111 and parameter modification unit 112. As a result, the control unit 21 generates a classifier 30 with corrected correction parameters γ1 to γN.

[0070] On the other hand, the identification device 100 according to Embodiment 2 is configured as shown in Figure 9. In the identification device 100 according to Embodiment 2, the control unit 11 does not have a generation phase function, but as an inference phase function, it includes a data reception unit 114, an identification processing unit 115, and a result output unit 116. The functions of each of these units are the same as in Embodiment 1. The identification device 100 acquires the classifier 30 generated by the classifier generation device 200 from the classifier generation device 200 via communication via the communication unit 15, etc., and stores it in the storage unit 12. The control unit 11, using the functions of the data reception unit 114, the identification processing unit 115, and the result output unit 116, executes the identification process shown in Figure 7 using the classifier 30 acquired from the classifier generation device 200. As a result, the control unit 11 identifies the class to which the input data to be identified belongs. Thus, in Embodiment 2, since the generation phase and inference phase functions are separated into different devices, more flexible operation becomes possible, such as utilizing the classifier 30 generated by the classifier generation device 200 in multiple classifier devices 100.

[0071] Although embodiments of the present invention have been described above, these embodiments are merely examples, and the scope of application of the present invention is not limited thereto. In other words, the embodiments of the present invention can be applied in various ways, and all embodiments fall within the scope of the present invention.

[0072] For example, in the above embodiment, the correction unit 32 in the classifier 30 converted the logit values ​​x1 to xN to correction values ​​y1 to yN according to equation (1), which is a linear conversion equation based on the correction parameters γ1 to γN. However, the conversion equation in the correction unit 32 is not limited to equation (1) and can be freely designed, and more complex conversions with more parameters are also possible. More specifically, the logit value xi can be corrected using any function that converts the logit value xi to the correction value yi when all variables except the logit value xi are fixed, and this monotonically non-decreasing function (a broadly defined monotonically increasing function) is a function determined by the correction parameters γ1 to γN, and when the input value (logit value xi) is fixed, the change in the output value (correction value yi) due to the correction parameter γi is also monotonically non-decreasing or monotonically non-increasing.

[0073] In the above embodiment, the parameter correction unit 112 determined the correction parameters γ1 to γN when the counter variable CT was greater than or equal to the maximum number of searches LIMIT as the final correction parameters γ1 to γN, provided that the maximum value MD of the difference was not less than or equal to the reference value SD. However, for example, the parameter correction unit 112 may save the correction parameters γ1 to γN when the value of the maximum value MD of the difference was smallest, and if the counter variable CT becomes greater than or equal to the maximum number of searches LIMIT in step S109, it may determine the saved correction parameters γ1 to γN as the final correction parameters γ1 to γN. In this way, even if the value of the maximum value MD of the difference oscillates or diverges, the parameter correction unit 112 can adopt the correction parameters γ1 to γN when the symmetry of the confusion matrix 50 is highest.

[0074] In the above embodiment, the parameter correction unit 112 corrected the correction parameters γ1 to γN without considering the accuracy of the classification by the classifier 30. However, the parameter correction unit 112 may also consider the accuracy. For example, the parameter correction unit 112 may correct the correction parameters γ1 to γN to the parameter that maximizes the average accuracy, which is the average of the accuracy rates of each class, among multiple parameters that reduce the bias in the total number of samples for each class for each correct label of the classifier 30 in the confusion matrix 50 among multiple classes. Specifically, the parameter correction unit 112 calculates the average accuracy when it normalizes the confusion matrix 50 in step S105. Then, in the process of correcting the correction parameters γ1 to γN while repeating the processing in steps S103 to S112, the parameter correction unit 112 stores the correction parameters γ1 to γN that maximize the average accuracy in the storage unit 12. Then, in step S109, if the counter variable CT exceeds the maximum number of searches LIMIT, the correction parameters γ1 to γN stored in the memory unit 12 are determined as the final correction parameters γ1 to γN. In this way, even if the value of the maximum absolute difference value MD oscillates or diverges, the parameter correction unit 112 can adopt the correction parameters γ1 to γN that resulted in the highest average accuracy.

[0075] Depending on the input data (training data and test data) provided to the classifier 30, there may be a trade-off between the high symmetry of the confusion matrix 50 and the high average accuracy. Therefore, the parameter correction unit 112 may present the user with several candidate correction parameters γ1 to γN, and the user may select the final correction parameters γ1 to γN. To perform such processing, the parameter correction unit 112 stores in the storage unit 12 the correction parameters γ1 to γN when the confusion matrix 50 has high symmetry (the value of the maximum absolute difference value MD is less than or equal to the difference reference value (e.g., 0.1)) and when the average accuracy is high (the average accuracy is greater than or equal to the accuracy reference value (e.g., 90%)) as candidate correction parameters γ1 to γN. The parameter correction unit 112 then presents the confusion matrix 50 and the average accuracy created using these stored correction parameters γ1 to γN to the user via the display unit 14. The parameter correction unit 112 then receives the selection of correction parameters γ1 to γN from the user via the operation unit 13 and determines the user-selected correction parameters γ1 to γN as the final correction parameters γ1 to γN. This ensures that even when there is a trade-off between the high symmetry of the confusion matrix 50 and the high average accuracy, classification can be performed using appropriate correction parameters γ1 to γN.

[0076] In the above embodiment, the parameter correction unit 112 created a confusion matrix 50 and modified the correction parameters γ1 to γN so that the bias in the total number of samples for each class for each correct label in the confusion matrix 50 is reduced among the multiple classes, in other words, to increase the symmetry of the confusion matrix 50. However, the parameter correction unit 112 is not limited to using the symmetry of the confusion matrix 50 as a criterion, and may also modify the correction parameters γ1 to γN based on other criteria. For example, the parameter correction unit 112 may modify the correction parameters γ1 to γN so that the sensitivity (recall rate) of a particular class among the multiple classes reaches a predetermined target value.

[0077] In the above embodiment, the functions of the learning unit 111 were provided in the identification device 100 or the classifier generation device 200. However, the functions of the learning unit 111 may be provided in an external device separate from the identification device 100 or the classifier generation device 200. In this case, the external device generates the classifier 30 by machine learning using the learning dataset 121. The identification device 100 or the classifier generation device 200 acquires the classifier 30 from the external device via the communication units 15 and 25. The parameter correction unit 112 then corrects the correction parameters γ1 to γN in the classifier 30 acquired from the external device through the process described above, and generates a classifier 30 with corrected correction parameters γ1 to γN.

[0078] In the above embodiment, the control units 11 and 21 functioned as the respective parts shown in Figures 1, 8, or 9 by having the CPU execute a program stored in the ROM or storage unit 12. However, the control units 11 and 21 may be dedicated hardware. Dedicated hardware includes, for example, a single circuit, a composite circuit, a programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof. If the control units 11 and 21 are dedicated hardware, each function of each part may be realized by separate hardware, or the functions of each part may be realized together by a single piece of hardware. Furthermore, some of the functions of each part may be realized by dedicated hardware, and other parts by software or firmware. In this way, the control unit 110 can realize the above-mentioned functions by hardware, software, firmware, or a combination thereof.

[0079] It is also possible to make an existing computer, such as a personal computer or cloud server, function as the identification device 100 or identification device 200 by applying the program that defines the operation of the identification device 100 or identification device 200 described above to the computer. Furthermore, the distribution method of such a program is arbitrary; for example, it may be distributed by storing it on a computer-readable recording medium such as a CD-ROM (Compact Disk ROM), DVD (Digital Versatile Disk), MO (Magneto Optical Disk), or memory card, or it may be distributed via a communication network such as the Internet.

[0080] Although preferred embodiments of the present invention have been described above, the present invention is not limited to the embodiments described above, and various modifications and substitutions can be made to the embodiments described above without departing from the scope of the claims. [Explanation of symbols]

[0081] 11...Control unit, 30...Identifier, 50...Confusion matrix, 112...Parameter correction unit, 115...Identification processing unit, 100...Identification device

Claims

1. An identification processing unit identifies the class to which the input data belongs, using a classifier that obtains a scale indicating the likelihood that the input data belongs to each of several classes, and outputs a predicted value based on a scale corrected by a correction parameter that corrects the scale indicating the likelihood that the input data belongs to each of several classes, An identification device characterized by comprising:

2. The aforementioned discriminator is A first acquisition unit that, in response to the input of the aforementioned input data, acquires logit values ​​that indicate the likelihood that the input data belongs to each of the plurality of classes, A second acquisition unit acquires the predicted value after correcting the logit value acquired by the first acquisition unit using the correction parameter, The identification device according to claim 1, characterized by comprising:

3. The second acquisition unit calculates a value obtained by linearly transforming the logit value acquired by the first acquisition unit using the correction parameter, and calculates the predicted value from the calculated value. The identification device according to feature 2.

4. The system further includes a parameter correction unit that, when test data with correct labels is input to the classifier, creates a confusion matrix based on the predicted values ​​output from the classifier, and modifies the correction parameters in the created confusion matrix so that the bias in the total number of samples for each class for each correct label of the classifier is reduced among the multiple classes, The identification processing unit uses the classifier, whose correction parameters have been modified by the parameter modification unit, to identify the class to which the input data belongs. The identification device according to any one of claims 1 to 3.

5. The parameter correction unit modifies the correction parameters such that, when the confusion matrix is ​​normalized by the number of correct data points for each class in the confusion matrix, the sum of the prediction rates for each class in the normalized confusion matrix approaches a reference value. The identification device according to feature 4.

6. The parameter modification unit modifies the correction parameter to the parameter that maximizes the average accuracy rate obtained by averaging the accuracy rates of each class, among a plurality of parameters that reduce the bias in the total number of samples for each class for each correct label of the classifier in the confusion matrix among the plurality of classes. The identification device according to feature 4.

7. A classifier generating device that obtains a scale indicating the likelihood that input data belongs to each of several classes, and outputs a predicted value based on the scale corrected by a correction parameter, A parameter correction unit creates a confusion matrix based on the predicted values ​​output from the classifier when test data with correct labels is input to the classifier, and modifies the correction parameters in the created confusion matrix so that the bias in the total number of samples for each class for each correct label of the classifier is small among the multiple classes. A discriminator generating device characterized by comprising the following features.

8. A method for generating a classifier that obtains a scale indicating the likelihood that input data belongs to each of several classes, and outputs a predicted value based on the scale corrected by a correction parameter, When test data with correct labels is input to the classifier, a confusion matrix is ​​created based on the predicted values ​​output from the classifier, and the correction parameters are modified in the created confusion matrix so that the bias in the total number of samples for each class for each correct label of the classifier is small among the multiple classes. A method for generating a classifier characterized by the following features.

9. A program that generates a classifier which obtains a scale indicating the likelihood that input data belongs to each of several classes, and outputs a predicted value based on the scale corrected by a correction parameter, Computers, A parameter correction unit creates a confusion matrix based on the predicted values ​​output from the classifier when test data with correct labels is input to the classifier, and modifies the correction parameters in the created confusion matrix so that the bias in the total number of samples for each class for each correct label of the classifier is small among the multiple classes. A program designed to function as such.

Images